Phase 0 complete: fuzzy entity tier, incremental sync, Start9 packaging

- Fuzzy tier (backend/ingest/fuzzy_resolve.py + llm.py): local Qwen adjudicates
  the deterministic resolver's flagged name-variant candidates; merges are
  durable via entity_merges (deterministic re-runs respect them), losers
  soft-deleted, logged. Idempotent.
- Incremental sync (backend/ingest/sync.py): re-embeds only rows changed since a
  watermark (ingest_sync_state); first run / --recreate = full. Tested full→0→1.
- Start9 packaging (start9/0.4): Dockerfile bundles ingest+mcp + fastembed/mcp;
  "Build search index" action runs the init in a subcontainer; MCP shipped as a
  manual stdio server (not a daemon); version 0.1.0:44. INGEST_PACKAGING.md.
- backfill.py: factored embed_and_upsert() shared with sync.

Verified end-to-end on synthetic data + live Sparks/Qwen/Qdrant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Keysat
2026-06-05 08:55:12 -05:00
parent c7ce44d963
commit f357c23c75
16 changed files with 808 additions and 48 deletions
+5 -3
View File
@@ -3,12 +3,14 @@
// from manifest/index.ts (id, title) and versions/ (version).
export const PACKAGE_ID = 'ten-database'
export const PACKAGE_TITLE = 'Ten31 Database'
// ExVer form of the current 0.4 wrapper release (upstream 0.1.0, wrapper rev 41).
// ExVer form of the current 0.4 wrapper release (upstream 0.1.0, wrapper rev 44).
// * 0.3.5 wrapper: 0.1.0.38 (legacy, aarch64)
// * First 0.4: 0.1.0:39 (shipped seed snapshot for migration)
// * Cleanup: 0.1.0:40 (seed removed + multi-threaded server + abuser auto-ban)
// * Current: 0.1.0:41 (frontend persists auth across refreshes)
export const PACKAGE_VERSION = '0.1.0:41'
// * 0.1.0:41 (frontend persists auth across refreshes)
// * 0.1.0:42 (Gmail integration) / 0.1.0:43 (Gmail POST-body hotfix)
// * Current: 0.1.0:44 (Phase-0 ingest + MCP server in image; build-index action)
export const PACKAGE_VERSION = '0.1.0:44'
export const DATA_MOUNT_PATH = '/data'
export const WEB_PORT = 8080