Structural fix for the duplicate-people class of bug: instead of matching a grid
contact "pill" to a contacts row heuristically by name/email (which drifted and
caused the 1406 double-count), link them by id.
Backend:
- Migration 0004: fundraising_contacts.contact_id (additive, nullable, logical FK
to contacts(id)) + index. Paired down migration.
- sync_fundraising_relational now stores the id that _upsert_contact_from_fundraising
already returns, so every grid contact carries its contacts-table id.
- _backfill_grid_contact_ids: one-time, idempotent backfill on startup (re-runs the
grid sync once if any row lacks contact_id), so existing data links immediately.
- entity_resolution: grid pass prefers the explicit contact_id link (match_kind
'grid_link') over heuristic email / name+investor, guarded by a PRAGMA check so
older DBs without the column still work.
Frontend:
- Fundraising grid "+ Row" -> "+ Investor" (clear, single investor entry point).
- Contacts page: the "+ Add Contact" trigger is replaced by a pointer to the grid;
the page is now a read/search/edit view (ContactDetailPanel still edits all
fields). New people are added from the grid. No contact data is removed.
Tests: backend/ingest/test_entity_resolution.py extended (explicit-link case, 11/11)
and a new backend/test_grid_contact_link.py integration test (init_db applies 0004,
sync populates contact_id to the right contact, re-sync is idempotent). py_compile +
frontend html.parser clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause: grid contacts (fundraising_contacts) are the SAME people as the
contacts table (the app syncs them by name/email), but resolution matched grid
rows by (name + investor-canon) where the two sides derive the investor key from
different tables that rarely line up — so nearly every grid contact minted a
duplicate person (715 + ~692 ≈ 1406), and the duplicate finder then flagged each
twin against its real self (~676 candidates).
Fix (entity_resolution.py):
- Grid pass matches a grid contact to its existing contacts-table person by
PROVABLE keys only (exact email, else exact name within the same investor) and
records membership; on a miss it MINTS NOTHING (the old else-branch mint was the
double-count source, and guessing by name across firms risks binding two
different same-named people).
- Targeted, audited cleanup soft-deletes leftover grid-only "twins" (person rows
with no 'contacts' link) and superseded pre-:48 'lp'/'organization' rows, guarded
so any row carrying enrichment/human data is never dropped (guardrail #3); the
tombstoned ids are logged to interaction_log (guardrail #5).
- _upsert_entity clears deleted_at on conflict so a re-emitted id is un-tombstoned
(no permanent burial); fuzzy-merge losers stay buried via _redirect.
entity_merge.py / server.py: the duplicate queue + pending count now filter to
candidates whose both sides are still live, so self-healed twins drop out.
Verified: offline reproduction test (backend/ingest/test_entity_resolution.py,
10/10) reproduces the 1406-style doubling and proves it collapses; no regression
on the synthetic dev set; two adversarial review passes. Known pre-existing
identity-key weaknesses (same name+firm+no email collision; shared role inbox
over-link) are unchanged by this fix and will be resolved structurally by the
contact_id link in the grid/contacts unification.
Run "Build search index" after upgrading to recompute the canonical layer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per Grant's clarification of the real data model:
- Investor entities come from the fundraising grid, one per row, all labeled
"investor" (drops the confusing lp/organization split). Grid is source of truth.
- People come ONLY from the contacts table. The grid's contacts (fundraising_
contacts) are matched to a contact-person and recorded as member_of links to
their investor, instead of creating duplicate person entities. This fixes the
~doubled people count (people now ≈ contacts, not contacts + grid contacts).
- System Status cards: Investors / People (resolved) / Contacts in CRM / Grid
contacts, so resolved-vs-source is visible at a glance.
Verified on synthetic: people == contacts count (no double-count); multi-contact
investors preserved via member_of.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- entity_resolution: emit member_of relationship edges (contact -> investor),
so one investor entity owns many contacts (institution) and a HNWI is the N=1
case; crm_tools.get_investor_contacts + get_entity contacts/member_of; MCP tool.
- seed_synthetic: multi-contact institutions to exercise it (Harbor & Vine = 5).
- server.py: GET /api/system/status (index/entity/thesis/activity health) for an
in-app status view (no shell needed to verify the index).
- docs/thesis-seed-v1.md: grounded v1 thesis (throughline, 6 pillars, objections,
per-segment angles, voice) drawn from Ten31's newsletter/site/essays.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Fuzzy tier (backend/ingest/fuzzy_resolve.py + llm.py): local Qwen adjudicates
the deterministic resolver's flagged name-variant candidates; merges are
durable via entity_merges (deterministic re-runs respect them), losers
soft-deleted, logged. Idempotent.
- Incremental sync (backend/ingest/sync.py): re-embeds only rows changed since a
watermark (ingest_sync_state); first run / --recreate = full. Tested full→0→1.
- Start9 packaging (start9/0.4): Dockerfile bundles ingest+mcp + fastembed/mcp;
"Build search index" action runs the init in a subcontainer; MCP shipped as a
manual stdio server (not a daemon); version 0.1.0:44. INGEST_PACKAGING.md.
- backfill.py: factored embed_and_upsert() shared with sync.
Verified end-to-end on synthetic data + live Sparks/Qwen/Qdrant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keysat