docs: spec organizations retirement; add email match-resolution detail
- ROADMAP: new "Retire organizations" cleanup — re-home email domain matching onto fundraising_investors (derive from contacts' email domains via a review-and-approve backfill), stop auto-creating investor-name-clone orgs, then retire the table. Third instance of the grid-is-canonical theme. - ROADMAP: email-unification bullet gains the match-resolution detail (investor_id required + contact_id nullable from the link's resolved person; org/domain-only and classic-contact-only cases) + proposal needs a contact field. - AGENTS.md: Current state notes the related organizations cleanup.
This commit is contained in:
@@ -111,7 +111,7 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
|
|||||||
_**Box live at v0.1.0:106 (deployed + verified 2026-06-21)** — clean StartOS migration chain (…→106), server up on :8080. **The fundraising grid + email capture is the canonical system of record.** History: git log + `start9/0.4/startos/versions/`._
|
_**Box live at v0.1.0:106 (deployed + verified 2026-06-21)** — clean StartOS migration chain (…→106), server up on :8080. **The fundraising grid + email capture is the canonical system of record.** History: git log + `start9/0.4/startos/versions/`._
|
||||||
|
|
||||||
- **Shipped (v0.1.0:106):** **retired `contacts.contact_type` (logical).** Desktop Contacts lost the Investors/Prospects tabs + TYPE badge → a grid-derived **Status** (existing-LP badge + pipeline-stage chip via `contact_grid_signals`); dashboard `total_lps`/`total_prospects` now count grid investor entities (committed>0 vs $0, graveyard + 'Untitled Investor' excluded) + fixed a `total_contacts` soft-delete leak. Column left physically inert; physical DROP deferred to a signed-off table-rebuild migration. **45/45** + new dashboard assertions, render-smoke green, reviewer APPROVE (no blockers).
|
- **Shipped (v0.1.0:106):** **retired `contacts.contact_type` (logical).** Desktop Contacts lost the Investors/Prospects tabs + TYPE badge → a grid-derived **Status** (existing-LP badge + pipeline-stage chip via `contact_grid_signals`); dashboard `total_lps`/`total_prospects` now count grid investor entities (committed>0 vs $0, graveyard + 'Untitled Investor' excluded) + fixed a `total_contacts` soft-delete leak. Column left physically inert; physical DROP deferred to a signed-off table-rebuild migration. **45/45** + new dashboard assertions, render-smoke green, reviewer APPROVE (no blockers).
|
||||||
- **New SPEC in ROADMAP (Grant 2026-06-21): retire the notes blob → unify ALL activity into `communications`.** Traced in code: the blob (`fundraising_investors.notes`) and `communications` are dual-written by `log-communication`; they drift, leak soft-deletes into the grid + grounding corpus, and **emails land in the blob, never in `communications`** (so the timeline misses emails). Plan: rebuild the leaf `communications` table (add `fundraising_investor_id` NOT NULL, relax `contact_id` nullable, drop `duration_minutes`/`attendees`/`outcome`/`opportunity_id`), unify emails into comms, simplify the log form (+ `next_action`→auto-reminder, backdatable date, desktop==mobile), make the grid Notes column a derived view + Log button, retrofit blobs→comms, then DELETE the blob. Multi-session; see the ROADMAP spec.
|
- **New SPEC in ROADMAP (Grant 2026-06-21): retire the notes blob → unify ALL activity into `communications`.** Traced in code: the blob (`fundraising_investors.notes`) and `communications` are dual-written by `log-communication`; they drift, leak soft-deletes into the grid + grounding corpus, and **emails land in the blob, never in `communications`** (so the timeline misses emails). Plan: rebuild the leaf `communications` table (add `fundraising_investor_id` NOT NULL, relax `contact_id` nullable, drop `duration_minutes`/`attendees`/`outcome`/`opportunity_id`), unify emails into comms, simplify the log form (+ `next_action`→auto-reminder, backdatable date, desktop==mobile), make the grid Notes column a derived view + Log button, retrofit blobs→comms, then DELETE the blob. Multi-session; see the ROADMAP spec. **Related cleanup also specced: retire `organizations`** (vestigial investor-name clones; re-home email domain matching onto `fundraising_investors`, derived from contacts' email domains) — third instance of the grid-is-canonical theme, sequence with the consolidation.
|
||||||
- **Bug A — Grant is handling:** `odell/marty/finance/ten31@` can't enroll for email capture ("could not resolve user_id") because the enroll flow requires a CRM `users` row; Grant is creating user accounts for those mailboxes.
|
- **Bug A — Grant is handling:** `odell/marty/finance/ten31@` can't enroll for email capture ("could not resolve user_id") because the enroll flow requires a CRM `users` row; Grant is creating user accounts for those mailboxes.
|
||||||
- **Next:** (A) begin the **notes-blob → communications** work — start by **extending the contacts census** (count investors with notes but zero contacts; pure-structured vs legacy free-text blobs) to size the retrofit + the contactless gap; (B) **contacts ↔ `fundraising_contacts` consolidation** (path a — every investor ≥1 contact) + **DELETE the TEMPORARY census endpoint/handler/route/button** once A/B/C captured; (C) the deferred `contact_type` physical DROP can ride the `communications` rebuild; (D) confirm the two stuck mailboxes + Grant's 4 new mailbox users enroll; (E) carried: bell approve-on-phone → Matrix-thread-clears spot-check.
|
- **Next:** (A) begin the **notes-blob → communications** work — start by **extending the contacts census** (count investors with notes but zero contacts; pure-structured vs legacy free-text blobs) to size the retrofit + the contactless gap; (B) **contacts ↔ `fundraising_contacts` consolidation** (path a — every investor ≥1 contact) + **DELETE the TEMPORARY census endpoint/handler/route/button** once A/B/C captured; (C) the deferred `contact_type` physical DROP can ride the `communications` rebuild; (D) confirm the two stuck mailboxes + Grant's 4 new mailbox users enroll; (E) carried: bell approve-on-phone → Matrix-thread-clears spot-check.
|
||||||
- **Open / risks:** v106 desktop Contacts Status + dashboard counts are **live-smoke / not yet device-confirmed**. Carried: **Claude/Architect path unverified live on the box**; vision OCR small-in-frame misread (`mara.com→marac.com`); doc drift — `crm-overview.md` narrative + `EVALUATION.md` still describe `lp_profiles` (active API/schema claims fixed; deeper Phase-0 narrative deferred to a doc pass).
|
- **Open / risks:** v106 desktop Contacts Status + dashboard counts are **live-smoke / not yet device-confirmed**. Carried: **Claude/Architect path unverified live on the box**; vision OCR small-in-frame misread (`mara.com→marac.com`); doc drift — `crm-overview.md` narrative + `EVALUATION.md` still describe `lp_profiles` (active API/schema claims fixed; deeper Phase-0 narrative deferred to a doc pass).
|
||||||
|
|||||||
+4
-1
@@ -88,6 +88,8 @@
|
|||||||
|
|
||||||
- **Consolidate `contacts` ↔ `fundraising_contacts` into one linked model.** Goal (Grant): everyone in `contacts` maps to a `fundraising_investors` row (an individual maps to their own row). Today `contacts` is the canonical person directory (FK target for `communications`/`opportunities`); `fundraising_contacts.contact_id` (migration `0004`) points INTO it; the mobile Contacts page reads `contacts`. Three populations: **A** linked (grid pill ↔ contact), **B** `contacts`-only (imported prospects / manual adds — need a grid row), **C** pill-only (`fundraising_contacts.contact_id IS NULL` — need a contact row). **Census-first:** before designing any migration, count A/B/C on the box — Grant runs the SQL himself (he is **not** providing a DB copy), so hand him a counts-only script. The census decides whether this is a ~20-row cleanup or a ~300-row structural migration with `communications`/`opportunities` repointing. Then Grant reconciles B (add grid rows/pills) and C (add contact rows) and ensures all are linked. **(v0.1.0:105) A TEMPORARY admin census ships to read A/B/C off the box without shell access: `GET /api/admin/contacts-census` (`handle_contacts_census`) + a Settings → Admin "Run census" button, mirroring `backend/scripts/contacts_census.sql` (counts only). DELETE the endpoint + route + button after the numbers are captured — all tagged `TEMPORARY` in code.**
|
- **Consolidate `contacts` ↔ `fundraising_contacts` into one linked model.** Goal (Grant): everyone in `contacts` maps to a `fundraising_investors` row (an individual maps to their own row). Today `contacts` is the canonical person directory (FK target for `communications`/`opportunities`); `fundraising_contacts.contact_id` (migration `0004`) points INTO it; the mobile Contacts page reads `contacts`. Three populations: **A** linked (grid pill ↔ contact), **B** `contacts`-only (imported prospects / manual adds — need a grid row), **C** pill-only (`fundraising_contacts.contact_id IS NULL` — need a contact row). **Census-first:** before designing any migration, count A/B/C on the box — Grant runs the SQL himself (he is **not** providing a DB copy), so hand him a counts-only script. The census decides whether this is a ~20-row cleanup or a ~300-row structural migration with `communications`/`opportunities` repointing. Then Grant reconciles B (add grid rows/pills) and C (add contact rows) and ensures all are linked. **(v0.1.0:105) A TEMPORARY admin census ships to read A/B/C off the box without shell access: `GET /api/admin/contacts-census` (`handle_contacts_census`) + a Settings → Admin "Run census" button, mirroring `backend/scripts/contacts_census.sql` (counts only). DELETE the endpoint + route + button after the numbers are captured — all tagged `TEMPORARY` in code.**
|
||||||
|
|
||||||
|
- **Retire `organizations` — re-home email domain matching onto the investor, then drop the table (Grant 2026-06-21).** Traced in code: organizations are largely vestigial. `_upsert_contact_from_fundraising` auto-creates an org **named after the investor** (`server.py:805`) for every grid investor — individuals included ("John Smith" the org) — with no website/email, so they're pure name-clones. There is **no Organizations page/nav** (a full CRUD API exists but is unsurfaced); the only visible role is the "Organization" display label on Contacts + opportunities (`organization_name`), which duplicates the investor. FK'd by `contacts.organization_id` + `opportunities.organization_id` (both `ON DELETE SET NULL`) + a soft `email_investor_links.organization_id`. **The one genuinely non-redundant function is email DOMAIN matching** (`matcher.py:110-122` indexes `organizations.email`/`website`, so `@acmecapital.com` resolves to a known firm when the sender isn't a saved contact) — but only for orgs that carry a website/email, which the auto-created clones do **not**. Plan: (1) **re-home the domain signal onto `fundraising_investors`** (add a `website`/`domain` field) and domain-index investors instead of orgs — the domain is **derivable from the investor's contacts' email domains**, so a one-time **backfill script with human review/approval** likely suffices (LLM-in-the-middle via Spark Control optional, probably unnecessary; an investor whose contacts span several domains is resolved case-by-case at approval); (2) **stop auto-creating name-clone orgs** in `_upsert_contact_from_fundraising`; (3) drop the "Organization" display in favor of the investor linkage (FKs `SET NULL` cleanly); (4) retire the `organizations` table — logical first, then a signed-off drop (same retire-in-place pattern as `contact_type`/`lp_profiles`). **Census check first:** how many orgs carry a real website/email (live matching coverage) vs. auto-created name-clones — sizes whether step 1 replaces a live capability or a near-dead one. **Third instance of the grid-is-canonical theme** (alongside the consolidation + the notes-blob→communications work); sequence together.
|
||||||
|
|
||||||
### Retire the notes blob → unify ALL activity into `communications` (SPEC, Grant 2026-06-21)
|
### Retire the notes blob → unify ALL activity into `communications` (SPEC, Grant 2026-06-21)
|
||||||
*A multi-session structural change. **End goal:** `communications` is the single source of truth for every touchpoint with an investor; the grid's free-text "Notes / Communication / Outreach" blob is **deleted entirely** (no archive — but only after the retrofit is verified). Grounding for the decision below was traced in code 2026-06-21.*
|
*A multi-session structural change. **End goal:** `communications` is the single source of truth for every touchpoint with an investor; the grid's free-text "Notes / Communication / Outreach" blob is **deleted entirely** (no archive — but only after the retrofit is verified). Grounding for the decision below was traced in code 2026-06-21.*
|
||||||
|
|
||||||
@@ -106,7 +108,8 @@
|
|||||||
|
|
||||||
**`log-communication` field set (simplified, desktop == mobile — Grant).** Types: **`email/call/meeting/note`** (drop `text` — redundant with `note`, Grant). Fields: type, **communication date (defaults today, backdatable)**, body (free text — fold the old `outcome`/attendees prose in here), and **`next_action` + `next_action_date`** which, when set, **auto-create a reminder** (the W1 `reminders` table) on the investor. Remove the duration/attendees/outcome/opportunity inputs. **Same fields on desktop and mobile** (one shared form).
|
**`log-communication` field set (simplified, desktop == mobile — Grant).** Types: **`email/call/meeting/note`** (drop `text` — redundant with `note`, Grant). Fields: type, **communication date (defaults today, backdatable)**, body (free text — fold the old `outcome`/attendees prose in here), and **`next_action` + `next_action_date`** which, when set, **auto-create a reminder** (the W1 `reminders` table) on the investor. Remove the duration/attendees/outcome/opportunity inputs. **Same fields on desktop and mobile** (one shared form).
|
||||||
|
|
||||||
**Email → `communications` unification.** Repoint email-proposal approval (`decide_email_activity_proposal`) to create an **`email`-type communication** (investor_id from `email_investor_links.fundraising_investor_id`; contact_id when the person is a known contact, else null) instead of appending to the blob. Full email body stays in the `emails` table; the communication is the touchpoint record + links back. Then the timeline shows emails, and consumers stop needing a parallel email path.
|
**Email → `communications` unification.** Repoint email-proposal approval (`decide_email_activity_proposal`) to create an **`email`-type communication** instead of appending to the blob. Full email body stays in the `emails` table; the communication is the touchpoint record + links back. Then the timeline shows emails, and consumers stop needing a parallel email path.
|
||||||
|
- **Match resolution → comm fields.** The matcher (`email_integration/matcher.py`) writes an `email_investor_links` row resolved at up to three levels: `fundraising_investor_id` + `fundraising_contact_id` (a grid pill, via `exact_email`), `contact_id` (a classic contact), or `organization_id` **only** (`domain_match`, suppressed whenever any exact hit exists). On approval, set `communications.fundraising_investor_id` (required) from the link's investor and `communications.contact_id` (nullable) from the link's resolved person (pill → `fundraising_contacts.contact_id` → classic `contact_id`, else null). **Two cases legitimately have NO person** — a domain/org-level match (the sender's domain matches a known org but that person isn't a contact) and an investor-alias address — which is precisely why `contact_id` is nullable. A classic-contact-only match resolves a contact but **no investor until that contact maps to a grid investor** (the path-a consolidation closes this) — so email-unification and the consolidation are coupled. **The proposal row needs a resolved-contact field:** `email_activity_proposals` carries only `investor_id` today (it just appends to the blob), so add the resolved `contact_id` so approval can populate the communication.
|
||||||
|
|
||||||
**The grid column → derived view + Log button.** "Notes / Communication / Outreach" becomes a **read-only derived view** (render the latest N communications for the row inline), with a quick **Log button** (reuse the existing log-communication modal) for scratch entries — replacing inline free-text editing. The detail timeline already reads from `communications`.
|
**The grid column → derived view + Log button.** "Notes / Communication / Outreach" becomes a **read-only derived view** (render the latest N communications for the row inline), with a quick **Log button** (reuse the existing log-communication modal) for scratch entries — replacing inline free-text editing. The detail timeline already reads from `communications`.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user