diff --git a/AGENTS.md b/AGENTS.md index 30cc9fd..e619bb4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -70,7 +70,7 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude ## Conventions -- **Investor model — the grid is canonical (since v0.1.0:78).** The `fundraising_*` grid is the **system of record**: an investor entity (row) → many contact "pills" → per-fund commitments. The classic `contacts` table is a **read-only per-person directory**, auto-populated from the grid — create/edit people in the grid, not the Contacts page. Email capture rolls multiple people up to one investor. The legacy single-fund `lp_profiles` model is **retired** (empty table kept, per never-hard-delete). Reconciling grid ↔ classic `contacts` to canonical IDs is the core entity-resolution task — see `docs/crm-overview.md`. **Derived read-only columns** (`pipeline`, `pipeline_stage`, `opportunity_id`, `reminder_status`, `existing_investor`, `last_activity_at`, `staleness`) are computed live and **injected on GET, never persisted** — any new one MUST be added to BOTH strip points (`server.py` `_computed_row_values` + frontend `stripComputedRows`) or it dirties the autosave / leaks into the blob. Pipeline stage is the 4-stage funnel `lead→engaged→diligence→commitment` (`PIPELINE_STAGES`), terminal at commitment. +- **Investor model — the grid is canonical (since v0.1.0:78).** The `fundraising_*` grid is the **system of record**: an investor entity (row) → many contact "pills" → per-fund commitments. The classic `contacts` table is a **read-only per-person directory**, auto-populated from the grid — create/edit people in the grid, not the Contacts page. Email capture rolls multiple people up to one investor. The legacy single-fund `lp_profiles` model is **retired** (empty table kept, per never-hard-delete). Reconciling grid ↔ classic `contacts` to canonical IDs is the core entity-resolution task — see `docs/crm-overview.md`. **Derived read-only columns** (`pipeline`, `pipeline_stage`, `opportunity_id`, `reminder_status`, `existing_investor`, `last_activity_at`, `staleness`) are computed live and **injected on GET, never persisted** — any new one MUST be added to BOTH strip points (`server.py` `_computed_row_values` + frontend `stripComputedRows`) or it dirties the autosave / leaks into the blob. **Exception — the contact-pill email-heal** (`fundraising_contact_emails_by_row`, injected in `handle_get_fundraising_state`, v0.1.0:99): it fills a *blank* pill `email` from the linked classic contact and deliberately has **NO** strip point, because `email` is a real blob field, not a computed column — the next one-row save legitimately persists the recovered value (it's a self-healing backfill; don't "fix" it by adding a strip point). Pipeline stage is the 4-stage funnel `lead→engaged→diligence→commitment` (`PIPELINE_STAGES`), terminal at commitment. - **Soft-delete only:** `deleted_at` and/or `status='retired'`; never hard-delete. Every READ path must filter `deleted_at IS NULL` — list handlers, get-by-id, nested related-data sub-selects, **and aggregate sub-selects (`COUNT`/`SUM`/`MAX`)**. Audits found leaks in all of these (2026-06-12 detail + nested; 2026-06-13 list-view `contact_count`/`total_funded`/`comm_count`); the **opportunities/pipeline** aggregates were fixed in v0.1.0:87 (`handle_pipeline_report` + dashboard pipeline metrics now filter `deleted_at`), but the **reports** subsystem's **communications-side** aggregates (dashboard `recent_comms`/`comms_this_month`/`meetings_this_month`, activity report) still leak (see Current state). Regression-guarded by `backend/test_soft_delete_reads.py` (+ `test_reminders.py` for the reminders read paths, incl. the recency rollup whose email-activity liveness signal is `email_account_messages.deleted_at`, not `emails`). (Thesis has a subtlety here — see the thesis guide.) - **Env:** secrets in `.env` (gitignored); names in `.env.example`. Verified names: `ANTHROPIC_API_KEY`, `SPARK_CONTROL_URL`, `SPARK_CONTROL_VERIFY_TLS`, `QDRANT_URL`, `X_API_KEY`, `CRM_DB_PATH`, `CRM_DEV_DB_PATH`. Also used: `CRM_SECRET_KEY` (beta/prod), `CRM_HOST`/`CRM_PORT`, `CRM_DATA_DIR`; digest mailer: `CRM_DIGEST_SENDER` (DWD impersonation sender) + `SMTP_HOST`/`SMTP_PORT`/`SMTP_SECURITY`/`SMTP_FROM`/`SMTP_USERNAME`/`SMTP_PASSWORD` (SMTP fallback); daily digest (Phase B): `CRM_DIGEST_ENABLED` + `CRM_DIGEST_SEND_HOUR` **only seed the first-boot default** — the live control is the DB policy (`app_settings.digest_policy`, set in Settings → Admin). - **Config placement:** operational/feature toggles live in the **admin panel**, DB-backed via `app_settings` (read-merge through a `load_*_policy(conn)` helper shared by the API + any scheduler; precedence DB-row → env-seed → default), so they're discoverable and take effect live. Reserve StartOS actions / env for **secrets and deploy-time config** (SMTP creds, API keys, DWD sender). Precedent: `digest_policy` (`GET/PATCH /api/admin/digest/policy`), `fundraising_backup_policy`. @@ -111,9 +111,9 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude _**Box live at v0.1.0:99 (deployed + verified 2026-06-20)**. This session = **Grant's real-phone device-test round 2**: four in-app fixes (CRM half → v99 s9pk, **installed & booted clean**) + a Matrix intake-bot cleanup (**redeployed on the Spark; intake room purged to 0, bot confirmed to hold room mod-power**) + a written plan for in-app camera card intake. **The fundraising grid + email capture is the canonical system of record.** History: git log + `start9/0.4/startos/versions/`._ - **Device-test round 2 — 4 fixes shipped to v0.1.0:99 (CRM half).** (1) **Intake fuzzy match** no longer over-indexes on generic firm words — `_name_similarity` scores **distinctive** tokens only (generic descriptors like "Investment Group"/"Capital"/"Family Office" stripped via `_GENERIC_ORG_WORDS`); "Fortitude Investment Group" no longer surfaces Aether/Russell. (2) **Mobile grid "Last contact"/staleness sort is reversible** (`SortSheet` opt-in `dir`/`onToggleDir`; other surfaces untouched). (3) **Mobile "Edit investor" prefills a contact's email** — `GET /api/fundraising/state` heals a blank grid pill email from `fundraising_contacts.contact_id → contacts.email` (fill-only, by pill order then name; next one-row save persists it; `fundraising_contact_emails_by_row`). (4) **Quick-log pencil icon renders** — `.quicklog-btn svg { width;height;flex:none }` (iOS collapses a sole, centered, attribute-only-sized flex-child svg; the v97 fix only changed its color). -- **Matrix intake-bot — thread auto-delete on decision + retroactive purge (Spark, bot-only, NOT in the s9pk).** Approve/reject now `redact_thread(intake_room, root)` (clears card + ack + main-timeline nudge + the user's photo/note), mirroring the email-review room; the scan now also catches the un-threaded nudge (`m.in_reply_to`). New one-time `backend/matrix_intake/redact_intake.py` (dry-run default; `--apply`) clears the room backlog. **Needs the bot to hold a redact/moderator power level in the intake room** to clear users' messages (manual Element step). No more in-Matrix "✅ logged" confirmation after a commit (by design, like email). Detail: `docs/guides/matrix-intake.md`. -- **In-app camera card intake (#7) — PLAN written, not built.** `docs/handoffs/in-app-card-intake-plan.md`: reuses the nio-free transcribe/parse core (`server.py` already imports `llm`; `matrix_intake/parse.py`+`spark.py` are nio-free) → **one endpoint** `POST /api/intake/card` + **one mobile component** (camera button left of the pencil). No bot refactor, no new dep, no migration. **Awaiting Grant's call** on 4 decisions (provenance tag, form-edits-only v1, member access, ships-in-s9pk). +- **Matrix intake-bot — thread auto-delete on decision + retroactive purge — DONE & LIVE (Spark, bot-only, NOT in the s9pk).** Approve/reject now `redact_thread(intake_room, root)` (clears card + ack + main-timeline nudge + the user's photo/note), mirroring the email-review room; the scan matches `rel_type=m.thread` children OR the bot's own `m.in_reply_to` nudge (narrowed in the reviewer pass so it can't catch an unrelated human reply to the root). One-time `backend/matrix_intake/redact_intake.py` (dry-run default; `--apply`) cleared the backlog to 0. **Bot already holds room mod-power** (confirmed live — user messages + a card image redacted, 0 failures). No more in-Matrix "✅ logged" confirmation after a commit (by design, like email). Detail: `docs/guides/matrix-intake.md` (compose service is `intake`; redaction is rate-limited, re-run `--apply` till a dry-run reports 0). +- **In-app camera card intake (#7) — PLAN + decisions LOCKED, not built yet.** `docs/handoffs/in-app-card-intake-plan.md`: reuses the nio-free transcribe/parse core (`server.py` already imports `llm`; `matrix_intake/parse.py`+`spark.py` are nio-free) → **one endpoint** `POST /api/intake/card` + **one mobile component** (camera button left of the pencil). No bot refactor, no new dep, no migration. Decisions: `source="app_card"`, **form-field edits only** (no NL-edit), **any authenticated member**, ships in the s9pk. **Ready to build on Grant's go-ahead.** - **Mobile-first redesign — deployed (v95–v97), on-device test in progress.** 4 surfaces (Grid·Contacts·Pipeline·Reminders) + light theme + installable PWA + 4-stage funnel; desktop untouched. Standing gate: light/dark across surfaces + login (375px gutters, safe-area), install→standalone launch, swipe/sheet interactions (only jsdom-smoked). Other live features: W2 NL query (v94), W1 reminders (v93), grid Pipeline (v88), Gmail capture + daily digest, Thesis/Architect (dual-approval), outreach — all draft-only. **Business-card intake (M3, Matrix bot) LIVE since v98** (vision OCR via the daily-driver model; `source="matrix_card"`; captures name/email/title/city/LinkedIn/phone/mobile, integrity-checked). -- **Tests: 42/42 backend green** (`python3 backend/run_tests.py`), `py_compile` clean, frontend render-smoke green (`make render-smoke`). New: `test_grid_email_heal.py` + intake generic-word cases. Vision/OCR (Matrix + the planned in-app path) is **live-smoke only**. -- **Next:** (1) Grant device-tests the 4 v99 fixes on his phone (esp. the **pencil icon** — root-caused + render-smoke-green but device-confirm) + re-tests cards (OCR/phone mapping); (2) Grant's call on the #7 plan (4 decisions); (3) finish the standing mobile on-device gate. (Intake auto-delete + retroactive purge are **done & live** — bot already held room mod-power, room cleared to 0 over two rate-limited passes.) +- **Tests: 42/42 backend green** (`python3 backend/run_tests.py`), `py_compile` clean, frontend render-smoke green (`make render-smoke`). New `test_grid_email_heal.py` + intake generic-word/all-generic-edge cases. A **reviewer-agent pass** of the whole session diff returned APPROVE (no blockers); its one substantive finding (the redact predicate) is fixed above. Vision/OCR (Matrix + the planned in-app path) is **live-smoke only**. +- **Next:** (1) Grant device-tests the 4 v99 fixes on his phone (esp. the **pencil icon** — root-caused + render-smoke-green but device-confirm) + re-tests cards (OCR/phone mapping); (2) **build #7 (in-app camera card intake) on go-ahead** — decisions locked, spec ready; (3) finish the standing mobile on-device gate (light/dark + install + swipe/sheets). - **Open / risks:** **vision OCR can misread a character** on a card small-in-frame (resolution-bound — Spark Control downscales to ~2 MP; `mara.com→marac.com` reproduced at temp 0; mitigations: fill the frame, or a future client-side crop); **iPhone HEIC** may not decode in vLLM (most clients send JPEG); phone/mobile/LinkedIn land on the **contact record**, not the grid pill (by design — only city syncs to the pill); intake redaction needs the bot's room **mod power** or users' messages linger; **Claude/Architect path unverified live on the box**; v2.0 reserve-asset spine **not canonical**; **PWA iOS status bar fixed `black`** in light theme (header seam; deferred, dark is default); doc drift — `crm-overview.md`/`EVALUATION.md` still call `lp_profiles` live; assorted minor UI cleanups (`.login-title` dead CSS, `MobileDetailRow` unused, Pipeline "Committed" tile shows grid-committed) tracked in git history. diff --git a/ROADMAP.md b/ROADMAP.md index 7d28135..34300b9 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -130,7 +130,7 @@ Use the **matrix-bridge** repo's pattern to listen on a dedicated ten31-database **Long-term — extract the intake bot to its own repo (recommended, not yet done).** Containerizing from this monorepo is the pragmatic now-state, but the bot is a genuinely separate deployable (own process, own `matrix-nio` dep, own lifecycle); its only CRM coupling is the HTTP API (a clean network contract) plus ~40 lines of stdlib Spark client (cheap to vendor). The tell: the spark-control Update button would run `git reset --hard origin/main` on the **whole CRM clone** — wrong blast radius. `matrix-bridge` is already a dedicated repo; mirror it. The extraction is a migration (new Gitea repo, move code + tests + guide, vendor the client, re-point the Spark deploy), so it's deferred until worth the lift — do it *before* wiring the spark-control card if both land in the same push. ### In-app camera business-card intake — PLAN written, awaiting go-ahead (Grant, 2026-06-20) -*A camera button in the mobile top bar (left of the quick-log pencil) → take/choose a photo → the same vision-transcribe → parse → fuzzy-match → edit/approve/reject flow the Matrix card intake (M3) runs, surfaced as an inline mobile sheet. **Detailed plan: `docs/handoffs/in-app-card-intake-plan.md`.*** Key finding: the reusable core is nio-free and already reachable from the CRM (`server.py` imports `llm`; `matrix_intake/parse.py`+`spark.py` import no `matrix-nio`), so it's **one endpoint** (`POST /api/intake/card`) + **one mobile component**, no bot refactor / new dep / migration. Four decisions pending Grant before building: provenance tag (`app_card`?), form-edits-only for v1 (no conversational NL-edit), any-member access, ships-in-s9pk. Reuses the New-investor sheet pre-filled + the proven `.quicklog-btn svg` icon-sizing fix for the camera button. +*A camera button in the mobile top bar (left of the quick-log pencil) → take/choose a photo → the same vision-transcribe → parse → fuzzy-match → edit/approve/reject flow the Matrix card intake (M3) runs, surfaced as an inline mobile sheet. **Detailed plan: `docs/handoffs/in-app-card-intake-plan.md`.*** Key finding: the reusable core is nio-free and already reachable from the CRM (`server.py` imports `llm`; `matrix_intake/parse.py`+`spark.py` import no `matrix-nio`), so it's **one endpoint** (`POST /api/intake/card`) + **one mobile component**, no bot refactor / new dep / migration. Decisions LOCKED (Grant 2026-06-20): `source="app_card"`, form-field edits only (no NL-edit) for v1, any-authenticated-member access, ships in the s9pk. Ready to build on go-ahead. Reuses the New-investor sheet pre-filled + the proven `.quicklog-btn svg` icon-sizing fix for the camera button. ### Scoped service-credential auth path for automated CRM writers *Surfaced 2026-06-17 while deploying the Matrix intake bot. **Decision: defer — the bot uses a dedicated member username/password for now.** The CRM has no API-key/service-token path; its only auth is username+password → JWT. A dedicated **member** login is appropriately scoped against what matters operationally (no admin: can't manage users, reset data, or change settings) and unblocks the live smoke today.*