diff --git a/AGENTS.md b/AGENTS.md index b1015ff..f061467 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -115,5 +115,5 @@ _**Box live at v0.1.0:105 (deployed + verified 2026-06-20)** — clean StartOS m - **New (v0.1.0:104):** admin-only **Purge Deleted Data** (Settings → Admin) — guarded, type-to-confirm hard-delete of soft-deleted rows; see the soft-delete convention + `test_purge_soft_deleted.py`. - **Verification:** **45/45** backend, render-smoke green, reviewer-agent APPROVE after fixing **1 blocker** (contact purge left a dangling `reminders.contact_id` — now NULLed + test-guarded). New UI behavior is **live-smoke / on-device only** (jsdom can't drive touch). - **Bug A — Grant is handling:** `odell/marty/finance/ten31@` can't enroll for email capture ("could not resolve user_id") because the enroll flow requires a CRM `users` row; Grant is creating user accounts for those mailboxes. -- **Next:** (A) confirm the two stuck mailboxes pulled current + Grant's 4 new mailbox users enroll; (B) **retire `contact_type`** — replace the Contacts Investors/Prospects tabs + TYPE badge with grid-derived `existing_investor`/`pipeline_stage`, then drop the column (see ROADMAP); (C) **contacts ↔ `fundraising_contacts` consolidation** — census-first (count A/linked, B/contacts-only, C/pill-only on the box; see ROADMAP). **A TEMPORARY admin census is LIVE on the box (v0.1.0:105): Settings → Admin → "Run census" (or `GET /api/admin/contacts-census`) reports A/B/C — run it, capture the numbers, then DELETE the endpoint + handler + route + button** (all tagged `TEMPORARY` in code; mirrors `backend/scripts/contacts_census.sql`). (D) carried: bell approve-on-phone → Matrix-thread-clears round-trip spot-check. +- **Next:** (A) **retire `contact_type`** (the next build) — replace the Contacts Investors/Prospects tabs + TYPE badge with grid-derived `existing_investor`/`pipeline_stage`, repoint the dashboard `total_lps`/`total_prospects` counts, then drop the column (live UI change → its own small design pass; see ROADMAP); (B) **contacts ↔ `fundraising_contacts` consolidation** — capture A/B/C from the live census (Settings → Admin → "Run census", or `GET /api/admin/contacts-census`), then **DELETE the TEMPORARY census endpoint + handler + route + button** (all tagged `TEMPORARY`; mirrors `backend/scripts/contacts_census.sql`); (C) confirm the two stuck mailboxes pulled current + Grant's 4 new mailbox users enroll; (D) carried: bell approve-on-phone → Matrix-thread-clears round-trip spot-check. - **Open / risks:** the Contacts pagination, the purge, and the email-sync auto-recovery are **live-smoke / not yet device-confirmed**. Carried: **Claude/Architect path unverified live on the box**; vision OCR small-in-frame misread (`mara.com→marac.com`); doc drift — `crm-overview.md` narrative + `EVALUATION.md` still describe `lp_profiles` (the active API/schema claims were fixed; the deeper Phase-0 narrative is deferred to a doc pass). diff --git a/ROADMAP.md b/ROADMAP.md index 8a96e9a..bb2a7a3 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -88,6 +88,13 @@ - **Consolidate `contacts` ↔ `fundraising_contacts` into one linked model.** Goal (Grant): everyone in `contacts` maps to a `fundraising_investors` row (an individual maps to their own row). Today `contacts` is the canonical person directory (FK target for `communications`/`opportunities`); `fundraising_contacts.contact_id` (migration `0004`) points INTO it; the mobile Contacts page reads `contacts`. Three populations: **A** linked (grid pill ↔ contact), **B** `contacts`-only (imported prospects / manual adds — need a grid row), **C** pill-only (`fundraising_contacts.contact_id IS NULL` — need a contact row). **Census-first:** before designing any migration, count A/B/C on the box — Grant runs the SQL himself (he is **not** providing a DB copy), so hand him a counts-only script. The census decides whether this is a ~20-row cleanup or a ~300-row structural migration with `communications`/`opportunities` repointing. Then Grant reconciles B (add grid rows/pills) and C (add contact rows) and ensures all are linked. **(v0.1.0:105) A TEMPORARY admin census ships to read A/B/C off the box without shell access: `GET /api/admin/contacts-census` (`handle_contacts_census`) + a Settings → Admin "Run census" button, mirroring `backend/scripts/contacts_census.sql` (counts only). DELETE the endpoint + route + button after the numbers are captured — all tagged `TEMPORARY` in code.** +### Captured tweaks (Matrix, 2026-06-18/20) +*Small UI/UX + capture-quality items captured via Matrix; not yet scheduled.* + +- **[P2] Backup history (Settings) defaults minimized, chevron-expand, pinned to the bottom** — it's rarely viewed, so it shouldn't take prime space. Frontend-only (`frontend/index.html`). (2026-06-18) +- **[P2] Preserve the active tab across a page refresh** — a refresh currently snaps back to the top/default tab. Persist the selected tab (e.g. localStorage / URL hash) and rehydrate on load. Frontend-only. (2026-06-18) +- **[P2] Email capture matches an investor on `To:`/`From:` only, not `Cc:`** — today if an investor's address appears anywhere on a message landing in a team mailbox (including when a teammate is merely cc'd on an outbound reply to the investor), it logs a spurious "received from investor" entry. Restrict the investor-link match to the to/from headers so a cc doesn't create a phantom inbound note. `backend/email_integration/` matching (see `docs/guides/email.md`). (2026-06-20) + ### Follow-ups/reminders + NL search + bot grid-mutations (agreed plan, 2026-06-18) *Agreed with Grant 2026-06-18. Three workstreams, sequenced **W1 → W2 → W3**. **Overarching constraint (Grant):** the dominant risk is **leaking LP data (names, $, notes, contacts) to third-party LLMs — NOT write-safety.** A wrong number is recoverable; investor substance reaching Claude is not. Consequences: W2 keeps LP rows off Claude (only the question text + schema vocabulary leave the box; entity names resolved locally); W3 keeps bot mutation-parsing on local Qwen. Because this DB *logs* commitments/pipeline but doesn't move money, a bot mutation is low-stakes → **any team member may approve one in Matrix**; the guardrail is "the bot can't silently mass-change numbers," enforced by the per-mutation human approval gate, not a tight money gate.* @@ -131,8 +138,8 @@ Use the **matrix-bridge** repo's pattern to listen on a dedicated ten31-database **Long-term — extract the intake bot to its own repo (recommended, not yet done).** Containerizing from this monorepo is the pragmatic now-state, but the bot is a genuinely separate deployable (own process, own `matrix-nio` dep, own lifecycle); its only CRM coupling is the HTTP API (a clean network contract) plus ~40 lines of stdlib Spark client (cheap to vendor). The tell: the spark-control Update button would run `git reset --hard origin/main` on the **whole CRM clone** — wrong blast radius. `matrix-bridge` is already a dedicated repo; mirror it. The extraction is a migration (new Gitea repo, move code + tests + guide, vendor the client, re-point the Spark deploy), so it's deferred until worth the lift — do it *before* wiring the spark-control card if both land in the same push. -### In-app camera business-card intake — PLAN written, awaiting go-ahead (Grant, 2026-06-20) -*A camera button in the mobile top bar (left of the quick-log pencil) → take/choose a photo → the same vision-transcribe → parse → fuzzy-match → edit/approve/reject flow the Matrix card intake (M3) runs, surfaced as an inline mobile sheet. **Detailed plan: `docs/handoffs/in-app-card-intake-plan.md`.*** Key finding: the reusable core is nio-free and already reachable from the CRM (`server.py` imports `llm`; `matrix_intake/parse.py`+`spark.py` import no `matrix-nio`), so it's **one endpoint** (`POST /api/intake/card`) + **one mobile component**, no bot refactor / new dep / migration. Decisions LOCKED (Grant 2026-06-20): `source="app_card"`, form-field edits only (no NL-edit) for v1, any-authenticated-member access, ships in the s9pk. Ready to build on go-ahead. Reuses the New-investor sheet pre-filled + the proven `.quicklog-btn svg` icon-sizing fix for the camera button. +### In-app camera business-card intake — DONE & live (v0.1.0:100, device-confirmed 2026-06-20) +*Shipped: a camera button in the mobile top bar (left of the quick-log pencil) → take/choose a photo → vision-transcribe → parse → fuzzy-match → edit/approve/reject, surfaced as an inline mobile sheet (`source="app_card"`, form-field edits only, any-authenticated-member). The reusable core is nio-free and already reachable from the CRM (`server.py` imports `llm`; `matrix_intake/parse.py`+`spark.py` import no `matrix-nio`), so it landed as **one endpoint** (`POST /api/intake/card`) + **one mobile component** — no bot refactor / new dep / migration; reuses the New-investor sheet pre-filled + the `.quicklog-btn svg` icon-sizing fix. History: commits `463f624` (feature) / `622d454` (handoff). (Plan doc removed — git history is the record.)* ### Scoped service-credential auth path for automated CRM writers *Surfaced 2026-06-17 while deploying the Matrix intake bot. **Decision: defer — the bot uses a dedicated member username/password for now.** The CRM has no API-key/service-token path; its only auth is username+password → JWT. A dedicated **member** login is appropriately scoped against what matters operationally (no admin: can't manage users, reset data, or change settings) and unblocks the live smoke today.* diff --git a/docs/handoffs/in-app-card-intake-plan.md b/docs/handoffs/in-app-card-intake-plan.md deleted file mode 100644 index c269682..0000000 --- a/docs/handoffs/in-app-card-intake-plan.md +++ /dev/null @@ -1,133 +0,0 @@ -# Plan: in-app business-card intake (mobile camera → transcribe → approve) - -**Status:** PLAN — decisions locked 2026-06-20 (see end); awaiting Grant's go-ahead to build. -**Goal:** a camera button in the mobile top bar (left of the quick-log pencil) → take a photo or -pick one from the library → the **same** vision-transcribe → parse → fuzzy-match → edit/approve/reject -flow the Matrix card intake runs (M3), surfaced as an inline mobile sheet instead of a Matrix thread. - -## Why this is cheaper than it looks - -The reusable core is already nio-free and already reachable from the CRM monolith: - -- `server.py` **already** imports the Spark client: `sys.path.insert(.../ingest); import llm` - (`server.py:6485`). `llm.chat_vision` (multimodal → Spark Control passthrough) is the exact call - the bot's card OCR uses. -- `backend/matrix_intake/parse.py` imports only `json`, `re`, `spark`; `spark.py` imports only - `os`, `sys`, `llm`. **Neither imports `matrix-nio`.** So `server.py` can import `parse` + - `spark` the same way it imports `llm`, with **no bot refactor and no matrix-nio in the CRM**. -- Match (`find_intake_match` / `find_intake_candidates`) and the write - (`/api/fundraising/log-communication`, create-if-missing + contact upsert + audit) are **already** - in `server.py`. The fuzzy matcher just got the generic-word fix (v-next). -- The whole `backend/` tree (incl. `ingest/` + `matrix_intake/`) is `COPY`'d into the s9pk image - (`start9/0.4/Dockerfile:62`), so the imports resolve on the box at runtime. - -So this is **one new endpoint + one new mobile component**, reusing everything downstream. No -migration, no new dependency, no change to the live Matrix bot. - -## Architecture decision - -**Recommended (Option A): `server.py` imports `matrix_intake.parse` + `matrix_intake.spark` -directly.** Add `backend/matrix_intake` to `sys.path` (guarded like the existing `ingest` insert) -and `import parse, spark`. Pros: zero bot churn, reuses the tested transcribe + email-integrity -parse verbatim, no divergence. Con: the CRM now imports two modules out of the bot package — a mild -coupling, but they're pure (no nio, no CRM HTTP; we do NOT import the bot's `crm_client`). - -**Alternative (Option B): extract a shared `backend/intake_core/`** (`vision.py` = transcribe, -`extract.py` = parse/normalize/revise) that both the bot and `server.py` import. Cleaner ownership, -but it refactors the live, working bot for no functional gain right now. **Defer** unless the -coupling in A bites later (e.g. if the bot's `parse` grows nio-coupled helpers). - -> Module-name caution (from the matrix-intake guide): the bot imports by **bare name** (`import -> spark`, `import llm`, `import config`). `server.py` already has `ingest/` on its path (so `llm`, -> `config`, `http_util` resolve). Adding `matrix_intake/` lets `parse`/`spark` resolve with **no -> name overlap** against `ingest`'s modules — verified. Import only `parse` + `spark` (not -> `crm_client`/`settings`/`bot`). - -## Server: one new endpoint - -`POST /api/intake/card` — authenticated **member+** (it's a UI feature for the team; NOT bot-gated, -NOT admin-only). Body is JSON, image as base64 (no multipart parsing in the stdlib handler): - -``` -{ "image_b64": "", "mime": "image/jpeg" } -→ 200 { "transcription": "...", - "proposal": { investor_name, contact_name, contact_email, contact_title, - city, linkedin_url, phone, mobile, note }, - "match": { id, name } | null, - "candidates": [ { id, name, score, matched_on }, ... ] } -``` - -Handler flow (mirrors `bot.handle_card` → `handle_intake`, minus Matrix): -1. Auth (reuse the standard authenticated gate). -2. Decode + size-guard the base64; default mime `image/jpeg`. -3. `transcription = spark.transcribe_card(image_b64, mime)` (Spark Control vision). On error → 502 - `{ ok:false, reason:"vision_unavailable" }`; on `<5` chars → 200 `{ ok:false, reason:"unreadable" }`. -4. `proposal = parse.parse_message("New investor — from a business card:\n"+transcription, roster=None)` - — the **email-integrity rule rides along** (an address/phone/LinkedIn is kept only if it literally - appears in the transcription, never minted), exactly as in Matrix. `roster=None` for v1 (the - team-roster framing is a Spark-side env; the box can pass None — prior parse behavior). -5. `match = find_intake_match(...)`, `candidates = find_intake_candidates(...)` from the proposal's - `investor_name`/`contact_email`. -6. Return `{ transcription, proposal, match, candidates }`. - -**Approve** reuses the existing write — the mobile sheet POSTs the (possibly-edited) proposal to -`POST /api/fundraising/log-communication` with `create_investor_if_missing` (new) or the picked -`_match_id`/candidate row (existing), tagged **`source="app_card"`** (a new provenance value -distinct from `matrix_card`/`matrix_intake`). **Reject** is client-only (discard the sheet) — no -server call, nothing written. - -**No NL-edit for v1.** In a form UI, editing the fields directly is easier than chatting "change the -email to…". The fields are inline-editable in the sheet. (A `/api/intake/card/revise` wrapping -`parse.revise` is a possible later enhancement if Grant wants conversational corrections.) - -## Mobile UI: one new component - -`MobileCardCapture` in the shell top bar, **left of `MobileQuickLog`** (the cluster at -`index.html:14609`). Reuses ``, `.sheet-input`, `StageChip`, and the New-investor -sheet patterns (8g). - -- **Trigger:** a camera-icon button + a hidden `` — - **omit `capture`** so iOS shows the native action sheet (Take Photo / Photo Library / Browse), - satisfying "take a photo OR use an existing photo." (`capture="environment"` would force the camera - and skip the library — not what we want.) -- **On select:** read the file as a data URL; **downscale via `` to ~2000px max dimension** - before base64 (native, no library) — keeps the payload under the StartOS reverse-proxy body cap - (the matrix-intake guide's known `413` risk) and the model downscales to ~2 MP anyway. Don't - over-shrink (hurts OCR). Show a "📇 Reading the card…" loading state. -- **POST** to `/api/intake/card`; on `unreadable`/`vision_unavailable` show a "try a clearer, - well-lit, fill-the-frame photo" message with a Retake button (same UX as the bot's `📇` replies). -- **Approval sheet** (the New-investor sheet, pre-filled from `proposal`): editable investor name + - contact name/email/title/city/phone/mobile/LinkedIn; a small "Existing investor?" block when - `match`/`candidates` came back (pick a candidate row, or "Add as new"); **Approve** → the - log-communication write; **Reject/Retake** → discard. -- **Icon:** reuse the proven `.quicklog-btn svg` sizing fix (explicit CSS `width/height` + `flex:none`) - so the new camera SVG doesn't hit the same iOS sole-flex-child collapse. - -## Scope / effort / risk - -- **Server:** ~1 endpoint + the `parse`/`spark` import guard + an `app_card` source tag. Small. -- **Frontend:** ~1 component (camera button + capture input + canvas downscale + approval sheet) + - CSS for the button. Medium (the approval sheet is the New-investor sheet pre-filled). -- **No** migration, **no** new dependency, **no** Matrix-bot change. -- **Tests:** an endpoint test that stubs `spark.transcribe_card` (like `test_spark.py` does) and - asserts the transcribe→parse→match shape + the email-integrity passthrough; the real vision/OCR - path stays **live-smoke only** (same as Matrix M3). Frontend is inspection + on-device. -- **Risks:** (1) large-image upload vs the reverse-proxy cap — mitigated by client-side canvas - downscale; (2) iOS file-input behavior across Safari/standalone-PWA — verify on-device; (3) OCR - accuracy is the **shared** Matrix limitation (resolution-bound; "fill the frame"), not new here; - (4) the `parse`/`spark` import must be lazy/guarded so a dev `./start.sh` without Spark reachable - still boots (the endpoint just 502s) — mirror the existing lazy `import llm` at `server.py:6485`. - -## Decisions (locked, Grant 2026-06-20) - -1. **Provenance tag = `source="app_card"`** — distinct from `matrix_card` / `matrix_intake`. -2. **v1 = editable form fields only** — no conversational NL-edit on the card (no - `/api/intake/card/revise`); the user taps a field and fixes it in the sheet. NL-edit can follow - later if it proves wanted. -3. **Access = any authenticated member** — the camera button + `POST /api/intake/card` use the - standard authenticated gate (not admin-only, not bot-gated); a human still approves every write. -4. **Delivery = the s9pk** (server endpoint + frontend) — a normal version bump → build → box - install, like v99 (NOT bot-only like the Matrix M3). - -Build scope is otherwise as specified above. Mobile-only (top-bar button); same captured fields + -literal-in-source integrity rule as the Matrix card flow.