grant/ten31-database
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Restrict comms_by_user/email_counts_by_user to matched-investor email

Browse Source 
Both NL-query intents counted/listed a user's ENTIRE captured sent corpus
(internal, vendor, personal mail) rather than only email to a matched investor
— they were missing the `EXISTS email_investor_links` gate that recent_emails
and the Communications panel's query_email_activity use. Their own docstrings
said "investor emails", so the behavior was wrong, not just loose.

Add the matched-only gate to both, mirroring query_email_activity. The runner
test now seeds an unmatched sent email and asserts it is excluded (without the
fix comms_by_user returns 3 not 2, this_week 2 not 1) — the prior fixture
linked every email, so the leak went uncaught.

Also documents the matched-only rule in the nl-query guide, and refreshes the
AGENTS.md Current state (v93 deployed; this fix pending a v94 s9pk since the
intents run on the box, not the bot).
This commit is contained in:
Keysat
2026-06-18 20:24:52 -05:00
parent f7b03ee109
commit 2d43bad6fc
4 changed files with 38 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+4 -4
View File
@@ -106,13 +106,13 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
## Current state
_Phase 0 + Phase 1 built; **box live at v0.1.0:91; repo at v0.1.0:92** (reminders, deploy pending). **The fundraising grid + email capture is the canonical system of record.** Active thread: **W2 natural-language query** (backend + Matrix `@bot` surface built; web "Ask" box next). Deploy/feature history: git log + `start9/0.4/startos/versions/`; longer-term backlog/debt: `ROADMAP.md` / `EVALUATION.md`._
_Phase 0 + Phase 1 built; **box live at v0.1.0:93; repo at v0.1.0:93** (reminders W1 + NL-query W2 deployed 2026-06-18). **The fundraising grid + email capture is the canonical system of record.** Active thread: **W2 natural-language query** (backend + Matrix Q&A live; web "Ask" box next). Deploy/feature history: git log + `start9/0.4/startos/versions/`; longer-term backlog/debt: `ROADMAP.md` / `EVALUATION.md`._
- **W2 — natural-language query (read-only): BACKEND + MATRIX `@bot` surface built + tested locally 2026-06-18; web "Ask" box next.** `backend/nl_query/` — 12 curated parameterized queries + a slot validator (the trust boundary; no generic SQL) + a **local-Qwen** translator (question→{intent,slots} via Spark Control; nothing leaves the box, **no Claude, no redaction** — the simplification Grant chose). `POST /api/query/nl` (also accepts direct `{intent,slots}`) + `GET /api/query/catalog`, `require_bot_or_admin`, audited (`entity_type='nl_query'`). **Local Qwen translated 12/12 of Grant's real example questions correctly against the live Spark** — settles local-only (Claude not needed). Soft-delete-correct per table (gotcha: `fundraising_*` has **no `deleted_at`** — `graveyard` is the axis; emails via a live `eam` sighting). Guide: `docs/guides/nl-query.md`. **Step 5 (Matrix Q&A) DONE** — thin client in `backend/matrix_intake/query.py` (trigger grammar + answer rendering) + `crm_client.nl_query` + `bot.py` wiring, read-only (no approval gate), tested in `test_query.py`. **Two entry points (room-per-purpose model):** a **dedicated Q&A room** (`MATRIX_QUERY_ROOM`) where every message is a question, **and** the `?`/`@bot` trigger still working in the intake room as a cross-room convenience. Ships on the **Spark** (git pull + restart, no s9pk for the bot). Q&A room `!RGlJEObVaIUtUVcHtx:matrix.gilliam.ai` created + bot invited (2026-06-18). **BUT the box-side `/api/query/nl` endpoint is NOT live yet** (box v91; verified 404 on 2026-06-18) — it lands with the **v93 s9pk** (reminders + W2). **So DON'T activate the bot deploy (set `MATRIX_QUERY_ROOM` + restart) until v93 is installed**, or every question 404s. Code committed + pushed; bot deploy is staged to follow the v93 install. **Next: step 4 web "Ask" box (Communications tab)** — the last thin client.
- **W2 — natural-language query (read-only): BACKEND + MATRIX Q&A LIVE (deployed v0.1.0:93, 2026-06-18); web "Ask" box next.** `backend/nl_query/` — 12 curated parameterized queries + a slot validator (the trust boundary; no generic SQL) + a **local-Qwen** translator (question→{intent,slots} via Spark Control; nothing leaves the box, **no Claude, no redaction** — the simplification Grant chose). `POST /api/query/nl` (also accepts direct `{intent,slots}`) + `GET /api/query/catalog`, `require_bot_or_admin`, audited (`entity_type='nl_query'`) — **live on the box** (verified 400/200 post-install). Soft-delete-correct per table (gotcha: `fundraising_*` has **no `deleted_at`** — `graveyard` is the axis; emails via a live `eam` sighting). Guide: `docs/guides/nl-query.md`. **Step 5 (Matrix Q&A) DONE + DEPLOYED** — thin client in `backend/matrix_intake/query.py` (trigger grammar + answer rendering) + `crm_client.nl_query` + `bot.py` wiring, read-only (no approval gate), tested in `test_query.py`. **Two entry points (room-per-purpose model):** a **dedicated Q&A room** (`MATRIX_QUERY_ROOM=!RGlJEObVaIUtUVcHtx:matrix.gilliam.ai`) where every message is a question, **and** the `?`/`@bot` trigger in the intake room as a cross-room convenience. Bot rebuilt + running on the Spark (logs: `answering questions in room …`). **End-to-end verified from inside the bot container** (3 questions → correct intents, live box, no errors; `investors_cold` hits the 500-row cap so Matrix shows 30 + a refine note). **Remaining: the actual in-room Matrix smoke (a human typing a question) — not yet done.** **Matched-only fix (2026-06-18, post-v93):** `comms_by_user` + `email_counts_by_user` were counting/listing the user's *entire* captured sent corpus, not just investor-linked email (missing the `EXISTS email_investor_links` gate that `recent_emails`/`query_email_activity` use) — **fixed + regression-tested in the repo, but the box still runs the leaky v93 behavior until a v94 s9pk** (these intents run on the box, not the bot). **Next: step 4 web "Ask" box (Communications tab)** — the last thin client.
- **W1 — reminders & follow-ups: BUILT + tested locally (v0.1.0:92), DEPLOY PENDING.** First-class tickler tied to the grid (migration `0006`; CRUD `GET/POST/PATCH/DELETE /api/reminders`; derived `reminder_status` grid column; Reminders page + dashboard card + digest section; the `last_activity_at` recency rollup that W2 reuses). Needs s9pk build + install (authorize first; verify `0006` against a DB copy). Deferred **W1b** = nurture-gap auto-suggested reminders.
- **W1 — reminders & follow-ups: LIVE (deployed v0.1.0:93, 2026-06-18).** First-class tickler tied to the grid (migration `0006` — applied cleanly on the box per logs; CRUD `GET/POST/PATCH/DELETE /api/reminders`; derived `reminder_status` grid column; Reminders page + dashboard card + digest section; the `last_activity_at` recency rollup that W2 reuses). `0006` was verified up/down against a copy of `crm.db` before install. Deferred **W1b** = nurture-gap auto-suggested reminders.
- **Done & live (detail in git log / ROADMAP):** email-proposal Matrix review + `bot` role (box v91); grid-driven Pipeline (v88); Matrix intake bot (Spark `matrix-intake` container); Gmail capture (DWD) + propose→approve + daily digest; Thesis Workshop + Architect (Claude, dual-approval); outreach drafts + radar. All draft-only.
- **Tests:** **35/35 backend green** (`python3 backend/run_tests.py`; +`nl_query/` + matrix `test_query.py` suites), `py_compile` clean; render-smoke gates `make`.
- **Next (priority order):** 1) **deploy reminders (v92) + W2 together** — bump to **v0.1.0:93**, build s9pk, install, browser-verify (authorize first; verify `0006` against a DB copy) — **this is the gate for the Matrix Q&A: the bot's step-5 surface 404s until `/api/query/nl` is on the box**; THEN activate the bot deploy (set `MATRIX_QUERY_ROOM` on the Spark + git pull + restart) + in-room smoke; 2) **W2 step 4** web Ask box (last NL-query client); 3) **W3** bot grid-mutations behind the Matrix approval gate (local-Qwen parse); 4) **W1b** nurture-gap reminders; 5) Grant + Jonathan freeze v2.0 canonical; 6) in-room smoke of the intake disambiguation numbered-pick grammar; then P2 debt (reports comms-aggregate soft-delete sweep, `?limit=abc` crash, auth regression test, oversized StartOS icon).
- **Next (priority order):** 1) **in-room Matrix smoke** of the Q&A room (type a real question; confirm the answer renders well on mobile — broad questions like "cold investors" hit the 500-row cap → 30 shown + refine note) + the intake `?`/`@bot` trigger; 2) **W2 step 4** web Ask box (last NL-query client); 3) **W3** bot grid-mutations behind the Matrix approval gate (local-Qwen parse); 4) **W1b** nurture-gap reminders; 5) Grant + Jonathan freeze v2.0 canonical; 6) in-room smoke of the intake disambiguation numbered-pick grammar; then P2 debt (reports comms-aggregate soft-delete sweep, `?limit=abc` crash, auth regression test, oversized StartOS icon).
- **Open / risks:** W2 translation only **happy-path-validated** (typos/ambiguous/no-match phrasings shake out in live use); **Claude/Architect path still unverified live on the box**; v2.0 reserve-asset spine is the *working approved* spine but **not canonical** (needs dual sign-off); doc drift — `crm-overview.md` + `EVALUATION.md` still call `lp_profiles` live.