A throwaway, admin-only diagnostic for the deferred contacts <-> fundraising_contacts
consolidation: GET /api/admin/contacts-census + a Settings -> Admin "Run census" button
report the A/B/C populations (linked / contacts-only / pill-only) plus the
communications/opportunities repointing surface. Counts only, no PII -- mirrors
backend/scripts/contacts_census.sql so the numbers can be read off the box without a
shell. All pieces are tagged TEMPORARY; delete the endpoint + route + button after the
census is captured. No schema change. 45/45 tests, render-smoke green.
For the deferred contacts <-> fundraising_contacts consolidation (ROADMAP backlog):
counts the linked (A), contacts-only (B), and pill-only (C) populations plus the
communications/opportunities repointing surface. Counts only -- no names/PII -- so it
can be run on the box's /data/crm.db without anything sensitive leaving.
Removals (net -570 lines):
- Delete the Instructions and Feedback (feature_requests) pages + backend.
- Retire lp_profiles + investor_type across server, ingest, and seeds; migration
0008 drops both empty tables (a sanctioned one-off exception to
never-hard-delete). 0001's lp_profiles ALTER is removed so a fresh DB doesn't
break the migration chain (live DBs already applied it).
Fixes:
- Email sync: a transient timeout no longer terminally parks a mailbox; the
scheduler retries 'retrying' each cycle and re-includes errored accounts on an
hourly backoff, so stuck mailboxes self-heal.
- Mobile Contacts: page through the full directory (server caps 500/page) -- one
fetch silently truncated at 720, hiding people from the list and from search.
- Mobile email review: clock icon to set a reminder inline; approval cards show
date/time.
New:
- Admin-only purge of soft-deleted rows (Settings -> Admin; type-to-confirm,
refuses any row still linked to live data).
Tests: 45/45 (adds test_sync_ready + test_purge_soft_deleted). Reviewer pass
applied (NULL reminders.contact_id on contact purge). Bumped to v0.1.0:104.
A mobile, in-app twin of the Matrix business-card flow (M3): photograph a
card in the app and it becomes a reviewed fundraising-grid add/note, with a
human approving every write.
Server — POST /api/intake/card (authenticated member+, read-only): lazily
imports the bot's nio-free parse + spark core, vision-transcribes the photo
(local VL via Spark Control — nothing to Claude), runs the same email/phone/
LinkedIn integrity rule + fuzzy matcher, and returns a proposal plus exact
match / fuzzy candidates. No write happens here.
Frontend — a camera button in the mobile top bar (left of the quick-log
pencil) → take or pick a photo → <canvas> downscale to JPEG (also normalizes
iPhone HEIC) → the endpoint → an editable review sheet (proposal fields +
existing-investor picker). Save reuses /api/fundraising/log-communication
tagged source="app_card".
No schema change, no migration, no new dependency, no Matrix-bot change. The
camera/canvas/OCR path is on-device-only (jsdom has no canvas); covered by
test_intake_card.py (stubbed vision+parse) + the render/mount smokes.
reviewer agent flagged the broadened redact_thread predicate (event_id OR in_reply==root)
as over-matching any plain reply to a thread root. Gate the bare-in_reply clause to the bot's
own sender (the nudge is always ours); thread children (cards/acks/human yes-no) still match by
rel_type=m.thread. Add unit edges for _name_similarity's all-generic fallback and a contact_id
NULL orphan case for the grid-blob email heal.
Grant's real-phone testing surfaced seven items; this lands six (the seventh,
in-app camera card intake, is planned in docs/handoffs/in-app-card-intake-plan.md).
CRM half — ships in the s9pk (v0.1.0:99):
- Intake fuzzy match no longer over-indexes on generic firm words. _name_similarity
now compares DISTINCTIVE tokens only (generic descriptors — "Investment Group",
"Capital", "Family Office" — stripped via _GENERIC_ORG_WORDS) for both the difflib
ratio and the Jaccard, so "Fortitude Investment Group" stops surfacing Aether/Russell
while "Aether Capital" still surfaces "Aether Investment Group". +2 regression cases.
- Mobile grid "Last contact"/staleness sort is reversible. SortSheet gains opt-in
dir/onToggleDir; other surfaces (Contacts/Pipeline) are untouched.
- Mobile "Edit investor" prefills a contact's saved email. GET /api/fundraising/state
heals a blank grid pill email from the linked classic contact
(fundraising_contacts.contact_id -> contacts.email), fill-only, by pill order then
name; the next one-row save persists it. +test_grid_email_heal.py.
- Mobile quick-log pencil icon renders. iOS collapses a sole, centered, attribute-only
-sized flex-child <svg>; .quicklog-btn svg now gets explicit CSS width/height + flex:none
(the pattern the working bottom-tab/sort-pill icons use). The v97 fix only changed color.
Matrix intake bot — ships on the Spark (bot-only, NOT the s9pk):
- Approve/reject now redacts the whole intake thread (card + ack + main-timeline nudge +
the user's own photo/note), mirroring the email-review room; redact_thread takes the
room as an arg and matches replies by m.thread OR m.in_reply_to (so the nudge clears).
No more in-Matrix confirmation after a commit (the thread vanishing is the ack).
Needs the bot to hold a redact/moderator power level in the intake room.
- New one-time backend/matrix_intake/redact_intake.py clears the room's pre-existing
backlog (dry-run default; --apply).
Tests 42/42 green; frontend render-smoke green. Frontend fixes are inspection + render
-smoke verified (on-device confirm pending); the bot redaction is live-smoke only.
Completes business-card contact capture. The transcription prompt now labels
Phone/Mobile/Fax on separate lines, and the extractor maps an office/main number ->
phone and a cell -> mobile, never a fax. Both carry the same digit-in-source
integrity rule as email/LinkedIn: a number is kept only if its digits literally
appear in the source (or, on revise, the instruction) -- never minted. The proposal
card shows Phone + Mobile and they're editable (aliases phone/tel/office, mobile/cell).
Server: _upsert_contact_from_fundraising now accepts contact.phone + contact.mobile
and writes them to the canonical contact record (contact-level, not grid pills),
shipped in s9pk v0.1.0:98. No schema change -- the contacts columns already exist.
41/41 backend suite green + the matrix_intake units; card flow end-to-end is live-smoke.
_upsert_contact_from_fundraising now reads contact.phone and writes contacts.phone on
both the insert and update paths, so a phone captured from a business card persists on
the canonical contact record. Phone stays contact-level (not a grid pill field),
matching how the team edits it. Validated by test_grid_add_investor.py.
This is the SERVER half of business-card phone capture, staged for the next s9pk
(version bump + build + install). The bot's phone extraction/card/payload lands in the
same deploy, so phone never shows on a card before the box can store it. NOT yet
built or installed to the box.
The card transcription prompt now reads emails/URLs/phones character-by-character,
explicitly forbids autocompleting toward a plausible domain (the mara.com ->
marac.com failure), and emits labeled lines (which also feeds the field extractor
cleaner input).
The extractor gains city + linkedin_url. city is a plain field (low-harm if wrong;
the human sees it on the card). linkedin_url follows the email-integrity rule: kept
only if it literally appears in the source / a revise instruction, never minted -- a
wrong profile URL points at the wrong person. Both flow to the contact via the
existing log-communication upsert (city also syncs to the grid contact pill).
Phone is intentionally NOT included yet: the bot's write path can't store it until a
small server-side change lands (next s9pk). See the matrix-intake guide.
The in-thread approval handler already routes any reply that isn't yes/no/
edit-grammar through local Qwen (parse.revise), but the card copy only mentioned
'edit field=value', so the natural-language path was undiscoverable. Lead with
plain-words edits; the deterministic field=value fast-path still works.
The intake bot now accepts a photo of a business card in the intake room and
turns it into the same new-investor proposal a typed note would. The only new
step is image -> text; everything downstream (parse, fuzzy match, in-thread
approval, log-communication write) is reused unchanged.
M3 was deferred only because Spark Control had no vision model. That blocker is
gone: the daily-driver Qwen is vision-capable under the same model id, and the
gateway forwards OpenAI multimodal content untouched, so no gateway/server/s9pk
change is needed -- this ships bot-only (git pull + rebuild on the Spark).
Transcribe-then-reuse (not vision-straight-to-JSON) is deliberate: the
transcription becomes the source text the email-integrity rule checks against,
so a mis-read address can't reach the CRM unapproved -- same guarantee as the
text path. Card commits tag source="matrix_card" for the audit log.
- llm.chat_vision: multimodal /v1/chat/completions, same model, same gateway
- spark.transcribe_card: faithful card->text, "" on a non-card (NONE sentinel)
- bot.on_image/handle_card: download image, transcribe, hand to handle_intake
- crm_client: source provenance overridable via the proposal's _source key
- tests: test_spark.py + a provenance case; 41/41 suite green
Make the app installable to the iOS home screen and launch standalone
(full-screen, no browser chrome, dark status bar). Add manifest.webmanifest,
square app icons (ten31-app-icon.svg -> 192/512/apple-touch-icon), the
apple-mobile-web-app + manifest <head> tags, viewport-fit=cover, and a
pre-auth /manifest.webmanifest route. No service worker by design.
Grid full-screen investor detail, conformed to the dc anatomy:
- G4: pipeline stage as a single tappable .detail-tap-card (chip + Change/Add)
- G5: dedicated Reminder card fed by the soonest active reminder; tri-state
(loading → disabled "Checking…" so a pre-load tap can't POST a duplicate;
none → "No reminder set"; object → edit). Edits PATCH in place, else POST.
- G6 (notes timeline) was already in place.
Open-in-Grid deep-link, now on all three mobile detail surfaces (Contacts,
Pipeline, Reminders): a shared shell openInvestorInGrid(rowId) sets a one-shot
gridUiAction object the mobile grid consumes on mount to open that investor's
detail; the desktop grid drains the unrecognized object so it can't linger.
Each surface gets its grid row id from a server-injected source_row_id:
contacts via contact_grid_signals, opportunities via the durable
fundraising_investor_id join, reminders via the investor_id join. All are
read-only/GET-only or field-allowlist writes, so none need a strip point.
Tests: source_row_id injection assertions for contacts, opportunities, and
reminders; full suite 40/40. Client surfaces jsdom-verified.
The mobile "New investor" sheet now captures three optional fields beyond name/contact/note,
matching the dc (GridApp.dc.html:737):
- Initial pipeline stage — a .stage-pick chip picker, defaulting to "Not in pipeline" so a
plain directory add never auto-creates an opportunity row (Grant's call).
- A framed "Flag as Priority" toggle.
- An optional reminder (title + a progressive due-date field).
submitCreate orchestrates one-row calls in order: create (log-communication
create_investor_if_missing, now carrying priority) -> if a stage was picked, link to the
pipeline at that stage (reusing applyStage's idempotent link-then-PATCH) -> if a reminder
title was given, POST /api/reminders keyed on the new row's source_row_id. The link and
reminder steps are non-fatal: a failure toasts but never loses the created investor, and a
create that returns no row id warns instead of a clean success.
Backend: handle_log_fundraising_communication honors an optional priority flag only on its
create-if-missing branch (an existing-row log never touches priority).
Guarded by test_grid_add_investor.py (priority-on-create, defaults-False, the create-branch-
only invariant, and the create->link / create->reminder handshakes on a freshly-synced row).
40/40 backend green; the create sheet was interaction-verified in a throwaway jsdom harness.
Bring the mobile Pipeline surface to the PipelineApp.dc.html default anatomy:
- Segmented control → horizontal-scroll pills with label + count badge; the active
pill tints to its own stage color via --seg-* (aliased to --chip-*, so it flips in light).
- Card → earmark corner + name + Priority pill / $amount · dot · recency / labeled
‹ Prev · Next › move footer (was name + contact·org sub + bare chevrons). Compact amount.
- Stage-column header → StageChip + investor count + committed sum.
- Page dots → tappable, active = 22px accent bar.
Backend: the opportunities list injects two derived read-only fields (mirroring the
Contacts-list pattern; opp writes use a field allowlist so neither round-trips):
- existing_investor (contact_grid_signals committed>0) so the card earmark agrees with
the detail's "Existing LP" pill.
- last_contact_date (MAX communication_date on the deal's contact, deleted_at-filtered)
→ card recency line + the Staleness sort (replaces the updated_at proxy).
Guarded by new soft-delete assertions in test_soft_delete_reads.py. 39/39 green.
Add a shared SortPill + SortSheet (label+hint option rows) and per-surface sort
tables:
- Grid: Name / Pipeline stage / Committed / Last contact / Priority, applied in
the displayed memo (name is the tiebreak; staleness ranks longest-since-contact
first, no-activity treated as most stale; committed uses the fund rollup).
- Pipeline: Name / Amount / Last activity / Priority, sorted within each stage.
"Last activity" uses opp.updated_at as a recency proxy until the Pipeline card
wires true last-contact recency (8f).
- Contacts: drop the investor/prospect type tabs (the prospect type is unused);
add a Priority sort alongside Name A-Z/Z-A and Last contact.
contact_grid_signals() now also surfaces the linked investor's priority flag,
injected on both contact read paths (same derive-on-read contract as committed /
pipeline_stage), powering the Contacts Priority sort. Extended
test_contacts_grid_signals.py covers it; 39/39 backend green.
Grid detail (G6): replace the single row.notes blob with a NoteTimeline fed by
a new investor-level read, GET /api/communications?source_row_id=<grid row id>
(filter maps the grid row -> fundraising_investors.source_row_id ->
fundraising_contacts.contact_id -> communications, soft-delete-respecting;
cancelled-flag fetch + commsReload after a log). Note-logging now uses the
shared LogCommunicationSheet, retiring the bespoke noteForm select sheet and the
dead .fs-note-log style.
New MobileQuickLog: a shell mobile-top-bar pencil reachable from every tab —
two-step sheet (pick investor via search + recent-first pool -> inline log form)
writing through the one-row /api/fundraising/log-communication path.
source_row_id and contact_id are kept mutually exclusive in
handle_list_communications so a future caller passing both can't get the empty
intersection. Guarded by test_grid_comm_timeline.py (cross-contact aggregation,
investor isolation, soft-delete); 39/39 backend green.
8a — Grid card: existing-LP earmark corner-triangle (replaces left-border), right-side
PRIORITY pill (replaces the rejected star), 4-stage chip, zero-commit dim; detail star ->
"Existing LP" pill. Contacts card: two-letter avatar initials + existing-LP ring + stage pill
+ recency; disposition badge dropped. New backend contact_grid_signals() injects derived
read-only committed/pipeline_stage on GET /api/contacts and /api/contacts/{id} (existing-LP
ring + stage pill); read-only directory, so no strip-point. DESIGN.md §4/§8 reconciled.
8b — Contacts and Pipeline detail surfaces converted from full-screen to drag-dismiss bottom
sheets matching the .dc.html anatomy: Contacts gets an email-copy pill, Log/Email actions, and
an Organization card; Pipeline gets stat tiles, an inline move-stage list, and a notes timeline
+ Log sheet. Both log via POST /api/communications; BottomSheet gains a `stacked` prop to layer
the Log sheet over a detail. Reviewer fixes: cancelled-flag fetch guards (stale-response race),
keyed single-contact signals query, multi-investor dedup test.
All deploy-pending (no s9pk built); not device-tested. 38/38 backend tests green.
Adds the editable half of BRIEF §3a's mobile grid set: rename an investor
and add/edit/remove its contact pills from the mobile detail sheet.
New POST /api/fundraising/update-row is the one-row read-fresh-modify-write
twin of log-communication: it mutates only the target row's name/contacts in
the canonical grid blob server-side, then bumps the version + re-syncs the
relational tables. It never accepts a whole-grid payload, so a stale mobile
client can't clobber concurrent edits to other rows (the reason mobile avoids
the whole-grid PUT). _sanitize_fundraising_contacts whitelists the known pill
fields as the trust boundary; removing a pill is soft on the classic contacts
directory (only the grid pill + fundraising_contacts row drop).
Frontend: MobileFundraisingGrid gains an Edit bottom-sheet (name input + pill
editor with client-side dedup); money stays desktop-only. New CSS is
theme-var-only so it flips in light mode.
Verified: test_fundraising_update_row.py (24 assertions, real HTTP), full
suite 37/37, render-smoke + a 375px jsdom interaction harness green.
Adds the mobile-first Fundraising Grid (<768px): a lean MobileFundraisingGrid
that reads /api/fundraising/state once and renders an investor card list over
the active view (name, committed $, pipeline-stage chip, staleness-colored
recency, Existing-Investor accent, Priority corner; graveyard muted) with a
bottom-sheet view picker and search. Tap a card -> full-screen detail with
read-only commitments/contacts/notes plus edit sheets: log a note, pipeline
stage, set a reminder, and a "+ New" investor create flow with client-side
dedup typeahead.
All writes go through the targeted one-row endpoints (log-communication,
pipeline link, opportunities stage PATCH, reminders) — NEVER the whole-grid
PUT, which would race the multi-user grid (BRIEF §3a). FundraisingGridPage is
now a useIsMobile() wrapper over the renamed-but-untouched desktop grid and
the new mobile one (rules-of-hooks-safe; desktop unchanged).
Backend: inject a read-only opportunity_id into grid rows
(opportunity_id_by_source_row; added to both strip points) so the mobile detail
can PATCH a linked opp's stage directly. Earliest-opp-wins ordering keeps it
consistent with pipeline_stage and the link's canonical pick.
Editing an existing investor's name + contact pills stays read-only here
(deferred to P3b — needs a narrow per-row PATCH + pill editor).
Tests: test_grid_pipeline_link extended (opportunity_id inject/strip/round-trip);
36/36 backend green, render-smoke green.
Phase 1 mobile foundation (additive, no desktop change): :root mobile vars, a
4-tab bottom nav bar + mobile account/logout popover wired into App, a
bottom-sheet CSS primitive, and .mobile-only/.desktop-only utilities -- all
display:none >=768px. The <BottomSheet> React component + useIsMobile() + the
per-surface 15px type bump are deferred to Phase 2 (first use); light theme to
Phase 6.
Review hardening (fresh-eyes pass on the Phase 0+1 diff): validate stage in
handle_create_opportunity + handle_update_opportunity against PIPELINE_STAGES --
the narrower 4-stage enum makes a stale-client write of a legacy value invisible
to the report ORDER BY CASEs and unsettable from the UI. Use the canonical
pipelineStageLabel in the opportunity detail select; document the intentional
graveyard omission in the existing_investor / staleness helpers.
Tests: stage-validation regression in test_grid_pipeline_link.py + empty
source_row_id guard in test_pipeline_stages_v2.py; 36/36 green, render-smoke green.
Collapse the inherited 6-stage opportunity funnel to the locked 4-stage
per-investor funnel (lead -> engaged -> diligence -> commitment), terminal at
commitment. Migration 0007 remaps existing stage values (outreach/meeting ->
engaged, due_diligence -> diligence, committed/funded -> commitment) and
archives the stray 'lost' value (the grid row is left intact). Inject read-only
existing_investor (total_invested>0), last_activity_at, and staleness
(''/'aging'>=30d/'stale'>=60d) into the grid GET, stripped on write. Frontend:
4-stage chip tints + Pipeline board / opp-form / mock on the new enum.
The visible desktop existing-investor star + staleness recency column + the
Stale saved view are deferred to mobile Phase 3 (data is injected + test-locked
now, so that phase stays pure-frontend). Longshot was already retired by prior
cleanup -- no-op.
Tests: test_pipeline_stages_v2.py (migration remap + derivation boundaries) +
updated grid-pipeline-link / soft-delete / nl_query; 36/36 green, render-smoke
green, fresh-DB migrate clean.
Both NL-query intents counted/listed a user's ENTIRE captured sent corpus
(internal, vendor, personal mail) rather than only email to a matched investor
— they were missing the `EXISTS email_investor_links` gate that recent_emails
and the Communications panel's query_email_activity use. Their own docstrings
said "investor emails", so the behavior was wrong, not just loose.
Add the matched-only gate to both, mirroring query_email_activity. The runner
test now seeds an unmatched sent email and asserts it is excluded (without the
fix comms_by_user returns 3 not 2, this_week 2 not 1) — the prior fixture
linked every email, so the leak went uncaught.
Also documents the matched-only rule in the nl-query guide, and refreshes the
AGENTS.md Current state (v93 deployed; this fix pending a v94 s9pk since the
intents run on the box, not the bot).
Read-only natural-language query over the curated nl_query endpoint, answered
in-thread. Two entry points (room-per-purpose model): a dedicated Q&A room
(MATRIX_QUERY_ROOM) where every top-level message is a question, plus the
?/@bot trigger in the intake room as a cross-room convenience. Both routes hit
the same handle_query -> crm_client.nl_query -> POST /api/query/nl; translation
runs on the box's local model, nothing leaves the box, and there is no write
path so no approval gate applies.
Pure logic (trigger parsing, answer rendering) in query.py with offline tests;
async room wiring in bot.py (live-smoke only, per the bot's convention).
Bot-side only, ships on the Spark via git pull + restart. Depends on the
box-side /api/query/nl endpoint, which lands with the v93 s9pk (reminders + W2):
until v93 is installed the Q&A surface 404s, so the bot deploy is staged to
follow that install.
Read-only "ask the database in plain English" backend. Translation runs on
the local Qwen via Spark Control (question -> {intent, slots}); nothing leaves
the box, no Claude and no redaction boundary (the simplification chosen after
pressure-testing). The safe surface is a curated catalog of ~12 hand-written
parameterized queries; a slot validator is the trust boundary (no generic SQL,
no dynamic identifiers). POST /api/query/nl + GET /api/query/catalog, gated
require_bot_or_admin, read-only, audited. Soft-delete-correct per table.
Local Qwen translated 12/12 real example questions correctly against the live
Spark. Web "Ask" box and Matrix bot still to come (steps 4-5).
The bot was granted a redact/mod power level in the review room, so it can now
clear a resolved thread entirely, not just the card: redact_thread redacts the
card root then scans recent history for its m.thread replies (the human's yes/no
+ any bot messages) and redacts those too, so decided threads drop out of the
threads view, not only the main timeline. Drops the in-thread confirmation on a
successful decision (the thread clearing is the ack; a confirmation would keep
the thread alive). redact_resolved.py extended to also clear replies of already-
resolved threads for the one-time backfill. Bot-only; no s9pk change.
Cards decided before the auto-redact behavior shipped are already 'closed' in
the CRM, so the bot's to_close sweep never redacts them. redact_resolved.py walks
the review room, keeps cards still pending (CRM 'open' list), and redacts the
rest. Dry-run by default; --apply to act. Run via docker compose on the Spark.
Three post-smoke refinements to the Matrix email-proposal review:
1. Dash separators (bot): every card/reply is framed with a dash rule top and
bottom so threads stop bleeding together vertically on mobile.
2. Remove decided threads (bot): on a conclusive approve/dismiss from either
surface, the bot redacts the card (client.room_redact) so the room clears
down to only undecided items. Redacting the bot's own card needs no power;
the web->Matrix path now redacts instead of posting a closure note.
3. Clearer note wording (server v91 + bot): the proposed grid note now names who
emailed whom -- "{teammate} emailed {investor}" (outbound) / "{sender} emailed
the team" (inbound) -- instead of an ambiguous "Sent"/"Received". Outbound
detection also matches our corporate domain (public providers excluded), so a
teammate's mail from a non-enrolled @ten31.xyz address no longer reads as
"Received". Going-forward only; no schema change. The card drops its bare
direction label since the note now carries the relationship.
Tests updated; 30/30 green, render-smoke green.
room_send to a room the bot is only invited to (not joined) fails M_FORBIDDEN;
join explicitly on startup (idempotent if already a member). Bot-only change —
ships via the Spark git pull, no s9pk bump.
The email-capture "proposed grid notes" gain two review surfaces:
1. Inline source email — each proposed-note card on the Email Capture page
gets a "View email" toggle that lazily fetches the existing
GET /api/email/detail and shows from/to/cc/date/subject + scrollable body,
so a reviewer can judge the note against the email it was drafted from.
2. CRM->Matrix review bridge — the CRM (box, stdlib, no matrix-nio) can't post
to Matrix, so the intake bot (Spark) PULLS: GET /api/intake/email-proposals
returns to_post/open/to_close work-lists; the bot posts a review card
(metadata + snippet + draft note) to a dedicated review room
(MATRIX_EMAIL_REVIEW_ROOM) and relays in-thread yes / no / NL-edit
(POST .../{id}/decide, note revised via local Qwen). Decisions sync both
ways: web decide -> bot announces + closes the thread; Matrix decide -> the
web panel's ~25s poll clears the card. State lives CRM-side in the new
email_proposal_matrix side row (email-integration migration 0003, additive
+ idempotent CREATE TABLE IF NOT EXISTS), so it survives a bot restart.
Adds a 'bot' role (authenticated, never admin; require_bot_or_admin) to gate
the email-proposal endpoints rather than handing the bot full admin — the
principled base for the coming agentic capabilities. Role controls reach;
the draft->approve gate still controls autonomy (a human approves every write).
Deploy split: endpoints + migration + role + frontend ship in the s9pk; the
bot poll loop + review-room handling ship on the Spark. The bot's CRM user
must be flipped member->bot and joined to the review room (one-time).
Tests: backend/test_email_proposal_matrix.py + matrix_intake/test_email_proposals.py
(30/30 suite green, render-smoke green, migration verified twice on a DB copy).
The fundraising grid (canonical) now drives the classic opportunities
Pipeline board, instead of the board being a disconnected second data-entry
surface. An "Add to Pipeline" row action creates a durably-linked opportunity
via the new opportunities.fundraising_investor_id (migration 0005, additive +
reversible), reusing the grid's already-synced contact — retiring the
POST /api/contacts side-door — and mapping the grid lead to the opp owner.
Ownership is split so the two stay reconciled: the grid owns whether the link
exists and the seed; the board owns stage/probability/owner. The link endpoint
is idempotent (one live opp per investor; a re-link never reseeds funnel
fields). "Is in pipeline?"/"what stage?" are derived from a live opp join and
injected as read-only grid columns on read, stripped on write, so they never
persist or dirty the autosave. Remove-from-pipeline soft-deletes the opp and
leaves the grid row fully intact; deleting an investor from the grid archives
its orphaned opp.
Also fixes the standing soft-delete leak in handle_pipeline_report and the
dashboard pipeline aggregates, which counted tombstoned opportunities.
Tests: backend/test_grid_pipeline_link.py (link/idempotent/round-trip/guards/
unlink-intact/re-link/orphan/aggregates); 28/28 suite green, render-smoke green.
Local-smoke found "jonathan is chatting with wyoming" extracted the teammate, not
the prospect. Feed the parser an optional team roster (INTAKE_TEAM_ROSTER) via a
build_system(roster) outreach frame: roster names/initials are the people doing
outreach and are never extracted; the other party is the investor/prospect. Same
framing on the revise leg. Unset roster = prior behavior.
Turn the bot from a bare nohup process (silently dies on a Spark reboot) into a docker-compose service. Dockerfile bundles backend/matrix_intake + the stdlib backend/ingest Spark client it reuses; .env is mounted read-only at runtime, never baked. The existing repo-root .dockerignore (shared with the s9pk build) already keeps data/ and .env out of context. Also adds a handoff doc for wiring a spark-control dashboard card in a later session.
Close the two locked post-deploy enhancements for the Matrix intake bot.
Fuzzy matching (server-side, ships in the s9pk): new find_intake_candidates in
server.py returns ranked deterministic near-matches (difflib name similarity +
token-set Jaccard, legal-suffix-aware, + email Levenshtein <= 2); GET
/api/intake/match now returns {match, candidates}. The bot surfaces a numbered
shortlist so a near-duplicate (Charlie/Charles, Acme Capital vs Acme Capital LLC,
a one-char email typo) is confirmed by a human instead of silently creating a
second investor. Exact match still auto-attaches; fuzzy candidates are never
auto-attached. The optional LLM-judge re-rank is deferred.
Conversational edits (bot-side, ships on the Spark): any in-thread reply that
isn't yes/no/edit field=value is treated as a natural-language revision and
re-run through local Qwen (parse.revise). Email integrity is preserved -- a
changed address must literally appear in the instruction; the model's email
field is structurally unreachable. No-op revisions re-prompt.
Docs/current-state brought current; 27/27 backend tests green.
The grid note line was "YYYY-MM-DD [type] Contact: summary"; for the default
"note" type the tag is noise. Omit it for "note"; keep it for informative
types (call, meeting, …). Shared by the Matrix intake bot and grid-UI logging.
Built + installed to the box (installed-version 0.1.0:85, clean 84->85
migration). No schema change.
Four bot-side UX fixes surfaced by the live smoke:
- Post a brief pointer in the main timeline (a reply to the user's message)
alongside the in-thread proposal card, so proposals aren't missed inside a
thread. Pointer only — approvals still happen in the thread, where the note
is visible (you can't make an informed yes/no without seeing it).
- A bare yes/no typed in the main timeline while a proposal is pending now
gets a "reply in the thread" redirect instead of "couldn't tell what to record."
- Clearer commit confirmations: "Created a new grid entry for X" vs
"Logged a note on X (existing grid entry)."
- Send a blank communication subject when a note is present so the grid's
one-line note summary shows the note text, not the "(Matrix)" label
(provenance stays in source="matrix_intake").
normalize()'s email regex matched non-@/non-space runs, so "Name <addr>"
(the most common contact format) yielded "<addr"; only trailing punctuation
was stripped, never leading. Tighten the regex to standard local@domain.tld
so the bare address is extracted from <…>, (…), and trailing-period forms.
Found via the live-deploy pre-flight. Add a regression test.
Also log two intake backlog items in ROADMAP: the scoped service-credential
auth path (deferred; bot uses a member login for now) and fuzzy match +
in-thread confirm (post-deploy).
New backend/matrix_intake/ runs as its own process (matrix-nio isolated from the
stdlib CRM): local-Qwen parse via Spark Control → in-thread human approval
(yes/edit/no) → write through the CRM's own log-communication endpoint, tagged
source=matrix_intake. Adds read-only GET /api/intake/match (returns grid row id,
no-duplicate contract); threads provenance through handle_log_fundraising_communication.
Reviewer-passed: pop-before-commit closes a double-approve race; edit-grammar fix.
Text-only v1; business-card photo (M3) deferred (no Spark vision model).
26/26 tests green; live Matrix smoke pending deploy.
Communications tab (search/query roadmap items 1 & 2):
- Fix the investor dropdown: the facet only listed grid investors, so it
came back empty whenever email matched a classic contact or org domain
(no grid id — the common case). It now mirrors the email list, resolving
each link to a typed identity (fund:/org:/contact:/addr:) with precedence
grid -> org -> contact -> address; investor_id accepts the typed key
(bare id = fund: for back-compat) and an unknown prefix matches nothing.
- Add a date-range filter and a click-to-expand full-body view
(GET /api/email/detail, admin, soft-delete-gated; body_text only, never
raw remote HTML).
- Add a "Search content" mode: GET /api/email/search wraps the ingest
hybrid_search over the Qdrant email index (doc_type=email), hydrated and
soft-delete-filtered against SQLite (canonical), 503 if Spark/Qdrant down.
Daily digest:
- Settings -> Admin builds a digest over a chosen window (last 24h or since
a date) as an in-app preview before sending (POST /api/admin/digest/preview),
so the local-Spark summarizer can be verified on demand even on a quiet day.
Manual send uses the same window; neither advances the daily cursor, so a
preview never suppresses the scheduled digest.
Code-only, migrations no-op. 22/22 backend tests, render-smoke pass.
The email-activity panel surfaced every captured message, including cold/
unknown-sender email with no investor association. Gate query_email_activity
on EXISTS(email_investor_links) so the panel shows only email tied to a known
investor/contact. Capture is unchanged — unmatched email is still stored
(metadata-only) and will appear automatically if its sender is later added as
an investor; this is a read-side filter only.
Graveyard investors are unaffected (their email has a link), so they remain
visible/searchable as an audit surface, hidden only from the filter picker.
The Communications tab is now an admin-only search over captured Gmail
(email_* tables), part of consolidating on the fundraising grid + email
capture as the canonical system of record.
- New GET /api/email/activity (admin-enforced server-side): filter by
investor / mailbox / direction with free-text search over subject,
snippet, and sender. Query logic in db.query_email_activity.
- Soft-delete honored on the per-mailbox sighting (emails carry no
deleted_at; deletion lives on email_account_messages).
- Direction decided at the email level (outbound if the sender is one of
our mailboxes), mirroring digest_builder.
- Graveyard investors are hidden from the filter dropdown (CRM-wide
graveyard=0 convention) but their email stays visible in the list and
findable by free-text search — this is an audit surface.
- Communications page rewritten to render the panel; the classic manual
"Log Communication" form is retired (the grid context menu remains the
manual-log path). Nav item + page are admin-only.
- Tests: email_integration/test_email_activity_panel.py (filters,
per-sighting soft-delete, roll-ups, graveyard handling, route 401/403);
full suite 22/22. Frontend render verified via a jsdom mount smoke test
plus the pinned classic-runtime Babel transform.
Code-only, no schema migration (version migrations are no-ops).
The web UI rendered a blank screen for every user. Root cause: the page
loaded @babel/standalone from unpkg with no version pin, so the CDN silently
served Babel 8.0.0. Babel 8 defaults @babel/preset-react to the automatic JSX
runtime, which prepends `import {jsx} from "react/jsx-runtime"` to the compiled
output. An ESM import is illegal in this classic (non-module) inline <script>,
so the browser rejected the whole bundle and React never mounted — hence the
blank screen. The prior "verified live" checks were server-up/curl, which can't
catch a browser-render failure.
- Pin @babel/standalone@7.29.7 (its preset-react defaults to the classic
React.createElement runtime). Verified via headless render: app mounts, login
screen renders, no console error. Follow-up: vendor + SRI-pin the CDN libs so
a third party can't swap our front-end deps in production again.
- Close three server-side admin gaps surfaced by a permissions audit — endpoints
that were UI-hidden from members but not API-enforced: GET /api/users,
/api/email/status, /api/email/accounts now require_admin. Removed the now-dead
non-admin mailbox-row filter. 21/21 backend tests green; py_compile clean.
The fundraising grid + email capture is the canonical system of record. lp_profiles
was a superseded single-fund model with no reachable create/edit path, and the LP
Tracker page was already orphaned (no nav entry + a redirect bouncing it to the grid).
- Remove /api/lp-profiles* endpoints + handlers, the unused lp-breakdown report,
the contact-dossier LP section, the demo-seed LP block, and (frontend) the
LPTrackerPage component + its lp-tracker->fundraising-grid redirect.
- Dashboard "Total Committed" now sums fundraising_investors.total_invested
(graveyarded investors excluded) instead of the orphaned lp_profiles table, which
read ~$0. "Total Funded" dropped: the grid tracks commitments, not a funded amount,
and the frontend never rendered it.
- Leave the empty lp_profiles table/index, the contact-delete soft-delete cascade,
and the --reset-all-data clear in place (never-hard-delete).
- Tests: add test_dashboard_report.py; update test_soft_delete_reads.py. 21/21 green.
Sends a once-a-day internal email to all active admins summarizing each team
member's email activity per investor, plus a team-wide by-investor view
(inbound + outbound, deduped). Narratives are generated on the LOCAL Spark
model, never Claude — the digest is intentionally un-anonymized, so substance
stays on Ten31 infra. This is an internal ops email, exempt from the
'agents draft, humans send' rule (which governs outward LP contact).
- backend/digest_builder.py: per-user + per-investor activity queries
(soft-delete filtered), per-user Spark narrative with a deterministic
fallback, two-section plain-text body, and the DB-backed policy resolver.
- backend/email_integration/digest_scheduler.py: always-on daily thread that
re-reads the policy each cycle and sends once/day; window cursor in
app_settings so a missed day rolls forward.
- server.py: POST /api/admin/digest/send-now and GET/PATCH
/api/admin/digest/policy; scheduler wired into main().
- Control lives in Settings -> Admin (enable toggle + send-time dropdown),
not StartOS actions; env vars only seed the first-boot default.
- Tests: backend/test_digest_builder.py.
Non-blocking items from the v76 reviewer pass. No redeploy needed — the box runs
v76 and its happy path is unaffected; these ride the next build:
- digest_mailer.send_digest: when Gmail is enabled but no sender resolves
(CRM_DIGEST_SENDER unset and no admin email), raise NoTransport so the caller
returns a clear 400 instead of a generic 502.
- gmail_send.send_via_gmail: wrap OSError/URLError (timeout/DNS) as a RuntimeError
("Gmail API unreachable: ...") to match the HTTPError handling; include the
sender in the HTTPError message for debuggability.
- credentials.py: correct the now-stale GMAIL_COMPOSE_SCOPE comment (the digest
mailer sends with this scope; only outreach drafts never send).
- test_gmail_send.py: add the HTTPError->RuntimeError branch, default_sender DB
fallback (+None case + env override), and the send_digest SMTP-tag path.
19/19 backend tests green.
The box's existing service-account domain-wide-delegation grant already includes
gmail.compose, which authorizes users.messages.send — verified 2026-06-15 by a
token-mint probe and a live messages.send to grant. So CRM-originated mail can
send through the account that already powers email capture: no SMTP account, no
app password, no admin change.
- backend/email_integration/gmail_send.py: send_via_gmail() impersonates a
domain user and POSTs users.messages.send (reuses credentials.py + the compose
scope; mirrors compose.py's REST pattern).
- backend/digest_mailer.py: send_digest() prefers Gmail DWD when enabled, falls
back to smtp_send otherwise. Sender = CRM_DIGEST_SENDER else first active admin.
- server.py: the admin test endpoint now routes through digest_mailer (so the
Settings button sends via DWD on the box with zero SMTP config). Recipient
restriction to the admin set and no-leak error handling preserved.
- test_gmail_send.py: build/send + transport routing (provider + urlopen faked).
19/19 backend green; s9pk typechecks.
SMTP (v75) stays as the fallback transport. Send-path decision + scope finding
recorded in ROADMAP.md and AGENTS.md.
Groundwork for the daily activity digest: give the CRM an outbound mail path.
Today nothing leaves the box (Gmail capture + drafts only), so this adds a
dedicated, per-package SMTP account independent of any StartOS system-wide SMTP.
- configureDigestSmtp Start9 action: writes host/port/from/username/password/
security to /data/secrets/smtp/* (password piped over stdin, never argv/env;
per-field files, owner-only) — mirrors the setAnthropicApiKey pattern.
- docker_entrypoint.sh reads those at boot and exports SMTP_* (operator env wins).
- backend/smtp_send.py: stdlib smtplib wrapper reading SMTP_* (one code path for
dev .env and the box); starttls/tls/none modes.
- POST /api/admin/digest/test-email (admin-only): proves the pipe. Recipients are
restricted to the active-admin set — an arbitrary `to` is rejected, so the
endpoint is not an open relay; send failures are logged, not echoed (an SMTP
auth error can carry the credential).
- Tests: test_smtp_send.py (sender), test_smtp_endpoint.py (gating + relay
restriction + no-leak). 18/18 backend green; s9pk typechecks.
Analysis/summarization for the digest body (Phase B) will run on Spark, never
Claude — the digest is deliberately un-anonymized. Decisions + Phase B plan in
ROADMAP.md.
Lock in the three v0.1.0:74 security/privacy fixes with regression tests, and
fix a same-class soft-delete leak surfaced while writing them.
- backend/test_assets_traversal.py: boots the real server, proves /assets/
path-traversal vectors (incl. a real decoy file and the live crm.db, plain
and URL-encoded) 404 and leak nothing, while a legit asset still serves 200.
- backend/test_soft_delete_reads.py: get-by-id 404s soft-deleted rows and
nested + list-view aggregates exclude soft-deleted children.
- backend/mcp/test_outreach_redaction.py: an unknown free-prose name is
tokenized away from the Claude payload but re-hydrated locally, and the path
fails closed (no Claude call) when the local NER model is down.
- backend/run_tests.py: aggregate runner (each backend/**/test_*.py in its own
subprocess); replaces the manual for-loop. 16/16 green.
A reviewer pass on the tests confirmed the soft-delete filter was missing from
list-view aggregate sub-selects: org contact_count/total_funded and contacts
comm_count/last_contact_date counted soft-deleted rows. Add `deleted_at IS NULL`
to those four (server.py) and regression-cover them.
The reports subsystem (dashboard/pipeline/LP-breakdown, ~16 aggregate queries)
has the same leak and is logged as P2 for a dedicated pass. Not yet built or
deployed — bump the package version before the next s9pk build.
ensure_positioning_framings adds 5 Architect framings to the core
positioning variant group alongside Option A/B, so the group holds 7
candidates and choose_variant retires 6. The two thesis tests still
asserted the pre-framings count of 2 — the tests were stale, not the
seed. Realign them, document the 2+5=7 seed structure in the thesis
guide, and refresh AGENTS.md Current state (13/13 tests green).
Keysat