ten31-database/AGENTS.md

Ten31 Venture CRM + Agentic System — AGENTS.md

The foundation is a self-hosted venture-fund CRM — a purpose-built fundraising tool that replaced Airtable to (1) keep sensitive LP/prospect data off third-party servers, (2) drop subscription cost, and (3) fit the fund's workflow: managing ~150 existing LPs, tracking 250+ prospects, and running the capital-raise pipeline. Core CRM domain: contacts (investor/prospect/advisor), organizations, opportunities (the deal pipeline), and communications; investor commitments live in the canonical fundraising_* grid (the legacy single-fund lp_profiles table was retired in v0.1.0:78). The fund (Ten31, ~$200M AUM, bitcoin/energy/AI thesis) runs it on a Start9 box, accessed over ClearNet (StartOS StartTunnel) with app-level user auth by a team of ~5 (Tailscale is not in use). Schema/API tour: docs/crm-overview.md.

The agentic system is new functionality built on top of that CRM — an in-house AI layer to widen the fundraising funnel, sharpen the thesis, and automate outreach drafting. Frontier reasoning runs on Claude (Agent SDK/API); privacy-sensitive and bulk work runs on local DGX Spark models via the Spark Control gateway. Phase 0/1 — no live outward-facing agents; agents draft, humans send.

Inbox check: At session start, if ~/Projects/standards/INBOX.md exists, scan it for items tagged (CRM) and surface them before proposing next steps; triage with /triage.

Stack (versions that matter)

• Python 3.11, standard library only at runtime. The CRM is one monolith, backend/server.py (~5k lines): a stdlib http.server.ThreadingHTTPServer + hand-written CRMHandler with manual path dispatch (do_GET/do_POST). Not FastAPI. backend/requirements.txt lists FastAPI/SQLAlchemy/Alembic/Pydantic/pytest-style deps but none are imported at runtime (vestigial).
• SQLite at data/crm.db (WAL, foreign_keys=ON), opened per-request via get_db(). Schema via ordered migrations.
• Frontend: single frontend/index.html, inline-Babel React. No build step.
• Optional runtime deps, used only if present: bcrypt, PyJWT (jwt), cryptography (Gmail module).
• MCP + ingest (in the Docker image, not the bare CRM): mcp==1.2.0 (FastMCP, backend/mcp/server.py), fastembed==0.4.2, anthropic, cryptography==42.0.5.
• Packaging: StartOS 0.4, TypeScript SDK (@start9labs/start-sdk) under start9/0.4/startos/. Live target is start9/0.4/.
• Local models (bge-m3 embeddings, bge-reranker-v2-m3, /api/search, Qdrant): always via Spark Control. Contract: docs/EMBEDDINGS.md.

Commands

# Run locally (dev, port 8080; or ./start.sh <port>) — runs python3 backend/server.py
./start.sh
# Run prod-mode (beta) — requires CRM_SECRET_KEY
./start_beta.sh
# Sanity-check edits (there is no compiler/build for the CRM)
python3 -m py_compile backend/server.py
# Run ONE test (tests are standalone scripts with `if __name__ == "__main__"`; no pytest installed)
python3 backend/redaction/test_scrub_leak.py        # substitute any backend/**/test_*.py
# Run all tests (aggregate runner — runs each backend/**/test_*.py in its own subprocess)
python3 backend/run_tests.py                         # add substrings to filter, e.g. `... soft_delete redaction`
# Build + install the s9pk — BUMP THE VERSION FIRST. See docs/guides/packaging.md.
cd start9/0.4 && make

• Migrations apply automatically at startup (backend/core_migrations.py, schema_migrations ledger). See docs/guides/migrations.md before adding one.
• Lint: none configured.

Directory layout (day-one)

• backend/server.py — the CRM monolith: HTTP handler, route dispatch, init_db(), auth (username/password → HS256 JWT, roles admin/member/bot).
• backend/core_migrations.py + backend/migrations/NNNN_*.sql (+ paired .down.sql) — additive schema migrations, applied at startup.
• backend/thesis_seed.py — Thesis Workshop seed + idempotent ensure_* one-time seeders, wired in server.init_db().
• backend/thesis_review.py — thesis version review/approval (human dual sign-off → canonical).
• backend/mcp/ — architect_agent.py (Claude thesis copilot), architect_tools.py, outreach_agent.py (LP draft assistant), architect_grounding.py, crm_tools.py, server.py (FastMCP).
• backend/email_integration/ — Gmail capture via domain-wide delegation + Tier-B draft creation (compose.py).
• backend/redaction/ — scrub.py + client.py: the scrub→Claude→re-hydrate privacy boundary.
• backend/ingest/ — chunk→embed→Qdrant + retrieval modes.
• backend/entity_*.py — entity resolution/merge (the two-investor-model reconciliation).
• backend/nl_query/ — read-only natural-language query (W2): intents.py (curated parameterized query catalog), runner.py (slot validator = trust boundary), translate.py (local-Qwen question→{intent,slots}). See the nl-query guide.
• backend/matrix_intake/ — Matrix intake bot (separate process; matrix-nio, isolated to this component): typed message → local-Qwen parse → in-thread approve → write via the CRM's own log-communication. See the matrix-intake guide.
• frontend/index.html — the entire UI.
• docs/ — architecture, phase plans, contracts, runbooks (see Deeper docs). docs/guides/ — scoped subsystem rules (see below).
• start9/0.4/ — StartOS package (startos/utils.ts holds PACKAGE_VERSION).
• data/crm.db — the live DB (gitignored). .env / .env.example — config (.env gitignored).

Scoped guides

Subsystem rules live in docs/guides/ and lazy-load in Claude Code via .claude/rules/ symlinks (scoped by paths: frontmatter). Read the guide before editing that area:

• Migrations or seeders (backend/migrations/, core_migrations.py, thesis_seed.py) → docs/guides/migrations.md
• Thesis logic (backend/thesis_*.py, backend/mcp/architect_*.py) → docs/guides/thesis.md
• Redaction or any MCP/Claude path (backend/redaction/, backend/mcp/) → docs/guides/redaction.md
• Ingest / retrieval (backend/ingest/) → docs/guides/spark-ingest.md
• Email capture / drafts + digest send (backend/email_integration/, backend/digest_mailer.py, backend/smtp_send.py) → docs/guides/email.md
• Building or deploying the s9pk (start9/) → docs/guides/packaging.md
• Matrix intake bot (backend/matrix_intake/) → docs/guides/matrix-intake.md
• Natural-language query (backend/nl_query/) → docs/guides/nl-query.md

Conventions

• Investor model — the grid is canonical (since v0.1.0:78). The fundraising_* grid is the system of record: an investor entity (row) → many contact "pills" → per-fund commitments. The classic contacts table is a read-only per-person directory, auto-populated from the grid — create/edit people in the grid, not the Contacts page. Email capture rolls multiple people up to one investor. The legacy single-fund lp_profiles model is retired (empty table kept, per never-hard-delete). Reconciling grid ↔ classic contacts to canonical IDs is the core entity-resolution task — see docs/crm-overview.md. Derived read-only columns (pipeline, pipeline_stage, opportunity_id, reminder_status, existing_investor, last_activity_at, staleness) are computed live and injected on GET, never persisted — any new one MUST be added to BOTH strip points (server.py _computed_row_values + frontend stripComputedRows) or it dirties the autosave / leaks into the blob. Pipeline stage is the 4-stage funnel lead→engaged→diligence→commitment (PIPELINE_STAGES), terminal at commitment.
• Soft-delete only: deleted_at and/or status='retired'; never hard-delete. Every READ path must filter deleted_at IS NULL — list handlers, get-by-id, nested related-data sub-selects, and aggregate sub-selects (COUNT/SUM/MAX). Audits found leaks in all of these (2026-06-12 detail + nested; 2026-06-13 list-view contact_count/total_funded/comm_count); the opportunities/pipeline aggregates were fixed in v0.1.0:87 (handle_pipeline_report + dashboard pipeline metrics now filter deleted_at), but the reports subsystem's communications-side aggregates (dashboard recent_comms/comms_this_month/meetings_this_month, activity report) still leak (see Current state). Regression-guarded by backend/test_soft_delete_reads.py (+ test_reminders.py for the reminders read paths, incl. the recency rollup whose email-activity liveness signal is email_account_messages.deleted_at, not emails). (Thesis has a subtlety here — see the thesis guide.)
• Env: secrets in .env (gitignored); names in .env.example. Verified names: ANTHROPIC_API_KEY, SPARK_CONTROL_URL, SPARK_CONTROL_VERIFY_TLS, QDRANT_URL, X_API_KEY, CRM_DB_PATH, CRM_DEV_DB_PATH. Also used: CRM_SECRET_KEY (beta/prod), CRM_HOST/CRM_PORT, CRM_DATA_DIR; digest mailer: CRM_DIGEST_SENDER (DWD impersonation sender) + SMTP_HOST/SMTP_PORT/SMTP_SECURITY/SMTP_FROM/SMTP_USERNAME/SMTP_PASSWORD (SMTP fallback); daily digest (Phase B): CRM_DIGEST_ENABLED + CRM_DIGEST_SEND_HOUR only seed the first-boot default — the live control is the DB policy (app_settings.digest_policy, set in Settings → Admin).
• Config placement: operational/feature toggles live in the admin panel, DB-backed via app_settings (read-merge through a load_*_policy(conn) helper shared by the API + any scheduler; precedence DB-row → env-seed → default), so they're discoverable and take effect live. Reserve StartOS actions / env for secrets and deploy-time config (SMTP creds, API keys, DWD sender). Precedent: digest_policy (GET/PATCH /api/admin/digest/policy), fundraising_backup_policy.
• Agent/bot API access — three roles now (admin/member/bot). require_admin is the only hard gate; everything else is "authenticated" (member, admin, and bot all pass). The bot role (added v0.1.0:89) is authenticated-but-never-admin: require_bot_or_admin gates agent-facing endpoints (e.g. /api/intake/email-proposals*) so a bot credential reaches only what it needs, never user-management/settings/security. Provision it via Settings → Admin edit-user dropdown (kept out of the teammate-invite form). Two axes to keep separate as more agent capability lands: the role controls reach (which endpoints); the per-feature human draft→approve gate controls autonomy (acting unattended). Money/merge/delete mutations stay behind the approval gate regardless of role. Don't build a finer capability/scope system until real NL-mutation endpoints exist to scope against.
• Design: before building or changing any user-facing UI, read design/DESIGN.md and design/tokens.tokens.json and conform to them. The mobile-first redesign landed (Claude Design round-trip distilled into the contract 2026-06-19): the authority for mobile/responsive work is DESIGN.md §8 + the tokens mobile and color.light groups; design/BRIEF.md is the input brief and design/_imports/2026-06-19/ the provenance + per-surface interaction reference (the comps are Claude Design runtime prototypes — re-author each surface in the app's React idiom + real API, not drop-in; the design source of truth is each *.dc.html at its DEFAULT data-props (compact/dark/plex/earmark — see GridApp.dc.html data-props), NOT the screenshots/ PNGs, which are option-history (rejected/stale combos: INVESTOR/PROSPECT disposition badges, 6-stage MEETING/FUNDED funnel, star flag). Don't anchor on the screenshots (cost a re-scope 2026-06-19; general learning in standards/guides/design.md Phase C)). A light theme is built (P6): it lives in :root[data-theme="light"] (set by a pre-paint boot script from localStorage.venture_crm_theme; dark is the default), with an app-wide toggle in the desktop sidebar footer + the mobile top bar. Colors are theme vars now — any new UI color MUST use a :root var (grow the set if needed), never a literal, or it won't flip in light (chips/badges flip via .stage-chip--{stage}/.due-chip--{overdue,today,week,later} + the --chip-*/--note-*/--badge-priority-*/--rem-*/--due-{overdue,today,week,later}-*/--money/--recency-*/--due-soon slots; authoritative dark+light pairs are in the Claude Design export design/_imports/2026-06-19_zip-file/ store.js + *App.dc.html). Mobile light is complete; desktop has known unthemed shades (Phase 7). (Note: inline style={{}} objects can't respond to media queries; responsive layout belongs in the CSS <style> block. The mobile foundation primitives are built — CSS: .bottom-tab-bar, the .bottom-sheet primitive, .mobile-only/.desktop-only, :root mobile vars; React (Phase 2): <BottomSheet> (scrim/Escape/drag-to-dismiss) + useIsMobile() (768px) + the MobileDetailRow/.fs-detail full-screen-detail + .contact-card/.az-header list patterns — build new mobile surfaces on these (P3 Grid reuses them directly; swap surfaces via a rules-of-hooks-safe useIsMobile() wrapper that mounts a Mobile*/Desktop* pair, never a per-component hook toggle). The inline-style→CSS migration is scoped, per-surface (~114 styles across 4 surfaces+shell, not ~1,300), folded into each surface's build; see ROADMAP.md.) Phase 8 card/detail primitives (reuse these, don't reinvent): EarmarkCorner (existing-LP corner triangle; inline variant for the org card), the priority-pill/lp-pill text pills, StageChip (+ sm), NoteTimeline, LogCommunicationSheet, SortPill/SortSheet (mono pill + label+hint option sheet, 8d — drive with a per-surface {id,pill,label,hint} table, e.g. GRID_SORTS/PIPELINE_SORTS/SORT_OPTIONS), MobileQuickLog (shell top-bar quick-log pencil), DueChip (8e — urgency-bucketed reminder due pill, .due-chip--{bucket} colors); <BottomSheet> takes a stacked prop to layer a sheet opened over another sheet (e.g. the Log sheet over a detail, or the 8e investor-picker over the add sheet). Mobile reminder writes link a real investor via source_row_id → server-resolved investor_id (the 8e add-flow picker; create POSTs source_row_id, never a free-text name — the old label never linked); PATCH /api/reminders can't reassign the investor (edit shows it read-only). The mobile Contacts + Pipeline detail surfaces are drag-dismiss bottom sheets (8b) that log via POST /api/communications; the Grid detail stays full-screen (its dc default) and reads its notes timeline via GET /api/communications?source_row_id=<grid row id> (investor-level: maps grid row → fundraising_investors.source_row_id → fundraising_contacts.contact_id → comms, soft-delete-respecting). The Contacts read path injects derived read-only committed + pipeline_stage + priority (contact_grid_signals() — existing-LP ring + stage pill + Priority sort) on both GET /api/contacts and /api/contacts/{id}; this needs no strip-point (the directory is read-only, never written back as a row) — unlike the grid's injected columns. The opportunities list (GET /api/opportunities) likewise injects derived read-only existing_investor (same contact_grid_signals committed>0 → the 8f Pipeline-card earmark, agreeing with the detail "Existing LP" pill) + last_contact_date (MAX(communication_date) on the deal's contact, deleted_at-filtered → card recency + Staleness sort); no strip-point (opp writes use a field allowlist, never a blob PUT). Phase 8f Pipeline-card primitives (reuse, don't reinvent): the card reuses EarmarkCorner/priority-pill/StageChip/recency; new patterns are the horizontal-scroll segmented stage control (.pipeline-seg-tab--{stage} whose .active tint comes from --seg-{bg,text,border} aliased to that stage's --chip-* vars) + label-with-count-badge, the labeled ‹ Prev·Next › card move footer, and tappable page-dots (active = 22px accent bar). Phase 8g add-investor primitives: the mobile "New investor" sheet gained an optional .stage-pick chip picker (a "Not in pipeline" button + the 4 StageChips; default off — Grant) and a framed .sheet-toggle-opt Priority toggle, plus an optional reminder (title + progressive due-date). Submit (submitCreate) orchestrates one-row calls in order — create (log-communication, which now honors an optional priority only on its create-if-missing branch) → optional pipeline link at the picked stage → optional POST /api/reminders keyed on the new row's source_row_id; the link/reminder steps are non-fatal.
• Commit style: imperative subject, concise body explaining the why; put the package version in the subject (… (v0.1.0:NN)) for shippable changes. No AI co-author / attribution trailers — commits are authored by the user.

Always

• Verify before shipping: python3 -m py_compile the edited files; for DB logic, run the change against a copy of data/crm.db, never production.
• Keep real LP data out of Claude: develop only on code/schema/synthetic-or-locally-redacted data; route any real record substance through backend/redaction first.
• Get explicit user authorization before any production deploy/install to $START9_BOX_HOST.

Never

• Never treat Qdrant (or any derived index) as source of truth — the CRM/SQLite is canonical and rebuildable-from.
• Never hard-delete CRM records or thesis history — soft-delete/archive only.
• Never let an agent send email, post, or contact an LP autonomously — agents draft; a human approves and sends.
• Never set a thesis_version canonical from code/seeds — that is human dual sign-off.
• Never call a Spark directly — go through Spark Control (SPARK_CONTROL_URL).
• Never commit secrets, data/crm.db, .env, or data/backups/ (all gitignored). Scan staged files before committing. (.claude/ is tracked — launch.json and rules/ symlinks ship with the repo; keep local-only settings in .claude/settings.local.json.)
• Never bulk-export the LP list to any third party; send only minimal non-sensitive context to Claude.
• Never assume FastAPI / SQLAlchemy / pytest are in play — they sit in requirements.txt unused; runtime is stdlib + SQLite.
• Never add a Co-Authored-By / "Generated with" trailer to commits or PRs — commits are the user's.

Deeper docs

• Full constitution + guardrails: docs/ten31-constitution.md
• Architecture & rationale: docs/Ten31_Agentic_Build_Plan.md
• Retrieval/embeddings contract: docs/EMBEDDINGS.md
• CRM schema/API tour: docs/crm-overview.md
• Current thesis handoff: docs/thesis-handoff.md
• Operations & runbooks: docs/OPERATIONS.md, docs/go-live-runbook.md, docs/gmail-enablement-runbook.md

Current state

Box live at v0.1.0:94; main ahead by mobile Phases 0–7 + P3b + drag-reorder + 8a–8g — all deploy-pending (no s9pk built). The fundraising grid + email capture is the canonical system of record. Active thread: mobile-first redesign → Phase 8, building to design/phase8-conformance.md (the 8a–8i spec, anchored on each *.dc.html DEFAULT data-props — NOT the screenshots/ PNGs). Plan (Grant, 2026-06-19): finish features first → then Grant device-tests + deploys (nothing verified on a real phone). History: git log + start9/0.4/startos/versions/.

• Mobile redesign — 4 core surfaces built (Grid · Contacts · Pipeline · Reminders), each a rules-of-hooks-safe useIsMobile() → Mobile*/Desktop* pair (desktop untouched). Foundation: bottom-tab bar + :root mobile vars; 4-stage enum; derived grid signals injected-on-GET/stripped-on-write at both points; mobile writes use one-row endpoints only (log-communication, pipeline link/stage, reminders, update-row) — never whole-grid PUT.
• Phase 8 done so far (8a–8f): cards re-authored (existing-LP earmark/avatar ring, PRIORITY/Existing-LP pills, 4-stage chip, recency; disposition badges dropped); Contacts + Pipeline detail → drag-dismiss bottom sheets (email-copy pill, Log/Email, Org card, stat tiles, inline move-stage, notes timeline); Grid-detail notes timeline + shared LogCommunicationSheet; top-bar quick-log pencil (MobileQuickLog, all tabs); sort controls on Grid/Pipeline/Contacts (shared SortPill/SortSheet; Contacts type-tabs dropped + Priority sort); 8e Reminders re-bucketed — Active/Done/All tabs dropped → title + urgency summary line + gradient add; 4 buckets Overdue/Today/This-week/Later each with a colored dot; urgency-colored DueChip pill (--due-* theme vars) replaces plain due text; collapsible Completed section (done+cancelled, strikethrough, tap-check reopens); card now note + org + due-chip (assignee dropped from card, still in edit sheet); swipe-right → snooze preset sheet (+1/+3/+7/+14d, dc-style) replacing the old fixed +7d; add-flow investor picker — a stacked searchable sheet over the canonical grid investors that writes source_row_id → a real server-resolved investor_id link (replaces the old free-text label that never actually linked; "team task" = no investor). Per-phase detail in git log + the Design convention's primitives list.
• 8f Pipeline card → dc anatomy: segmented control is now horizontal-scroll pills (label + count badge; the active pill tints to its own stage color via --seg-*); cards re-authored to earmark corner + name + Priority pill / $amount · dot · recency / labeled ‹ Prev·Next › move footer (was name + contact·org sub + bare ‹/›); stage-column header is StageChip + investor count + committed sum; page dots tappable (active = 22px accent bar). Backend: the opportunities list injects derived read-only existing_investor (via contact_grid_signals committed>0 — agrees with the detail's "Existing LP" pill) + last_contact_date (MAX(communication_date) on the deal's contact, soft-delete-filtered — drives the card recency + the Staleness sort). No strip-point needed (opp writes use a field allowlist, never a blob PUT). Card amount stays expected_amount (the deal forecast); earmark is the orthogonal existing-LP flag.
• 8g add-investor stage + priority (+ reminder): the mobile "New investor" sheet now also captures an optional initial pipeline stage (a .stage-pick chip picker — defaults to "Not in pipeline", Grant's call: no surprise opportunity rows), a "Flag as Priority" toggle, and an optional reminder (title + progressive due-date). On submit submitCreate orchestrates one-row calls in sequence: create (log-communication create_investor_if_missing, now carrying priority) → if a stage was picked, link at that stage (reusing applyStage's idempotent link-then-PATCH) → if a reminder title was given, POST /api/reminders with the new row's source_row_id. The link/reminder steps are non-fatal (a failure toasts but never loses the created investor). Backend: handle_log_fundraising_communication honors an optional priority only on the create branch (existing-row logs never touch priority). Guarded by test_grid_add_investor.py (priority-on-create + defaults-False + the create→link and create→reminder handshakes on a freshly-synced row).
• Live (deployed): W2 NL query (v94); W1 reminders (v93); grid Pipeline (v88); Matrix intake + Gmail capture (DWD) + daily digest; Thesis/Architect (dual-approval); outreach — all draft-only.
• Tests: 40/40 backend green (python3 backend/run_tests.py; +test_grid_comm_timeline.py for the 8c timeline filter, +priority assertions in test_contacts_grid_signals.py; +8f opportunities-list last_contact_date/existing_investor soft-delete assertions in test_soft_delete_reads.py; +test_grid_add_investor.py for the 8g create flow), py_compile clean; 8c–8g surfaces interaction-verified via throwaway 375px jsdom harnesses (deleted after — 8f mounted MobilePipeline and 8g MobileFundraisingGrid against the vendored React/Babel + mocked api; 8g drove the create sheet end-to-end and asserted the create→link→reminder POST payloads). jsdom note: legacy ReactDOM.render doesn't fire onChange for synthetic input events — drive controlled inputs with react-dom/test-utils Simulate.change, not the prototype value-setter trick.
• Next — Phase 8, in order, build to design/phase8-conformance.md: 8h loose ends (incl. Grid detail G4/G5/G6 stage-card/reminder-card/timeline; "Open-in-Grid" deep-link-to-investor) → 8i shell SVG icons + ·Ten31· wordmark. Skip Pipeline accordion (Grant). Then (after feature-complete): deploy P0–P8 + P3b in one s9pk (authorize + version-bump first) and device-test light/dark on a phone.
• Open / risks: all mobile work + light theme built but never deployed or device-tested (smoke/jsdom only); MobileDetailRow now unused-but-retained (legacy-usage sweep); Pipeline detail "Committed" tile shows grid-committed not deal-expected (deal forecast in a footnote); W2 happy-path only; Claude/Architect path unverified live on the box; v2.0 reserve-asset spine not canonical; doc drift — crm-overview.md/EVALUATION.md still call lp_profiles live.