Read-only natural-language query over the curated nl_query endpoint, answered
in-thread. Two entry points (room-per-purpose model): a dedicated Q&A room
(MATRIX_QUERY_ROOM) where every top-level message is a question, plus the
?/@bot trigger in the intake room as a cross-room convenience. Both routes hit
the same handle_query -> crm_client.nl_query -> POST /api/query/nl; translation
runs on the box's local model, nothing leaves the box, and there is no write
path so no approval gate applies.
Pure logic (trigger parsing, answer rendering) in query.py with offline tests;
async room wiring in bot.py (live-smoke only, per the bot's convention).
Bot-side only, ships on the Spark via git pull + restart. Depends on the
box-side /api/query/nl endpoint, which lands with the v93 s9pk (reminders + W2):
until v93 is installed the Q&A surface 404s, so the bot deploy is staged to
follow that install.
Read-only "ask the database in plain English" backend. Translation runs on
the local Qwen via Spark Control (question -> {intent, slots}); nothing leaves
the box, no Claude and no redaction boundary (the simplification chosen after
pressure-testing). The safe surface is a curated catalog of ~12 hand-written
parameterized queries; a slot validator is the trust boundary (no generic SQL,
no dynamic identifiers). POST /api/query/nl + GET /api/query/catalog, gated
require_bot_or_admin, read-only, audited. Soft-delete-correct per table.
Local Qwen translated 12/12 real example questions correctly against the live
Spark. Web "Ask" box and Matrix bot still to come (steps 4-5).
Durable updates after the email-proposal review session:
- AGENTS.md: roles admin/member -> admin/member/bot; add a Conventions entry on
the bot role and the reach(role)-vs-autonomy(approval gate) principle.
- matrix-intake guide: rewrite the bridge section to final behavior (redact_thread
whole-thread redaction, the Element 'show deleted' client-setting dependency for
full clearing, the redact_resolved.py backfill tool, deploy gotchas).
- Current state rewritten lean (14->8 bullets); test count 27->30.
The email-capture "proposed grid notes" gain two review surfaces:
1. Inline source email — each proposed-note card on the Email Capture page
gets a "View email" toggle that lazily fetches the existing
GET /api/email/detail and shows from/to/cc/date/subject + scrollable body,
so a reviewer can judge the note against the email it was drafted from.
2. CRM->Matrix review bridge — the CRM (box, stdlib, no matrix-nio) can't post
to Matrix, so the intake bot (Spark) PULLS: GET /api/intake/email-proposals
returns to_post/open/to_close work-lists; the bot posts a review card
(metadata + snippet + draft note) to a dedicated review room
(MATRIX_EMAIL_REVIEW_ROOM) and relays in-thread yes / no / NL-edit
(POST .../{id}/decide, note revised via local Qwen). Decisions sync both
ways: web decide -> bot announces + closes the thread; Matrix decide -> the
web panel's ~25s poll clears the card. State lives CRM-side in the new
email_proposal_matrix side row (email-integration migration 0003, additive
+ idempotent CREATE TABLE IF NOT EXISTS), so it survives a bot restart.
Adds a 'bot' role (authenticated, never admin; require_bot_or_admin) to gate
the email-proposal endpoints rather than handing the bot full admin — the
principled base for the coming agentic capabilities. Role controls reach;
the draft->approve gate still controls autonomy (a human approves every write).
Deploy split: endpoints + migration + role + frontend ship in the s9pk; the
bot poll loop + review-room handling ship on the Spark. The bot's CRM user
must be flipped member->bot and joined to the review room (one-time).
Tests: backend/test_email_proposal_matrix.py + matrix_intake/test_email_proposals.py
(30/30 suite green, render-smoke green, migration verified twice on a DB copy).
Local-smoke found "jonathan is chatting with wyoming" extracted the teammate, not
the prospect. Feed the parser an optional team roster (INTAKE_TEAM_ROSTER) via a
build_system(roster) outreach frame: roster names/initials are the people doing
outreach and are never extracted; the other party is the investor/prospect. Same
framing on the revise leg. Unset roster = prior behavior.
Bot now runs as a docker-compose service on the Spark (verified live, listening). Docs (matrix-intake guide ops, ROADMAP, AGENTS Current state) updated. Also logs the live-smoke parse bug (teammate read as investor -> team-roster fix), the spark-control dashboard-card handoff, and the long-term dedicated-repo extraction.
Turn the bot from a bare nohup process (silently dies on a Spark reboot) into a docker-compose service. Dockerfile bundles backend/matrix_intake + the stdlib backend/ingest Spark client it reuses; .env is mounted read-only at runtime, never baked. The existing repo-root .dockerignore (shared with the s9pk build) already keeps data/ and .env out of context. Also adds a handoff doc for wiring a spark-control dashboard card in a later session.
Box installed to 0.1.0:86 (migration chain ...85->86 clean, candidates endpoint verified live); bot pulled + restarted on the Spark. Only the Matrix live-smoke remains.
Close the two locked post-deploy enhancements for the Matrix intake bot.
Fuzzy matching (server-side, ships in the s9pk): new find_intake_candidates in
server.py returns ranked deterministic near-matches (difflib name similarity +
token-set Jaccard, legal-suffix-aware, + email Levenshtein <= 2); GET
/api/intake/match now returns {match, candidates}. The bot surfaces a numbered
shortlist so a near-duplicate (Charlie/Charles, Acme Capital vs Acme Capital LLC,
a one-char email typo) is confirmed by a human instead of silently creating a
second investor. Exact match still auto-attaches; fuzzy candidates are never
auto-attached. The optional LLM-judge re-rank is deferred.
Conversational edits (bot-side, ships on the Spark): any in-thread reply that
isn't yes/no/edit field=value is treated as a natural-language revision and
re-run through local Qwen (parse.revise). Email integrity is preserved -- a
changed address must literally appear in the instruction; the model's email
field is structurally unreachable. No-op revisions re-prompt.
Docs/current-state brought current; 27/27 backend tests green.
New backend/matrix_intake/ runs as its own process (matrix-nio isolated from the
stdlib CRM): local-Qwen parse via Spark Control → in-thread human approval
(yes/edit/no) → write through the CRM's own log-communication endpoint, tagged
source=matrix_intake. Adds read-only GET /api/intake/match (returns grid row id,
no-duplicate contract); threads provenance through handle_log_fundraising_communication.
Reviewer-passed: pop-before-commit closes a double-approve race; edit-grammar fix.
Text-only v1; business-card photo (M3) deferred (no Spark vision model).
26/26 tests green; live Matrix smoke pending deploy.
Communications tab (search/query roadmap items 1 & 2):
- Fix the investor dropdown: the facet only listed grid investors, so it
came back empty whenever email matched a classic contact or org domain
(no grid id — the common case). It now mirrors the email list, resolving
each link to a typed identity (fund:/org:/contact:/addr:) with precedence
grid -> org -> contact -> address; investor_id accepts the typed key
(bare id = fund: for back-compat) and an unknown prefix matches nothing.
- Add a date-range filter and a click-to-expand full-body view
(GET /api/email/detail, admin, soft-delete-gated; body_text only, never
raw remote HTML).
- Add a "Search content" mode: GET /api/email/search wraps the ingest
hybrid_search over the Qdrant email index (doc_type=email), hydrated and
soft-delete-filtered against SQLite (canonical), 503 if Spark/Qdrant down.
Daily digest:
- Settings -> Admin builds a digest over a chosen window (last 24h or since
a date) as an in-app preview before sending (POST /api/admin/digest/preview),
so the local-Spark summarizer can be verified on demand even on a quiet day.
Manual send uses the same window; neither advances the daily cursor, so a
preview never suppresses the scheduled digest.
Code-only, migrations no-op. 22/22 backend tests, render-smoke pass.
Record the v82 vendor+SRI + render-smoke work in durable docs: packaging guide
gains the verified-build gate + re-vendor instructions; Current state rewritten
and compressed for v82; ROADMAP logs the deferred pre-compile-JSX alternative.
The email-activity panel surfaced every captured message, including cold/
unknown-sender email with no investor association. Gate query_email_activity
on EXISTS(email_investor_links) so the panel shows only email tied to a known
investor/contact. Capture is unchanged — unmatched email is still stored
(metadata-only) and will appear automatically if its sender is later added as
an investor; this is a read-side filter only.
Graveyard investors are unaffected (their email has a link), so they remain
visible/searchable as an audit surface, hidden only from the filter picker.
Docs-only: packaging guide notes start-cli install is silent on success (verify
with installed-version/logs); AGENTS.md adds the operational-toggles-in-the-admin-
panel convention and tightens the digest Current state.
Sends a once-a-day internal email to all active admins summarizing each team
member's email activity per investor, plus a team-wide by-investor view
(inbound + outbound, deduped). Narratives are generated on the LOCAL Spark
model, never Claude — the digest is intentionally un-anonymized, so substance
stays on Ten31 infra. This is an internal ops email, exempt from the
'agents draft, humans send' rule (which governs outward LP contact).
- backend/digest_builder.py: per-user + per-investor activity queries
(soft-delete filtered), per-user Spark narrative with a deterministic
fallback, two-section plain-text body, and the DB-backed policy resolver.
- backend/email_integration/digest_scheduler.py: always-on daily thread that
re-reads the policy each cycle and sends once/day; window cursor in
app_settings so a missed day rolls forward.
- server.py: POST /api/admin/digest/send-now and GET/PATCH
/api/admin/digest/policy; scheduler wired into main().
- Control lives in Settings -> Admin (enable toggle + send-time dropdown),
not StartOS actions; env vars only seed the first-boot default.
- Tests: backend/test_digest_builder.py.
Extend docs/guides/email.md paths: frontmatter (and its AGENTS.md index entry) to
include backend/digest_mailer.py and backend/smtp_send.py, so the guide auto-loads
when editing the outbound-digest send path — not just backend/email_integration/**.
Portability-checker: compliant.
- docs/guides/email.md: new "Outbound mail — the daily digest" section (Gmail-DWD
primary → SMTP fallback; gmail.compose send capability; the internal-digest
exemption from the agents-draft rule).
- AGENTS.md: add digest env names (CRM_DIGEST_SENDER, SMTP_*); consolidate the
v75/v76 deploy bullets into one current bullet; drop finished v74 narrative.
thesis-seed v1–v4 are superseded by v5 (the version seeded by
thesis_seed.py) and had no inbound references. refresh_seed.sh and
seed/README.md are 0.3.5-era seed-snapshot helpers the 0.4 entrypoint
no longer uses (DEPLOY_040 labels both LEGACY). data/test_write was a
stray 0-byte write-probe. Folder-rename housekeeping; no runtime change.
ensure_positioning_framings adds 5 Architect framings to the core
positioning variant group alongside Option A/B, so the group holds 7
candidates and choose_variant retires 6. The two thesis tests still
asserted the pre-framings count of 2 — the tests were stale, not the
seed. Realign them, document the 2+5=7 seed structure in the thesis
guide, and refresh AGENTS.md Current state (13/13 tests green).
Fixes from the 2026-06-12 full-eval (P0 + two P1s); code-only, no schema
change. Without these the "private CRM" premise was breachable on the LAN:
- P0: the /assets/ route joined the request path onto FRONTEND_DIR without
normalizing '..' (get_path/urlparse pass it through), so an unauthenticated
GET /assets/../../data/crm.db read any file the process could — the LP DB,
the JWT signing secret (-> admin-token forgery), the Gmail key. Add a realpath
containment check that 404s anything resolving outside FRONTEND_ROOT.
- P1: the LP-outreach drafter built its redaction Boundary with no ner_fn, so
unknown people/firms in raw email bodies reached Claude in the clear. Pass the
local-Qwen NER backstop (ner_fn=_ner_local), matching architect_grounding;
fails closed via the existing scrub_unavailable path if the local model is down.
- P1: get-by-id handlers leaked soft-deleted records by direct ID. Add
deleted_at IS NULL to every get-by-id path — contacts, organizations,
opportunities, lp_profiles — and to the nested related-data sub-selects in
the contact/opportunity detail payloads, matching the list-handler convention.
Bumps the package to v0.1.0:74 (utils.ts + versions/v0.1.0.74.ts + graph).
Full report in EVALUATION.md; remaining P2/P3 triaged in AGENTS.md Current state.
Move subsystem mechanics (migrations, thesis gate, redaction, ingest,
email, packaging) out of AGENTS.md into docs/guides/<topic>.md, each
scoped by paths: frontmatter and symlinked from .claude/rules/ so Claude
Code lazy-loads them. AGENTS.md keeps whole-repo facts and universal
guardrails plus a one-line index per guide. Fix the inaccurate
".claude/ is gitignored" note — it is tracked.
Add a concise day-one AGENTS.md (stack, exact build/run/test/deploy
commands, directory layout, conventions, Always/Never). Preserve the
existing CLAUDE.md project constitution as docs/ten31-constitution.md
(referenced from AGENTS.md) and point CLAUDE.md -> AGENTS.md so Claude
Code loads the canonical guide.
Swap the dead "scarcity as the connecting idea" / bitcoin-as-settlement
spine for the v2.0 reserve-asset spine (bitcoin = apex non-debasable
reserve asset; debasement = forcing function; AI = abundance engine;
throughline is an asset-value/capital-flow claim, not settlement; three
seams Energy<->Compute, Debasement<->Bitcoin, AI<->Data-Ownership)
everywhere it was still encoded in live code, the seed, and the docs.
- architect_agent.py / outreach_agent.py: both system prompts carried
"scarcity as the connecting idea" and shipped settlement framing into
every generated draft; rewritten to the reserve-asset spine.
- thesis_seed.py: THROUGHLINE, PILLAR_1, the AI/energy-operator segment
angle, and THESIS_V2 corrected and voice-cleaned (no em dash / "X, not
Y" / "bet"). PILLAR_2/3 (real revenue, founder access) kept.
- ensure_thesis_v2_promoted / revert_thesis_v2_promotion: make the v2.0
spine the working APPROVED spine and re-ground/clean the core nodes,
deployment-state-invariant (structural targeting, not body text) and
fully reversible (captures prior body/title/status/deleted_at). NODE
level only: never sets a thesis_version canonical (guardrail #4); no
hard deletes (guardrail #3). Wired into init_db after the v2 candidate
stage.
- docs/thesis-handoff.md replaced wholesale with the complete v2.0 doc;
Ten31_Agentic_Build_Plan.md + PHASE_1.md throughline glosses updated.
The v2.0 spine remains an unratified draft from the signal-engine
workstream: canonical freeze stays the partners' dual sign-off, and
Appendix-A conviction/exposure figures stay Grant's working read.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Incorporates the signal-engine workstream's v2.0 thesis correction: the spine is bitcoin
as the apex NON-DEBASABLE RESERVE ASSET (debasement = forcing function, AI = abundance
engine), NOT "infrastructure settles on bitcoin" (the settlement/payments claim — Strike's
payments thesis died in backtest). thesis_seed.ensure_thesis_v2_candidate stages the
v2.0 root/forcing-function, throughline, the verifiable-vs-contrarian decomposition, and
the 3 seams (Energy↔Compute, Debasement↔Bitcoin, AI↔Data-Ownership) as CANDIDATE nodes
under the core line (idempotent sentinel; provenance + "unratified, exposure unconfirmed"
on the section). Nothing canonical (guardrail #4). docs/thesis-handoff.md gets a
SUPERSEDED-spine banner pointing to v2.0.
NOT done (gated on partner ratification): the live THROUGHLINE/PILLAR_1 constants and
architect_agent.py's system prompt ("scarcity as the connecting idea") still encode the
old spine — until ratified+updated, Vary/Revise/outreach regenerate the old framing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 Workstream D. Lets the Architect ground the thesis in REAL recurring LP
objections without any LP identity reaching the Claude API. Layered, defense-in-depth,
fail-closed by construction (docs/redaction-rehydration.md).
backend/redaction/:
- scrub.py: the leak-proof core. Drops Tier-1 (labelled/structured account/wire/SSN/
IBAN/SWIFT/passport, separator-tolerant); tokenizes known LP entities (dictionary from
the canonical layer, unicode-folded + hyphen-extended) and structured PII (emails,
scheme-less/social URLs, intl+ext phones, currency-cued amounts, ISO/worded/numeric/
quarter dates, addresses, bare long digit runs); pre-neutralizes injected [TYPE_N]
strings; single-pass rehydrate; metadata-only audit logging (the pseudonym map is the
de-anon key — local-only, never logged/sent). Hardened across THREE adversarial
leak-hunts (worded/coded amounts, intl phones, NFD/ligature/zero-width names, slash/
comma SSN, SWIFT, alpha-prefixed accounts, substance-preserving false-positive fixes).
- client.py: Boundary — one scrub/rehydrate contract, SCRUB_BACKEND=local (default) or
gateway (Spark Control /scrub + /rehydrate). Fails closed (db_path required; dictionary
build errors propagate; strict rehydrate returns tokenized-not-de-anon text).
- test_scrub_leak.py, test_reidentification.py: golden-file leak + re-identification
suites (synthetic only, guardrail #9), regression-locking every leak-hunt vector.
backend/mcp/architect_grounding.py: the flow — retrieve (local) -> minimize-first
(local Qwen) -> scrub (+ local-Qwen NER backstop for unknown names) -> Claude over the
de-identified register only -> re-hydrate locally -> human review. FAILS CLOSED if the
local model is unreachable or a hallucinated token appears. test_grounding_boundary.py
proves nothing sensitive reaches Claude and the three fail-closed paths.
server.py: POST /api/architect/ground (admin) wires retrieval -> ground_objections.
docker_entrypoint.sh: SCRUB_BACKEND (default local). docs/spark-control-scrub-endpoints.md:
the gateway handover spec (Option 1 — caller supplies the entity dictionary).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Frontend: ThesisWorkshopPage / ThesisWorkshopNode / ThesisWorkshopOptions —
the collaborative iteration screen where partners generate a variable number
of competing thesis options (1, 2, 3, A1/A2/A3 ...) for any node, give
feedback, and regenerate. Reuses the shared api() helper; flexible option
count is the core UX constraint.
Backend Architect agent (architect_agent.py) + routes shipped in dd25bbc;
this completes the user-facing surface and bumps the StartOS package to
0.1.0:49 (anthropic dep already in the image, key loaded from
/data/secrets/anthropic-api-key — self-disabling until present).
Also lands thesis seed iterations v3 and v5 (voice/messaging corrections).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Dual sign-off is now the default (thesis_required_approvals defaults to 2).
- Entity-merge review queue (migration 0003): the fuzzy/Qwen tier no longer
auto-merges — it writes CANDIDATES (entity_merge_candidates) with a same/different
suggestion + confidence + reason for a human to approve (merge) or reject (keep
separate). entity_merge.py applies/rejects (durable via entity_merges, soft-delete,
repoint links+edges); decided pairs aren't re-surfaced.
- entity_jobs.py: UI-triggered background index jobs (rebuild/update/find-duplicates)
as subprocesses with a one-at-a-time lock; status in /api/system/status.
- server.py: /api/index/{rebuild,update}, /api/entities/find-duplicates,
/api/entities/merge-candidates [+ /{id} decide] — admin-gated.
- docs/thesis-seed-v2.md: concrete, plain-English rewrite per Grant's feedback.
Backend verified end-to-end on synthetic data (candidate gen -> approve/reject).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- entity_resolution: emit member_of relationship edges (contact -> investor),
so one investor entity owns many contacts (institution) and a HNWI is the N=1
case; crm_tools.get_investor_contacts + get_entity contacts/member_of; MCP tool.
- seed_synthetic: multi-contact institutions to exercise it (Harbor & Vine = 5).
- server.py: GET /api/system/status (index/entity/thesis/activity health) for an
in-app status view (no shell needed to verify the index).
- docs/thesis-seed-v1.md: grounded v1 thesis (throughline, 6 pillars, objections,
per-segment angles, voice) drawn from Ten31's newsletter/site/essays.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Fuzzy tier (backend/ingest/fuzzy_resolve.py + llm.py): local Qwen adjudicates
the deterministic resolver's flagged name-variant candidates; merges are
durable via entity_merges (deterministic re-runs respect them), losers
soft-deleted, logged. Idempotent.
- Incremental sync (backend/ingest/sync.py): re-embeds only rows changed since a
watermark (ingest_sync_state); first run / --recreate = full. Tested full→0→1.
- Start9 packaging (start9/0.4): Dockerfile bundles ingest+mcp + fastembed/mcp;
"Build search index" action runs the init in a subcontainer; MCP shipped as a
manual stdio server (not a daemon); version 0.1.0:44. INGEST_PACKAGING.md.
- backfill.py: factored embed_and_upsert() shared with sync.
Verified end-to-end on synthetic data + live Sparks/Qwen/Qdrant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keysat