ten31-database/docs/ten31-constitution.md

Ten31 Agentic System — Project Memory

This file is the project constitution. Read it first; it states settled decisions and non-negotiable guardrails. Where anything here conflicts with a one-off prompt, this file wins.

What we're building

Ten31 is an investment platform (bitcoin ecosystem, energy, AI infrastructure, freedom tech) that raises from LPs and deploys into private companies. We are building an in-house system of AI agents to widen the fundraising funnel, sharpen and propagate our investment thesis, and automate marketing/branding. Build agents on the Claude Agent SDK, connected to our systems via MCP. Frontier reasoning runs on Claude; privacy-sensitive and high-volume work runs on local models on our DGX Sparks, fronted by Spark Control.

Full architecture and rationale: see @./docs/Ten31_Agentic_Build_Plan.md. Current phase tasks and acceptance criteria: see @./docs/PHASE_0.md. Embedding/retrieval API contract + ingest recipe (authoritative): see @./docs/EMBEDDINGS.md.

We are in Phase 0. Phase 0 builds the data + retrieval substrate. There are NO live, outward-facing agents in Phase 0.

Settled architecture

• Reasoning model: Claude via the Agent SDK / API (API-key auth, not claude.ai login).
• Local models (Sparks, via Spark Control gateway):
  • Chat/triage: Qwen3.6 35B-A3B on Spark 1.
  • Embeddings: BAAI/bge-m3 (dense, 1024-dim, L2-normalized) → /v1/embeddings (OpenAI shape).
  • Reranker: BAAI/bge-reranker-v2-m3 (cross-encoder) → /v1/rerank (Cohere shape).
  • Served by spark-embed, a small FastAPI server on Spark 2 (NGC PyTorch image — not HF TEI, which ships no arm64 CUDA image). Shipped in Spark Control v0.15.0.
  • Audio: transcription + diarization + TTS on Spark 2.
• Canonical data store: the self-built CRM on the Start9 server. This is the single source of truth for LP/prospect data.
• Vector index: Qdrant v1.16.0 on Spark 2 (ports 6333/6334). Derived and rebuildable from the CRM (~8–15 min full re-embed) — NOT a second source of truth. But it holds the only live copy of the index, so it is never auto-restarted; the ingest pipeline must be idempotent so a rebuild is always safe.
• Retrieval: one orchestrated call, POST /api/search (embed query → Qdrant dense+sparse RRF with payload pre-filter → cross-encoder rerank → top_k). The sparse/BM25 leg is generated client-side with FastEmbed (Qdrant/bm25) at both ingest and query time, with Qdrant applying IDF over our own corpus — so exact entity/name matching is weighted by our term statistics, not bge-m3's pretrained sparse. Authoritative contract + ingest recipe: @./docs/EMBEDDINGS.md.
• Gateway: Spark Control (on Start9) fronts all local model services behind one trusted URL with shared TLS, access control, and observability.

Environment & services

• All local model calls go through Spark Control, never directly to a Spark.
• Endpoints: /v1/chat/completions, /v1/embeddings, /v1/rerank, /api/search (orchestrated hybrid retrieval), /v1/audio/transcriptions, /v1/audio/speech.
• Secrets live in .env (gitignored). Never commit secret values. Required variables (names only):
  • ANTHROPIC_API_KEY
  • SPARK_CONTROL_URL — gateway for /v1/embeddings, /v1/rerank, /api/search (reads + dense embeds)
  • QDRANT_URL — direct Qdrant on Spark 2 (http://<spark2>:6333) for collection admin + ingest upserts
  • X_API_KEY — the X (Twitter) API key for Scout/Analyst enrichment. Note: this is not a CRM auth key; the CRM has no service-key/API-key path today (see below).
  • CRM connection vars:
    • CRM_DB_PATH — absolute path to the SQLite file (default <CRM_DATA_DIR>/crm.db). The CRM has no network DB protocol — ingest "connects" by opening this file directly (read-only, mode=ro), co-located with the Start9 /data volume.
    • CRM_DATA_DIR — the /data volume root (holds crm.db, backups/, secrets/, email_attachments/).
    • CRM_BASE_URL — http://<host>:8080 (env CRM_HOST/CRM_PORT), for any HTTP access to the running CRM.
    • CRM_SECRET_KEY — the CRM's own JWT signing secret (set on the Start9 deployment, persisted at /data/.crm-secret); only needed if the MCP server authenticates over HTTP rather than reading SQLite directly.
• A .env.example lists the variable names with empty values.

The agents (target roster — built in later phases)

• Scout — monitors public sources (X via API, filings, etc.); flags trigger events. (Phase 2)
• Analyst — builds LP dossiers, enriches records, maps warm-intro paths. (Phase 2)
• Architect — owns/refines the canonical thesis; collaborative copilot. (Phase 1)
• Scribe — distributes the thesis as content across channels. (Phase 1)
• Closer — drafts outreach, nurture, meeting prep. Humans approve/send everything. (Phase 3)
• Orchestrator — schedules and routes work; picks per-agent retrieval modes. (Phase 3)

Division of labor

• Spark developer (separate): TEI serving (BGE-M3 + reranker) and Qdrant on Spark 2, exposed via Spark Control /v1/embeddings + /v1/rerank.
• This repo (Claude Code + the partners): CRM schema extensions, ingest/sync pipeline, CRM MCP server, retrieval-mode library, and (later phases) the agents.

Guardrails — NON-NEGOTIABLE

1. Sovereignty. Sensitive LP and relationship data stays on our infrastructure (Start9 + Sparks). Send only the minimum necessary, non-sensitive context to the Claude API. Never bulk-export the LP list to any third party.
2. CRM is canonical. Qdrant and any other store are derived. Never treat a derived index as the source of truth; never let them silently diverge.
3. No destructive data ops. Never hard-delete CRM records or history. Soft-delete/archive only. Migrations must be reversible and reviewed before running.
4. Human-in-the-loop on anything outbound. No agent sends email, posts publicly, or contacts an LP/prospect autonomously. Agents draft; a partner approves and sends. (Especially Closer and Scribe.)
5. Log every agent action to the interaction log, for compliance and debugging.
6. Compliance gate before Phase 3. No cold/outbound capability ships until counsel has defined solicitation posture (e.g. 506(b) vs 506(c)), accreditation/QP verification, and recordkeeping rules.
7. Secrets never committed. Use .env / a secrets store. No keys, tokens, or credentials in code, configs, or docs.
8. Enrichment is one-way and public. Per-prospect public lookups that write INTO the CRM; never push our data outward.
9. Development data handling — keep real LP data out of Claude during the build. Claude Code (the engineering partner) runs on the Anthropic API, so anything it reads is sent to a third party. Therefore Claude Code works only on code, the schema, and synthetic or properly-redacted data — never the real LP list, live records, or raw note/email prose. The real backfill and ingest run on Ten31 infrastructure (Start9 + Sparks) via local models; sensitive rows are never pasted into a Claude Code session or sent to the Claude API during development. To produce a realistic test corpus, redact/pseudonymize a copy on the Sparks (local) — do not hand-feed real records to Claude to "clean up." This is the same sovereignty boundary as guardrail #1, applied to the engineering workflow itself.

Conventions

Filled in from the CRM code (2026-06). Full detail: @./docs/crm-overview.md.

• Language / runtime: Python 3.11, standard library only at runtime. The CRM is one file, backend/server.py (~4.5k lines): a stdlib http.server.ThreadingHTTPServer + hand-written CRMHandler with manual path dispatch. Not FastAPI — backend/requirements.txt lists FastAPI/SQLAlchemy/Alembic/Pydantic but none are imported (vestigial). The only non-stdlib runtime deps are optional bcrypt/jwt and (for the Gmail module) cryptography.
• Storage: a single SQLite DB (data/crm.db), WAL mode, foreign_keys=ON, opened per-request via get_db(). Two parallel investor models coexist (classic contacts/lp_profiles + the fundraising_* grid) — see docs/crm-overview.md §2.3; reconciling them to canonical IDs is the core Phase-0 entity-resolution task.
• Migrations: additive and reversible only. Core schema uses ordered backend/migrations/NNNN_*.sql files applied once at startup by backend/core_migrations.py, tracked in a schema_migrations ledger; ship a paired NNNN_*.down.sql for rollback. (The Gmail module has its own runner under backend/email_integration/migrations/.) SQLite ALTER is add-column/rename only — which enforces the additive guardrail.
• Run locally: ./start.sh (dev defaults, port 8080). ./start_beta.sh for a Tailscale/production-mode launch (requires CRM_SECRET_KEY). No build step.
• Tests / lint: none in-repo. Sanity-check edits with python3 -m py_compile backend/server.py. Verify migrations against a copy of crm.db, never production.
• Production: Start9 package ten-database. start9/0.4/ is the live target (TypeScript SDK manifest under start9/0.4/startos/); start9/0.3.5/ (YAML manifest) is the superseded prior generation. All state on the persistent /data volume.
• Auth: username/password → HS256 JWT (Bearer header), two roles (admin/member), no row-level authorization. X_API_KEY (in this file's env list) is the X/Twitter key — there is no CRM service-key path in code; an MCP/ingest client must read SQLite directly or authenticate as a real CRM user.
• Prefer clear, reviewable changes over cleverness. Keep the ingest pipeline and MCP server modular so retrieval modes and sources can be added without rewrites.

First actions for a new session

1. Read @./docs/PHASE_0.md and @./docs/EMBEDDINGS.md (the latter is the authoritative embedding/retrieval contract and ingest recipe).
2. Read the CRM source in the repo; produce a short written summary of the storage engine, schema, and API surface, and fill in the Conventions section above and the CRM env vars.
3. Confirm Spark Control is reachable and /v1/embeddings, /v1/rerank, and /api/search respond (these shipped in v0.15.0; check GET /api/endpoints).
4. Proceed through the Phase 0 workstreams in order. Do not build any outward-facing agent behavior in Phase 0.