diff --git a/AGENTS.md b/AGENTS.md index 669491f..2a98c6b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -68,7 +68,7 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude - **Two coexisting investor models** (classic `contacts`/`lp_profiles` + the `fundraising_*` grid). Reconciling them to canonical IDs is the core entity-resolution task — see `docs/crm-overview.md`. - **Soft-delete only:** `deleted_at` and/or `status='retired'`; never hard-delete. Every READ path must filter `deleted_at IS NULL` — list handlers, get-by-id, nested related-data sub-selects, **and aggregate sub-selects (`COUNT`/`SUM`/`MAX`)**. Audits found leaks in all of these (2026-06-12 detail + nested; 2026-06-13 list-view `contact_count`/`total_funded`/`comm_count`); the **reports** subsystem aggregates still leak (see Current state). Regression-guarded by `backend/test_soft_delete_reads.py`. (Thesis has a subtlety here — see the thesis guide.) -- **Env:** secrets in `.env` (gitignored); names in `.env.example`. Verified names: `ANTHROPIC_API_KEY`, `SPARK_CONTROL_URL`, `SPARK_CONTROL_VERIFY_TLS`, `QDRANT_URL`, `X_API_KEY`, `CRM_DB_PATH`, `CRM_DEV_DB_PATH`. Also used: `CRM_SECRET_KEY` (beta/prod), `CRM_HOST`/`CRM_PORT`, `CRM_DATA_DIR`. +- **Env:** secrets in `.env` (gitignored); names in `.env.example`. Verified names: `ANTHROPIC_API_KEY`, `SPARK_CONTROL_URL`, `SPARK_CONTROL_VERIFY_TLS`, `QDRANT_URL`, `X_API_KEY`, `CRM_DB_PATH`, `CRM_DEV_DB_PATH`. Also used: `CRM_SECRET_KEY` (beta/prod), `CRM_HOST`/`CRM_PORT`, `CRM_DATA_DIR`; digest mailer: `CRM_DIGEST_SENDER` (DWD impersonation sender) + `SMTP_HOST`/`SMTP_PORT`/`SMTP_SECURITY`/`SMTP_FROM`/`SMTP_USERNAME`/`SMTP_PASSWORD` (SMTP fallback). - **Commit style:** imperative subject, concise body explaining the *why*; put the package version in the subject (`… (v0.1.0:NN)`) for shippable changes. **No AI co-author / attribution trailers** — commits are authored by the user. ## Always @@ -103,11 +103,9 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude _Phase 0 substrate + Phase 1 thesis/outreach are built; **box and repo at v0.1.0:76** (Gmail-DWD digest send; deployed & verified live 2026-06-16). Repo carries minor post-76 review polish (committed, rides the next build). Longer-term backlog: `ROADMAP.md`._ - **Working (all draft-only):** CRM + ingest (chunk→embed→Qdrant + retrieval) + redaction boundary; Gmail capture (DWD) + email-activity propose→approve; Thesis Workshop + Architect (Claude) with dual-approval gate; Outreach Draft Assistant + follow-up radar + per-user voice + Tier-B in-thread Gmail draft creation. -- **Deployed & verified live (2026-06-15): v0.1.0:75** on the box (`$START9_BOX_HOST` / immense-voyage.local). `start-cli package installed-version ten-database` → `0.1.0:75`; boot log shows the v75-only entrypoint line `[entrypoint] Digest SMTP: not configured` and the server healthy on `:8080`. Shipped two batches at once: (a) the post-v74 **list-view soft-delete aggregate fix** (`server.py`: org `contact_count`/`total_funded`, contacts `comm_count`/`last_contact_date` now filter `deleted_at`) + 3 regression tests + aggregate runner; (b) **daily-digest Phase A** — `configureDigestSmtp` action writes a per-package SMTP account to `/data/secrets/smtp/*` (password over stdin; independent of any StartOS system SMTP), `docker_entrypoint.sh` exports `SMTP_*`, `backend/smtp_send.py` (stdlib smtplib), admin **`POST /api/admin/digest/test-email`** (recipients restricted to the active-admin set — not an open relay), and a **Settings → Admin "Send Test Digest Email" button**. -- **Deployed & verified live (2026-06-16): v0.1.0:76** (`installed-version` → `0.1.0:76`; boot log shows Gmail ENABLED + server healthy on `:8080`). The digest send path now prefers **Gmail domain-wide delegation** over SMTP. The box's DWD grant **includes `gmail.compose`** (send-capable; the narrow `gmail.send` is *not* granted) — verified 2026-06-15 by a token-mint probe **and a live `messages.send` to grant** (received). `backend/email_integration/gmail_send.py` impersonates a domain user and calls `users.messages.send` (reuses `credentials.py` + the compose scope, mirrors `compose.py`); `backend/digest_mailer.py` routes **Gmail-DWD → SMTP fallback**; the admin test endpoint + Settings button go through it. Sender = `CRM_DIGEST_SENDER` else the first active admin. **No app password / SMTP config needed.** Remaining confirmation: Grant clicks Settings→Admin **Send Test Digest Email** (needs admin login) to prove the app path; the raw DWD send is already proven. SMTP (v75) remains the fallback. -- **Live since v74 (2026-06-13):** login works; `/assets/` traversal 404s (plain + URL-encoded), root health 200. On boot, `ensure_thesis_v2_promoted` makes the v2.0 reserve-asset spine the working *approved* spine (node-level, reversible). -- **Shipped in v0.1.0:74** (security/privacy hardening from the 2026-06-12 full-eval; report in `EVALUATION.md`): closed a pre-auth `/assets/` path traversal (could read crm.db / JWT secret / Gmail key); wired the local-Qwen NER backstop into the outreach redaction boundary (free-prose email bodies were reaching Claude with unknown names in the clear); added `deleted_at IS NULL` to every get-by-id + nested sub-select read path. Verified locally (py_compile, query exec, redaction/outreach tests, containment logic) + two reviewer passes. -- **Tests (2026-06-15):** **19/19 backend tests green** via `python3 backend/run_tests.py` (+`test_smtp_send.py`/`test_smtp_endpoint.py`/`test_gmail_send.py` this session). `py_compile` clean; the s9pk TypeScript typechecks (`cd start9/0.4 && npm run check`, deps installed); `docker_entrypoint.sh` passes `sh -n`. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`). +- **Deployed & verified live: v0.1.0:76** (box `$START9_BOX_HOST`/immense-voyage.local; `installed-version`→`0.1.0:76`, healthy on `:8080`). **Daily-digest send is live:** `backend/digest_mailer.py` routes **Gmail-DWD (primary) → SMTP (fallback)**. DWD path (`backend/email_integration/gmail_send.py`, `gmail.compose`→`users.messages.send`, sender=`CRM_DIGEST_SENDER` else first admin) needs **no app password** and is proven by a live send to grant; the box's DWD grant has `gmail.compose` but not the narrow `gmail.send`. v75 added the SMTP fallback (**Configure Digest SMTP** action → `/data/secrets/smtp/*`, entrypoint exports `SMTP_*`), a Settings→Admin **Send Test Digest Email** button (admin-only, recipients restricted to the admin set), and the list-view soft-delete aggregate fix. Subsystem detail: `docs/guides/email.md`. +- **Live since v74 (2026-06-13):** login works; `/assets/` traversal 404s (plain + URL-encoded), root health 200. On boot, `ensure_thesis_v2_promoted` makes the v2.0 reserve-asset spine the working *approved* spine (node-level, reversible). Security/privacy hardening (path-traversal close, outreach NER backstop, get-by-id soft-delete) shipped in v74 — detail in `EVALUATION.md`. +- **Tests (2026-06-16):** **19/19 backend tests green** via `python3 backend/run_tests.py` (+`test_smtp_send.py`/`test_smtp_endpoint.py`/`test_gmail_send.py` this session). `py_compile` clean; the s9pk TypeScript typechecks (`cd start9/0.4 && npm run check`, deps installed); `docker_entrypoint.sh` passes `sh -n`. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`). - **Decided, not yet built:** CRM as canonical thesis backbone with the signal-engine reading from it (reconciliation unwired); reply-all for Tier-B drafts (drafts currently reply to the LP only). - **Known debt (P2, not deploy-blocking):** the **reports subsystem** (`handle_dashboard_report`/`handle_pipeline_report`/`handle_lp_breakdown_report`, ~16 aggregate queries over contacts/opportunities/communications/lp_profiles) still counts soft-deleted rows — the list/detail aggregates were fixed (v74 + the org/contacts list-view follow-up) but the reports were not; needs its own pass + report-endpoint tests; `?limit=abc` crashes the request thread (authenticated list path); scrub-gateway TLS verify off; `cryptography==42.0.5`; unpkg/no-SRI frontend; stale user-visible `start9/0.4/assets/ABOUT.md`; hardcoded Spark/Qdrant IPs in the s9pk; the 5.4k-line `server.py` monolith. P3 batch + full list in `EVALUATION.md`. - **Other gaps:** the v2.0 spine is the *working* spine but **not a canonical `thesis_version`** (needs Grant + Jonathan dual sign-off); Appendix-A conviction/exposure (incl. ~40% Strike) stay Grant's working read, not canonical, not fed to the engine; live features (Claude/Qdrant/Gmail) unverified on the box. diff --git a/docs/guides/email.md b/docs/guides/email.md index 64323e2..5c100cf 100644 --- a/docs/guides/email.md +++ b/docs/guides/email.md @@ -16,6 +16,31 @@ Read this before editing Gmail capture or draft creation. - **Agents draft; humans send.** Never let an agent send email, post, or contact an LP autonomously. Tier-B `compose.py` only *creates* a Gmail draft for human review. +## Outbound mail — the daily digest (internal; exempt from "agents draft") + +The CRM sends an internal **daily activity digest** to the fund's own admins. This is the +ONE automated send path, and it does **not** violate the hard rule above: that rule governs +outward **LP/prospect** contact. An internal ops email to the team's own inboxes is a +different category. **Never extend this path to send to LPs/prospects.** + +- **Transport selector: `backend/digest_mailer.py`** (top-level, not in this package) — + `send_digest(conn, to_addrs, subject, body)` picks **Gmail-DWD (preferred) → SMTP (fallback)**. + DWD-impersonation sender = `CRM_DIGEST_SENDER` env, else the first active admin. +- **Gmail-DWD path: `gmail_send.py`** (this package) — reuses `credentials.py`'s + `DWDCredentialProvider` with the **`gmail.compose`** scope to call `users.messages.send` + (REST, mirrors `compose.py`; body is `{raw}` not the draft's `{message:{raw}}`). The + deployment's DWD grant includes `gmail.compose` (which authorizes send) but **not** the + narrow `gmail.send` — so request `gmail.compose`. Verified live 2026-06-15 (token mint + + a real `messages.send`). +- **SMTP fallback: `backend/smtp_send.py`** (top-level) — stdlib smtplib reading `SMTP_*` env, + populated on the box by the **Configure Digest SMTP** Start9 action (writes + `/data/secrets/smtp/*`; entrypoint exports `SMTP_*`). A dedicated per-package account, + independent of any StartOS system-wide SMTP. +- The admin **`POST /api/admin/digest/test-email`** restricts recipients to the active-admin + set (not an open relay), and logs send failures rather than echoing them (an auth error can + carry a token/credential). Digest *content* generation (Phase B) runs on **Spark, never + Claude** — the digest is deliberately un-anonymized. + ## Known gap - Tier-B drafts currently reply to the **LP only**; reply-all is the next change (see AGENTS.md → Current state).