Add daily activity digest — Phase B (v0.1.0:77)
Sends a once-a-day internal email to all active admins summarizing each team member's email activity per investor, plus a team-wide by-investor view (inbound + outbound, deduped). Narratives are generated on the LOCAL Spark model, never Claude — the digest is intentionally un-anonymized, so substance stays on Ten31 infra. This is an internal ops email, exempt from the 'agents draft, humans send' rule (which governs outward LP contact). - backend/digest_builder.py: per-user + per-investor activity queries (soft-delete filtered), per-user Spark narrative with a deterministic fallback, two-section plain-text body, and the DB-backed policy resolver. - backend/email_integration/digest_scheduler.py: always-on daily thread that re-reads the policy each cycle and sends once/day; window cursor in app_settings so a missed day rolls forward. - server.py: POST /api/admin/digest/send-now and GET/PATCH /api/admin/digest/policy; scheduler wired into main(). - Control lives in Settings -> Admin (enable toggle + send-time dropdown), not StartOS actions; env vars only seed the first-boot default. - Tests: backend/test_digest_builder.py.
This commit is contained in:
@@ -68,7 +68,7 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
|
||||
|
||||
- **Two coexisting investor models** (classic `contacts`/`lp_profiles` + the `fundraising_*` grid). Reconciling them to canonical IDs is the core entity-resolution task — see `docs/crm-overview.md`.
|
||||
- **Soft-delete only:** `deleted_at` and/or `status='retired'`; never hard-delete. Every READ path must filter `deleted_at IS NULL` — list handlers, get-by-id, nested related-data sub-selects, **and aggregate sub-selects (`COUNT`/`SUM`/`MAX`)**. Audits found leaks in all of these (2026-06-12 detail + nested; 2026-06-13 list-view `contact_count`/`total_funded`/`comm_count`); the **reports** subsystem aggregates still leak (see Current state). Regression-guarded by `backend/test_soft_delete_reads.py`. (Thesis has a subtlety here — see the thesis guide.)
|
||||
- **Env:** secrets in `.env` (gitignored); names in `.env.example`. Verified names: `ANTHROPIC_API_KEY`, `SPARK_CONTROL_URL`, `SPARK_CONTROL_VERIFY_TLS`, `QDRANT_URL`, `X_API_KEY`, `CRM_DB_PATH`, `CRM_DEV_DB_PATH`. Also used: `CRM_SECRET_KEY` (beta/prod), `CRM_HOST`/`CRM_PORT`, `CRM_DATA_DIR`; digest mailer: `CRM_DIGEST_SENDER` (DWD impersonation sender) + `SMTP_HOST`/`SMTP_PORT`/`SMTP_SECURITY`/`SMTP_FROM`/`SMTP_USERNAME`/`SMTP_PASSWORD` (SMTP fallback).
|
||||
- **Env:** secrets in `.env` (gitignored); names in `.env.example`. Verified names: `ANTHROPIC_API_KEY`, `SPARK_CONTROL_URL`, `SPARK_CONTROL_VERIFY_TLS`, `QDRANT_URL`, `X_API_KEY`, `CRM_DB_PATH`, `CRM_DEV_DB_PATH`. Also used: `CRM_SECRET_KEY` (beta/prod), `CRM_HOST`/`CRM_PORT`, `CRM_DATA_DIR`; digest mailer: `CRM_DIGEST_SENDER` (DWD impersonation sender) + `SMTP_HOST`/`SMTP_PORT`/`SMTP_SECURITY`/`SMTP_FROM`/`SMTP_USERNAME`/`SMTP_PASSWORD` (SMTP fallback); daily digest (Phase B): `CRM_DIGEST_ENABLED` (opt-in auto-send) + `CRM_DIGEST_SEND_HOUR` (local hour, default 18).
|
||||
- **Commit style:** imperative subject, concise body explaining the *why*; put the package version in the subject (`… (v0.1.0:NN)`) for shippable changes. **No AI co-author / attribution trailers** — commits are authored by the user.
|
||||
|
||||
## Always
|
||||
@@ -100,13 +100,13 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
|
||||
|
||||
## Current state
|
||||
|
||||
_Phase 0 substrate + Phase 1 thesis/outreach are built; **box and repo at v0.1.0:76** (Gmail-DWD digest send; deployed & verified live 2026-06-16). Repo carries minor post-76 review polish (committed, rides the next build). Longer-term backlog: `ROADMAP.md`._
|
||||
_Phase 0 substrate + Phase 1 thesis/outreach are built; **box last verified live at v0.1.0:76**; **repo at v0.1.0:77** (digest **Phase B** — daily activity-digest builder/scheduler + by-team-member & by-investor sections + admin-panel control + on-demand send), built and being installed to the box. Longer-term backlog: `ROADMAP.md`._
|
||||
|
||||
- **Working (all draft-only):** CRM + ingest (chunk→embed→Qdrant + retrieval) + redaction boundary; Gmail capture (DWD) + email-activity propose→approve; Thesis Workshop + Architect (Claude) with dual-approval gate; Outreach Draft Assistant + follow-up radar + per-user voice + Tier-B in-thread Gmail draft creation.
|
||||
- **Deployed & verified live: v0.1.0:76** (box `$START9_BOX_HOST`/immense-voyage.local; `installed-version`→`0.1.0:76`, healthy on `:8080`). **Daily-digest send is live:** `backend/digest_mailer.py` routes **Gmail-DWD (primary) → SMTP (fallback)**. DWD path (`backend/email_integration/gmail_send.py`, `gmail.compose`→`users.messages.send`, sender=`CRM_DIGEST_SENDER` else first admin) needs **no app password** and is proven by a live send to grant; the box's DWD grant has `gmail.compose` but not the narrow `gmail.send`. v75 added the SMTP fallback (**Configure Digest SMTP** action → `/data/secrets/smtp/*`, entrypoint exports `SMTP_*`), a Settings→Admin **Send Test Digest Email** button (admin-only, recipients restricted to the admin set), and the list-view soft-delete aggregate fix. Subsystem detail: `docs/guides/email.md`.
|
||||
- **Deployed & verified live: v0.1.0:76** (box `$START9_BOX_HOST`/immense-voyage.local; `installed-version`→`0.1.0:76`, healthy on `:8080`). **Daily-digest send is live:** `backend/digest_mailer.py` routes **Gmail-DWD (primary) → SMTP (fallback)**. DWD path (`backend/email_integration/gmail_send.py`, `gmail.compose`→`users.messages.send`, sender=`CRM_DIGEST_SENDER` else first admin) needs **no app password** and is proven by a live send to grant; the box's DWD grant has `gmail.compose` but not the narrow `gmail.send`. v75 added the SMTP fallback (**Configure Digest SMTP** action → `/data/secrets/smtp/*`, entrypoint exports `SMTP_*`), a Settings→Admin **Send Test Digest Email** button (admin-only, recipients restricted to the admin set), and the list-view soft-delete aggregate fix. **Digest Phase B is built in-repo (not yet on the box):** `backend/digest_builder.py` builds two sections — *by team member* (per-user **Spark** narrative, never Claude) and *by investor* (team-wide, inbound + outbound, deduped) — soft-delete filtered. `backend/email_integration/digest_scheduler.py` is an always-on daily thread reading a **DB-backed policy** (`app_settings.digest_policy`) each cycle. Enable/send-time live in the **admin panel** (`GET/PATCH /api/admin/digest/policy` + a Settings toggle + time dropdown; env `CRM_DIGEST_ENABLED`/`SEND_HOUR` only seed the first-boot default). Plus `POST /api/admin/digest/send-now` + a Settings **Send Digest Now** button. Subsystem detail: `docs/guides/email.md`.
|
||||
- **Live since v74 (2026-06-13):** login works; `/assets/` traversal 404s (plain + URL-encoded), root health 200. On boot, `ensure_thesis_v2_promoted` makes the v2.0 reserve-asset spine the working *approved* spine (node-level, reversible). Security/privacy hardening (path-traversal close, outreach NER backstop, get-by-id soft-delete) shipped in v74 — detail in `EVALUATION.md`.
|
||||
- **Tests (2026-06-16):** **19/19 backend tests green** via `python3 backend/run_tests.py` (+`test_smtp_send.py`/`test_smtp_endpoint.py`/`test_gmail_send.py` this session). `py_compile` clean; the s9pk TypeScript typechecks (`cd start9/0.4 && npm run check`, deps installed); `docker_entrypoint.sh` passes `sh -n`. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`).
|
||||
- **Tests (2026-06-16):** **20/20 backend tests green** via `python3 backend/run_tests.py` (+`test_digest_builder.py` this session — per-user + per-investor queries, soft-delete, inbound dedup, two-section compose, fallback, DB policy resolver, scheduler guards). `py_compile` clean. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`).
|
||||
- **Decided, not yet built:** CRM as canonical thesis backbone with the signal-engine reading from it (reconciliation unwired); reply-all for Tier-B drafts (drafts currently reply to the LP only).
|
||||
- **Known debt (P2, not deploy-blocking):** the **reports subsystem** (`handle_dashboard_report`/`handle_pipeline_report`/`handle_lp_breakdown_report`, ~16 aggregate queries over contacts/opportunities/communications/lp_profiles) still counts soft-deleted rows — the list/detail aggregates were fixed (v74 + the org/contacts list-view follow-up) but the reports were not; needs its own pass + report-endpoint tests; `?limit=abc` crashes the request thread (authenticated list path); scrub-gateway TLS verify off; `cryptography==42.0.5`; unpkg/no-SRI frontend; stale user-visible `start9/0.4/assets/ABOUT.md`; hardcoded Spark/Qdrant IPs in the s9pk; the 5.4k-line `server.py` monolith. P3 batch + full list in `EVALUATION.md`.
|
||||
- **Other gaps:** the v2.0 spine is the *working* spine but **not a canonical `thesis_version`** (needs Grant + Jonathan dual sign-off); Appendix-A conviction/exposure (incl. ~40% Strike) stay Grant's working read, not canonical, not fed to the engine; live features (Claude/Qdrant/Gmail) unverified on the box.
|
||||
- **Next:** 1) Grant clicks Settings→Admin **Send Test Digest Email** (the app-path confirmation; raw DWD send already proven); 2) **digest Phase B** — daily scheduler + per-user→per-investor activity query (`deleted_at IS NULL`) + **Spark-narrative** summary (never Claude) → email all admins (decisions locked in `ROADMAP.md`); 3) **reports-subsystem soft-delete sweep** (~16 aggregates still leak; fix + tests); 4) `?limit=abc` crash (P2); 5) Grant + Jonathan freeze v2.0 canonical; 6) build reply-all; 7) confirm Appendix-A + Maple/OpenSecret/Primal, then promote.
|
||||
- **Next:** 1) **bump version + build/deploy** so Phase B reaches the box, then Grant: Settings→Admin **Send Digest Now** to validate the real digest, and tick **Send automatically every day** (arming is now in-app — no env/StartOS change needed); 2) **reports-subsystem soft-delete sweep** (~16 aggregates still leak; fix + tests); 3) `?limit=abc` crash (P2); 4) Grant + Jonathan freeze v2.0 canonical; 5) build reply-all; 6) confirm Appendix-A + Maple/OpenSecret/Primal, then promote.
|
||||
|
||||
Reference in New Issue
Block a user