Email capture & drafts (Gmail)

Read this before editing Gmail capture or draft creation.

What it does

backend/email_integration/ captures Gmail via domain-wide delegation (credentials.py, matcher.py, parser.py, db.py, sync.py, scheduler.py, routes.py) and creates Tier-B in-thread drafts (compose.py). It has its own migrations/.
Captured email becomes CRM activity through a propose → approve flow — nothing lands on a contact record until a human approves the proposal.

Hard rule

Agents draft; humans send. Never let an agent send email, post, or contact an LP autonomously. Tier-B compose.py only creates a Gmail draft for human review.

Outbound mail — the daily digest (internal; exempt from "agents draft")

The CRM sends an internal daily activity digest to the fund's own admins. This is the ONE automated send path, and it does not violate the hard rule above: that rule governs outward LP/prospect contact. An internal ops email to the team's own inboxes is a different category. Never extend this path to send to LPs/prospects.

Transport selector: backend/digest_mailer.py (top-level, not in this package) — send_digest(conn, to_addrs, subject, body) picks Gmail-DWD (preferred) → SMTP (fallback). DWD-impersonation sender = CRM_DIGEST_SENDER env, else the first active admin.
Gmail-DWD path: gmail_send.py (this package) — reuses credentials.py's DWDCredentialProvider with the gmail.compose scope to call users.messages.send (REST, mirrors compose.py; body is {raw} not the draft's {message:{raw}}). The deployment's DWD grant includes gmail.compose (which authorizes send) but not the narrow gmail.send — so request gmail.compose. Verified live 2026-06-15 (token mint + a real messages.send).
SMTP fallback: backend/smtp_send.py (top-level) — stdlib smtplib reading SMTP_* env, populated on the box by the Configure Digest SMTP Start9 action (writes /data/secrets/smtp/*; entrypoint exports SMTP_*). A dedicated per-package account, independent of any StartOS system-wide SMTP.
The admin POST /api/admin/digest/test-email restricts recipients to the active-admin set (not an open relay), and logs send failures rather than echoing them (an auth error can carry a token/credential).

Phase B — the daily digest itself (built)

Content builder: backend/digest_builder.py (top-level). build_digest(conn, since_iso, until_iso, chat_fn=None) returns {subject, body, has_activity, user_count, email_count, investor_count} and composes two sections:
- By team member — collect_user_activity: per registered user, both directions (per-mailbox eam.is_sent), with one Spark narrative paragraph per user (ingest/llm.py → Spark Control /v1/chat/completions), never Claude (the digest is deliberately un-anonymized — real LP names + substance stay local). Deterministic count-only fallback if Spark is unreachable (always-send must not fail).
- By investor — collect_investor_activity: re-pivots the same window across the whole team, deduped per email (a reply to several teammates counts once), direction decided at the email level (outbound if from_email is one of our mailboxes, else inbound). Structured list, no extra Spark calls.
- Soft-delete filters: email_account_messages.deleted_at IS NULL + users.is_active = 1, and the org/contact name joins drop soft-deleted rows (falling back to the matched address).
Control is DB-backed, set from the admin panel — digest_builder.load_digest_policy(conn) reads app_settings.digest_policy = {enabled, send_hour}. Precedence: DB row wins (the Settings → Admin toggle + send-time dropdown), else CRM_DIGEST_ENABLED/ CRM_DIGEST_SEND_HOUR seed a first-boot default, else {false, 18}. GET/PATCH /api/admin/digest/policy (admin-only) read/write it. Not a StartOS action — it's an operational toggle, so it lives in-app where it's discoverable and takes effect live.
Scheduler: backend/email_integration/digest_scheduler.py (co-located with the sync scheduler). One daemon thread, always started; each cycle (60s) re-reads the DB policy and sends once per local day at/after send_hour only when enabled — so toggling in the panel takes effect with no restart. Content window = (last send, now]; cursor (digest_last_sent_at) + once-per-day guard (digest_last_sent_date) live in app_settings, so a missed day rolls into the next digest. Recipients = all active admins.
On-demand: POST /api/admin/digest/send-now (admin-only) → maybe_send_digest(force=True) builds the real last-24h digest and sends to the admin set regardless of the policy and without touching the daily cursor (a preview never suppresses the scheduled send). Surfaced as a "Send Digest Now" button in Settings → Admin, beside "Send Test Digest Email".
Decisions (locked): 6 PM default send · always-send (empty days get a "no activity" note) · per-user narrative + by-investor structured section · enable/time controlled in the admin panel. Tests: backend/test_digest_builder.py (per-user + per-investor queries, soft-delete, inbound dedup, two-section compose, fallback, policy resolver, scheduler guards — stubbed LLM + transport).

Email-activity panel (Communications tab) — admin-only

The Communications tab (frontend) is the admin-only search over captured Gmail. The classic manual "Log Communication" form was retired (the Fundraising Grid context menu is the manual-log path). Backed by GET /api/email/activity (routes.py:_h_activity, require_admin server-side) → db.query_email_activity(conn, ...) (the pure, tested query). Filters: investor_id, account_id (mailbox), direction (inbound/outbound), q (free-text over subject/snippet/from). Non-obvious semantics to preserve:

Soft-delete lives on the per-mailbox sighting, not the email: emails has no deleted_at. An email is "live" iff it has a sighting with email_account_messages. deleted_at IS NULL — the query gates on EXISTS(... deleted_at IS NULL). (Investor links are email-level and carry no deleted_at, so they need no separate filter.)
Direction is decided at the email level — outbound if from_email is one of our email_accounts addresses, else inbound — mirroring digest_builder._own_addresses.
Graveyard investors are hidden from the filter dropdown (CRM-wide graveyard = 0), but their captured email still shows in the list and stays findable by free-text search — it's an audit surface, so history is never hidden, only the picker is.

Tests: backend/email_integration/test_email_activity_panel.py.

Known gap

Tier-B drafts currently reply to the LP only; reply-all is the next change (see AGENTS.md → Current state).

See also docs/gmail-enablement-runbook.md.

7.0 KiB Raw Blame History