Add Gmail-DWD send path for the digest mailer (v0.1.0:76)

The box's existing service-account domain-wide-delegation grant already includes
gmail.compose, which authorizes users.messages.send — verified 2026-06-15 by a
token-mint probe and a live messages.send to grant. So CRM-originated mail can
send through the account that already powers email capture: no SMTP account, no
app password, no admin change.

- backend/email_integration/gmail_send.py: send_via_gmail() impersonates a
  domain user and POSTs users.messages.send (reuses credentials.py + the compose
  scope; mirrors compose.py's REST pattern).
- backend/digest_mailer.py: send_digest() prefers Gmail DWD when enabled, falls
  back to smtp_send otherwise. Sender = CRM_DIGEST_SENDER else first active admin.
- server.py: the admin test endpoint now routes through digest_mailer (so the
  Settings button sends via DWD on the box with zero SMTP config). Recipient
  restriction to the admin set and no-leak error handling preserved.
- test_gmail_send.py: build/send + transport routing (provider + urlopen faked).
  19/19 backend green; s9pk typechecks.

SMTP (v75) stays as the fallback transport. Send-path decision + scope finding
recorded in ROADMAP.md and AGENTS.md.
This commit is contained in:
Keysat
2026-06-15 20:17:27 -05:00
parent e62306be27
commit 47dfd110a0
10 changed files with 383 additions and 56 deletions
+8 -1
View File
@@ -20,7 +20,14 @@ X_API_KEY=
CRM_DB_PATH=./data/crm.db CRM_DB_PATH=./data/crm.db
CRM_DEV_DB_PATH=./data/crm_dev.db CRM_DEV_DB_PATH=./data/crm_dev.db
# ── Daily-digest outbound SMTP (dev override of the per-package mailbox) ── # ── Daily-digest sender ──
# The digest mailer prefers Gmail domain-wide delegation (the service account that
# already powers email capture; its grant includes gmail.compose, which can send) and
# falls back to SMTP below. For the Gmail/DWD path it sends impersonating this domain
# user; if unset, it uses the first active admin's email.
CRM_DIGEST_SENDER=
# ── Daily-digest outbound SMTP fallback (dev override of the per-package mailbox) ──
# On the Start9 box these are set by the "Configure Digest SMTP" action (written # On the Start9 box these are set by the "Configure Digest SMTP" action (written
# to /data/secrets/smtp/* and exported by docker_entrypoint.sh). For dev, set them # to /data/secrets/smtp/* and exported by docker_entrypoint.sh). For dev, set them
# here. SMTP_SECURITY is one of: starttls (587) | tls (465) | none. # here. SMTP_SECURITY is one of: starttls (587) | tls (465) | none.
+5 -4
View File
@@ -100,14 +100,15 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
## Current state ## Current state
_Phase 0 substrate + Phase 1 thesis/outreach are built; **box and repo both at v0.1.0:75** (deployed & verified live 2026-06-15). Longer-term backlog: `ROADMAP.md`._ _Phase 0 substrate + Phase 1 thesis/outreach are built; **box at v0.1.0:75, repo at v0.1.0:76** (the Gmail-DWD digest-send path; committed, redeploy pending). Longer-term backlog: `ROADMAP.md`._
- **Working (all draft-only):** CRM + ingest (chunk→embed→Qdrant + retrieval) + redaction boundary; Gmail capture (DWD) + email-activity propose→approve; Thesis Workshop + Architect (Claude) with dual-approval gate; Outreach Draft Assistant + follow-up radar + per-user voice + Tier-B in-thread Gmail draft creation. - **Working (all draft-only):** CRM + ingest (chunk→embed→Qdrant + retrieval) + redaction boundary; Gmail capture (DWD) + email-activity propose→approve; Thesis Workshop + Architect (Claude) with dual-approval gate; Outreach Draft Assistant + follow-up radar + per-user voice + Tier-B in-thread Gmail draft creation.
- **Deployed & verified live (2026-06-15): v0.1.0:75** on the box (`$START9_BOX_HOST` / immense-voyage.local). `start-cli package installed-version ten-database``0.1.0:75`; boot log shows the v75-only entrypoint line `[entrypoint] Digest SMTP: not configured` and the server healthy on `:8080`. Shipped two batches at once: (a) the post-v74 **list-view soft-delete aggregate fix** (`server.py`: org `contact_count`/`total_funded`, contacts `comm_count`/`last_contact_date` now filter `deleted_at`) + 3 regression tests + aggregate runner; (b) **daily-digest Phase A**`configureDigestSmtp` action writes a per-package SMTP account to `/data/secrets/smtp/*` (password over stdin; independent of any StartOS system SMTP), `docker_entrypoint.sh` exports `SMTP_*`, `backend/smtp_send.py` (stdlib smtplib), admin **`POST /api/admin/digest/test-email`** (recipients restricted to the active-admin set — not an open relay), and a **Settings → Admin "Send Test Digest Email" button**. **SMTP not yet configured on the box** — Grant runs the action + restart + test to light it up. - **Deployed & verified live (2026-06-15): v0.1.0:75** on the box (`$START9_BOX_HOST` / immense-voyage.local). `start-cli package installed-version ten-database``0.1.0:75`; boot log shows the v75-only entrypoint line `[entrypoint] Digest SMTP: not configured` and the server healthy on `:8080`. Shipped two batches at once: (a) the post-v74 **list-view soft-delete aggregate fix** (`server.py`: org `contact_count`/`total_funded`, contacts `comm_count`/`last_contact_date` now filter `deleted_at`) + 3 regression tests + aggregate runner; (b) **daily-digest Phase A**`configureDigestSmtp` action writes a per-package SMTP account to `/data/secrets/smtp/*` (password over stdin; independent of any StartOS system SMTP), `docker_entrypoint.sh` exports `SMTP_*`, `backend/smtp_send.py` (stdlib smtplib), admin **`POST /api/admin/digest/test-email`** (recipients restricted to the active-admin set — not an open relay), and a **Settings → Admin "Send Test Digest Email" button**.
- **Repo ahead (v0.1.0:76 — committed, redeploy pending):** the digest send path now prefers **Gmail domain-wide delegation** over SMTP. The box's DWD grant **includes `gmail.compose`** (send-capable; the narrow `gmail.send` is *not* granted) — verified 2026-06-15 by a token-mint probe **and a live `messages.send` to grant** (both succeeded). `backend/email_integration/gmail_send.py` impersonates a domain user and calls `users.messages.send` (reuses `credentials.py` + the compose scope, mirrors `compose.py`); `backend/digest_mailer.py` routes **Gmail-DWD → SMTP fallback**; the admin test endpoint + Settings button go through it. Sender = `CRM_DIGEST_SENDER` else the first active admin. **Net: no app password needed** — once v76 is redeployed, the test button sends via DWD with zero SMTP config. SMTP (v75) remains the fallback.
- **Live since v74 (2026-06-13):** login works; `/assets/` traversal 404s (plain + URL-encoded), root health 200. On boot, `ensure_thesis_v2_promoted` makes the v2.0 reserve-asset spine the working *approved* spine (node-level, reversible). - **Live since v74 (2026-06-13):** login works; `/assets/` traversal 404s (plain + URL-encoded), root health 200. On boot, `ensure_thesis_v2_promoted` makes the v2.0 reserve-asset spine the working *approved* spine (node-level, reversible).
- **Shipped in v0.1.0:74** (security/privacy hardening from the 2026-06-12 full-eval; report in `EVALUATION.md`): closed a pre-auth `/assets/` path traversal (could read crm.db / JWT secret / Gmail key); wired the local-Qwen NER backstop into the outreach redaction boundary (free-prose email bodies were reaching Claude with unknown names in the clear); added `deleted_at IS NULL` to every get-by-id + nested sub-select read path. Verified locally (py_compile, query exec, redaction/outreach tests, containment logic) + two reviewer passes. - **Shipped in v0.1.0:74** (security/privacy hardening from the 2026-06-12 full-eval; report in `EVALUATION.md`): closed a pre-auth `/assets/` path traversal (could read crm.db / JWT secret / Gmail key); wired the local-Qwen NER backstop into the outreach redaction boundary (free-prose email bodies were reaching Claude with unknown names in the clear); added `deleted_at IS NULL` to every get-by-id + nested sub-select read path. Verified locally (py_compile, query exec, redaction/outreach tests, containment logic) + two reviewer passes.
- **Tests (2026-06-15):** **18/18 backend tests green** via `python3 backend/run_tests.py` (+`test_smtp_send.py`/`test_smtp_endpoint.py` this session). `py_compile` clean; the s9pk TypeScript typechecks (`cd start9/0.4 && npm run check`, deps installed); `docker_entrypoint.sh` passes `sh -n`. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`). - **Tests (2026-06-15):** **19/19 backend tests green** via `python3 backend/run_tests.py` (+`test_smtp_send.py`/`test_smtp_endpoint.py`/`test_gmail_send.py` this session). `py_compile` clean; the s9pk TypeScript typechecks (`cd start9/0.4 && npm run check`, deps installed); `docker_entrypoint.sh` passes `sh -n`. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`).
- **Decided, not yet built:** CRM as canonical thesis backbone with the signal-engine reading from it (reconciliation unwired); reply-all for Tier-B drafts (drafts currently reply to the LP only). - **Decided, not yet built:** CRM as canonical thesis backbone with the signal-engine reading from it (reconciliation unwired); reply-all for Tier-B drafts (drafts currently reply to the LP only).
- **Known debt (P2, not deploy-blocking):** the **reports subsystem** (`handle_dashboard_report`/`handle_pipeline_report`/`handle_lp_breakdown_report`, ~16 aggregate queries over contacts/opportunities/communications/lp_profiles) still counts soft-deleted rows — the list/detail aggregates were fixed (v74 + the org/contacts list-view follow-up) but the reports were not; needs its own pass + report-endpoint tests; `?limit=abc` crashes the request thread (authenticated list path); scrub-gateway TLS verify off; `cryptography==42.0.5`; unpkg/no-SRI frontend; stale user-visible `start9/0.4/assets/ABOUT.md`; hardcoded Spark/Qdrant IPs in the s9pk; the 5.4k-line `server.py` monolith. P3 batch + full list in `EVALUATION.md`. - **Known debt (P2, not deploy-blocking):** the **reports subsystem** (`handle_dashboard_report`/`handle_pipeline_report`/`handle_lp_breakdown_report`, ~16 aggregate queries over contacts/opportunities/communications/lp_profiles) still counts soft-deleted rows — the list/detail aggregates were fixed (v74 + the org/contacts list-view follow-up) but the reports were not; needs its own pass + report-endpoint tests; `?limit=abc` crashes the request thread (authenticated list path); scrub-gateway TLS verify off; `cryptography==42.0.5`; unpkg/no-SRI frontend; stale user-visible `start9/0.4/assets/ABOUT.md`; hardcoded Spark/Qdrant IPs in the s9pk; the 5.4k-line `server.py` monolith. P3 batch + full list in `EVALUATION.md`.
- **Other gaps:** the v2.0 spine is the *working* spine but **not a canonical `thesis_version`** (needs Grant + Jonathan dual sign-off); Appendix-A conviction/exposure (incl. ~40% Strike) stay Grant's working read, not canonical, not fed to the engine; live features (Claude/Qdrant/Gmail) unverified on the box. - **Other gaps:** the v2.0 spine is the *working* spine but **not a canonical `thesis_version`** (needs Grant + Jonathan dual sign-off); Appendix-A conviction/exposure (incl. ~40% Strike) stay Grant's working read, not canonical, not fed to the engine; live features (Claude/Qdrant/Gmail) unverified on the box.
- **Next:** 1) **on the box: configure the digest mailbox**run **Configure Digest SMTP** (dedicated SMTP creds), restart, then Settings→Admin **Send Test Digest Email** to confirm a real message lands; 2) **digest Phase B** — daily scheduler + per-user→per-investor activity query (`deleted_at IS NULL`) + **Spark-narrative** summary (never Claude) → email all admins (decisions locked in `ROADMAP.md`); 3) **reports-subsystem soft-delete sweep** (~16 aggregates still leak; fix + tests); 4) `?limit=abc` crash (P2); 5) Grant + Jonathan freeze v2.0 canonical; 6) build reply-all; 7) confirm Appendix-A + Maple/OpenSecret/Primal, then promote. - **Next:** 1) **redeploy v0.1.0:76**, then Settings→Admin **Send Test Digest Email**it should send via **Gmail DWD** to all admins with no SMTP setup (the live DWD send is already proven; this confirms it through the app path); 2) **digest Phase B** — daily scheduler + per-user→per-investor activity query (`deleted_at IS NULL`) + **Spark-narrative** summary (never Claude) → email all admins (decisions locked in `ROADMAP.md`); 3) **reports-subsystem soft-delete sweep** (~16 aggregates still leak; fix + tests); 4) `?limit=abc` crash (P2); 5) Grant + Jonathan freeze v2.0 canonical; 6) build reply-all; 7) confirm Appendix-A + Maple/OpenSecret/Primal, then promote.
+4 -2
View File
@@ -87,11 +87,13 @@
## Backlog (post-Phase-1 agentic) ## Backlog (post-Phase-1 agentic)
### Daily activity digest (email to the team) ### Daily activity digest (email to the team)
*Requested 2026-06-15. **Phase A built in v0.1.0:75** (outbound SMTP send capability + admin test-email endpoint; not yet deployed). Phase B (digest content + Spark summarization + daily scheduler) remains.* *Requested 2026-06-15. **Phase A built + deployed** (v0.1.0:75 live on the box; send path then moved to **Gmail DWD** in v0.1.0:76, redeploy pending). Phase B (digest content + Spark summarization + daily scheduler) remains.*
**Decisions (locked 2026-06-15):** recipients = **all active admins**; summarization = **Spark-LLM narrative** (never Claude — un-anonymized substance stays local); granularity = **grouped by user** (→ per investor). **Decisions (locked 2026-06-15):** recipients = **all active admins**; summarization = **Spark-LLM narrative** (never Claude — un-anonymized substance stays local); granularity = **grouped by user** (→ per investor).
**Phase A — DONE (v0.1.0:75):** `configureDigestSmtp` Start9 action writes a per-package SMTP account to `/data/secrets/smtp/*`; `docker_entrypoint.sh` exports `SMTP_*`; `backend/smtp_send.py` (stdlib smtplib) + admin `POST /api/admin/digest/test-email` (recipient-restricted to the admin set — not an open relay). Tests: `test_smtp_send.py`, `test_smtp_endpoint.py`. **Send transport — DECIDED 2026-06-15: Gmail domain-wide delegation** (not SMTP). The box's existing service-account grant (which powers email capture) **includes `gmail.compose`**, which authorizes `users.messages.send` — verified by a token-mint probe **and a live `messages.send` to grant**. So the digest sends through the account the CRM already uses: **no app password, no new account, no admin change.** The narrow `gmail.send` scope is *not* granted, so the sender must request `gmail.compose`.
**Phase A — DONE:** (v0.1.0:75) `configureDigestSmtp` Start9 action + `docker_entrypoint.sh` `SMTP_*` export + `backend/smtp_send.py` + admin `POST /api/admin/digest/test-email` (recipient-restricted to the admin set — not an open relay) + Settings button. (v0.1.0:76, redeploy pending) `backend/email_integration/gmail_send.py` (`users.messages.send` via DWD/compose) + `backend/digest_mailer.py` (**Gmail-DWD preferred, SMTP fallback**); the endpoint + button route through it; sender = `CRM_DIGEST_SENDER` else first active admin. Tests: `test_smtp_send.py`, `test_smtp_endpoint.py`, `test_gmail_send.py`.
**Phase B — TODO:** daily scheduler (co-locate with `email_integration/scheduler.py`); per-user→per-investor activity query (`deleted_at IS NULL` throughout); Spark-narrative summary of captured email substance; compose + send to all admins. **Phase B — TODO:** daily scheduler (co-locate with `email_integration/scheduler.py`); per-user→per-investor activity query (`deleted_at IS NULL` throughout); Spark-narrative summary of captured email substance; compose + send to all admins.
+64
View File
@@ -0,0 +1,64 @@
"""Transport selection for CRM-originated email (daily digest, admin test sends).
Prefers Gmail-over-DWD — it reuses the service account that already powers email
capture (the grant includes gmail.compose, which can send), so there's no extra
credential to manage — and falls back to SMTP (`smtp_send`) when DWD isn't
available. One entry point so the digest and the admin test endpoint share the
same routing. Stdlib only.
"""
import os
class NoTransport(Exception):
"""Neither Gmail DWD nor SMTP is configured."""
def transport():
"""Return the active transport: 'gmail-dwd', 'smtp', or None."""
try:
from email_integration import gmail_send
if gmail_send.gmail_available():
return "gmail-dwd"
except Exception:
pass
try:
import smtp_send
if smtp_send.smtp_configured():
return "smtp"
except Exception:
pass
return None
def default_sender(conn):
"""Domain user to send as for the DWD path. `CRM_DIGEST_SENDER` if set, else
the first active admin's email."""
s = os.environ.get("CRM_DIGEST_SENDER", "").strip()
if s:
return s
if conn is None:
return None
row = conn.execute(
"SELECT email FROM users WHERE role='admin' AND is_active=1 "
"AND email IS NOT NULL AND TRIM(email)!='' ORDER BY created_at LIMIT 1"
).fetchone()
return row["email"].strip() if row and row["email"] else None
def send_digest(conn, to_addrs, subject, body, sender_email=None):
"""Send via the active transport. Returns the transport's result dict with a
'transport' key added; raises NoTransport if neither is configured."""
t = transport()
if t == "gmail-dwd":
from email_integration import gmail_send
sender = sender_email or default_sender(conn)
result = gmail_send.send_via_gmail(sender, to_addrs, subject, body, conn=conn)
result["transport"] = "gmail-dwd"
return result
if t == "smtp":
import smtp_send
result = smtp_send.send_email(to_addrs, subject, body)
result["transport"] = "smtp"
return result
raise NoTransport("No email transport configured: enable Gmail (DWD) or set "
"SMTP via the 'Configure Digest SMTP' action.")
+65
View File
@@ -0,0 +1,65 @@
"""Send an email via the Gmail API using the same domain-wide delegation that
powers capture and draft creation.
The DWD grant on this deployment includes the `gmail.compose` scope (verified
2026-06-15: token mint + a live messages.send both succeed), and `gmail.compose`
authorizes `users.messages.send`. So CRM-originated mail (the daily digest) can
send through the existing service account — no SMTP account, no app password, no
admin change. Sends impersonating `sender_email`, which must be a Workspace user
in the delegated domain. Mirrors the REST pattern in compose.py; stdlib only.
"""
import base64
import email.message
import json
import os
import urllib.error
import urllib.parse
import urllib.request
from . import config as _cfg
from . import credentials as _creds
def gmail_available():
"""True when DWD send is usable: integration enabled, DWD auth, key present."""
cfg = _cfg.CONFIG
if not cfg.enabled or cfg.primary_auth != "dwd":
return False
return bool(cfg.dwd_key_path) and os.path.exists(cfg.dwd_key_path)
def _build_raw(from_addr, to_addrs, subject, body):
msg = email.message.EmailMessage()
msg["From"] = from_addr
msg["To"] = ", ".join(to_addrs)
msg["Subject"] = subject or "(no subject)"
msg.set_content(body or "")
return base64.urlsafe_b64encode(msg.as_bytes()).decode("ascii")
def send_via_gmail(sender_email, to_addrs, subject, body, conn=None):
"""Send one message as `sender_email` to `to_addrs` via the Gmail API (DWD).
Returns {'sent_to', 'from', 'message_id'}; raises on failure."""
if isinstance(to_addrs, str):
to_addrs = [to_addrs]
to_addrs = [a for a in (str(x).strip() for x in to_addrs) if a]
if not to_addrs:
raise ValueError("no recipients")
if not sender_email:
raise ValueError("no sender_email (DWD impersonation needs a domain user)")
provider = _creds.build_provider(lambda: conn)
token = provider.access_token_for(sender_email, _creds.GMAIL_COMPOSE_SCOPE).token
raw = _build_raw(sender_email, to_addrs, subject, body)
url = ("https://gmail.googleapis.com/gmail/v1/users/"
f"{urllib.parse.quote(sender_email)}/messages/send")
req = urllib.request.Request(
url, data=json.dumps({"raw": raw}).encode("utf-8"), method="POST",
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=20) as resp:
result = json.loads(resp.read())
except urllib.error.HTTPError as e:
detail = e.read().decode("utf-8", "replace")[:300]
raise RuntimeError(f"Gmail API send failed (HTTP {e.code}): {detail}")
return {"sent_to": to_addrs, "from": sender_email, "message_id": result.get("id")}
+40 -45
View File
@@ -4237,63 +4237,58 @@ class CRMHandler(BaseHTTPRequestHandler):
}) })
def handle_admin_send_test_email(self, user, body): def handle_admin_send_test_email(self, user, body):
"""Send a test email through the configured digest SMTP account, to prove """Send a test email through the active transport (Gmail DWD preferred,
the outbound pipe before the daily digest itself is built (Phase A).""" SMTP fallback) to prove the outbound pipe before the daily digest (Phase B)."""
if not require_admin(user): if not require_admin(user):
return self.send_error_json("Admin only", 403) return self.send_error_json("Admin only", 403)
from smtp_send import send_email, smtp_configured, SMTPNotConfigured import digest_mailer
if not smtp_configured():
return self.send_error_json(
"SMTP is not configured. Set it via the 'Configure Digest SMTP' "
"Start9 action (or SMTP_* in .env for dev), then restart.", 400)
# Recipients are restricted to the active-admin set (the real digest
# audience). An explicit `to` may NARROW to specific admins but can never
# introduce an outside address — this endpoint is not an open mail relay.
conn = get_db() conn = get_db()
try: try:
# Recipients are restricted to the active-admin set (the real digest
# audience). An explicit `to` may NARROW to specific admins but can never
# introduce an outside address — this endpoint is not an open mail relay.
rows = conn.execute( rows = conn.execute(
"SELECT email FROM users WHERE role = 'admin' AND is_active = 1 " "SELECT email FROM users WHERE role = 'admin' AND is_active = 1 "
"AND email IS NOT NULL AND TRIM(email) != ''" "AND email IS NOT NULL AND TRIM(email) != ''"
).fetchall() ).fetchall()
admin_emails = [str(r['email']).strip() for r in rows if str(r['email']).strip()]
admin_lower = {e.lower() for e in admin_emails}
to = body.get('to')
if to:
requested = [to] if isinstance(to, str) else list(to)
requested = [str(e).strip() for e in requested if str(e).strip()]
outside = [e for e in requested if e.lower() not in admin_lower]
if outside:
return self.send_error_json(
"Test email may only go to an active admin address; "
f"not allowed: {', '.join(outside)}", 400)
recipients = requested
else:
recipients = admin_emails
if not recipients:
return self.send_error_json(
"No recipient: give an active admin an email address first.", 400)
subject = "Ten31 CRM — test digest email"
email_body = (
"This is a test message from the Ten31 CRM daily-digest mailer.\n\n"
f"Triggered by {user.get('full_name') or user.get('user_id')} at {now()}.\n\n"
"If you received this, outbound email works and the daily digest can "
"be delivered to this address."
)
result = digest_mailer.send_digest(conn, recipients, subject, email_body)
except digest_mailer.NoTransport as exc:
return self.send_error_json(str(exc), 400)
except Exception as exc:
# Never echo the exception to the client — an auth error can carry a
# credential or token. Log it server-side instead.
print(f"[digest] test send failed: {type(exc).__name__}: {exc}", file=sys.stderr)
return self.send_error_json("Send failed — see server logs for details.", 502)
finally: finally:
conn.close() conn.close()
admin_emails = [str(r['email']).strip() for r in rows if str(r['email']).strip()]
admin_lower = {e.lower() for e in admin_emails}
to = body.get('to')
if to:
requested = [to] if isinstance(to, str) else list(to)
requested = [str(e).strip() for e in requested if str(e).strip()]
outside = [e for e in requested if e.lower() not in admin_lower]
if outside:
return self.send_error_json(
"Test email may only go to an active admin address; "
f"not allowed: {', '.join(outside)}", 400)
recipients = requested
else:
recipients = admin_emails
if not recipients:
return self.send_error_json(
"No recipient: give an active admin an email address first.", 400)
subject = "Ten31 CRM — test digest email"
email_body = (
"This is a test message from the Ten31 CRM daily-digest mailer.\n\n"
f"Triggered by {user.get('full_name') or user.get('user_id')} at {now()}.\n\n"
"If you received this, outbound SMTP works and the daily digest can be "
"delivered to this address."
)
try:
result = send_email(recipients, subject, email_body)
except SMTPNotConfigured as exc:
return self.send_error_json(f"SMTP not configured: {exc}", 400)
except Exception as exc:
# Never echo the exception to the client — an SMTP auth error can carry
# the server's reply (and potentially the credential). Log it instead.
print(f"[smtp] test send failed: {type(exc).__name__}: {exc}", file=sys.stderr)
return self.send_error_json("Send failed — see server logs for details.", 502)
return self.send_json({"data": {"status": "sent", **result}}) return self.send_json({"data": {"status": "sent", **result}})
+166
View File
@@ -0,0 +1,166 @@
#!/usr/bin/env python3
"""Tests for gmail_send (DWD Gmail-API send) + digest_mailer (transport routing).
No network: the credential provider and urllib.request.urlopen are monkeypatched.
Run directly or via backend/run_tests.py.
"""
import base64
import json
import os
import sys
import urllib.request
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from email_integration import gmail_send
import digest_mailer
import smtp_send
FAILS = []
def check(cond, msg):
print((" ok " if cond else " XX ") + msg)
if not cond:
FAILS.append(msg)
class _Tok:
def __init__(self, t):
self.token = t
class _Provider:
def __init__(self):
self.calls = []
def access_token_for(self, email, scope):
self.calls.append((email, scope))
return _Tok("tok-" + email)
class _Resp:
def __init__(self, payload):
self._p = payload
def read(self):
return json.dumps(self._p).encode()
def __enter__(self):
return self
def __exit__(self, *a):
return False
def main():
# 1. _build_raw round-trips
raw = gmail_send._build_raw("a@x", ["b@x", "c@x"], "Subj", "Hello body")
decoded = base64.urlsafe_b64decode(raw).decode()
check("From: a@x" in decoded and "Subject: Subj" in decoded
and "b@x, c@x" in decoded and "Hello body" in decoded,
"_build_raw round-trips From/To/Subject/body")
# 2. send_via_gmail posts correctly (fake provider + fake urlopen)
captured = {}
orig_open = urllib.request.urlopen
orig_build = gmail_send._creds.build_provider
prov = _Provider()
gmail_send._creds.build_provider = lambda factory: prov
def fake_open(req, timeout=None):
captured["url"] = req.full_url
captured["auth"] = req.headers.get("Authorization")
captured["body"] = json.loads(req.data.decode())
return _Resp({"id": "msg123", "threadId": "thr123"})
urllib.request.urlopen = fake_open
try:
res = gmail_send.send_via_gmail("grant@ten31.xyz",
["a@ten31.xyz", "b@ten31.xyz"], "S", "B")
finally:
urllib.request.urlopen = orig_open
gmail_send._creds.build_provider = orig_build
check(res["message_id"] == "msg123", "send returns message_id")
check(res["from"] == "grant@ten31.xyz", "send reports from")
check(res["sent_to"] == ["a@ten31.xyz", "b@ten31.xyz"], "send reports recipients")
check(captured["url"].endswith("/users/grant%40ten31.xyz/messages/send"),
"posts to messages/send for the impersonated user")
check(captured["auth"] == "Bearer tok-grant@ten31.xyz", "uses the compose-scoped token")
check("raw" in captured["body"] and "message" not in captured["body"],
"send body is {raw:...} (not nested under message, unlike drafts)")
check(prov.calls and prov.calls[0][1] == gmail_send._creds.GMAIL_COMPOSE_SCOPE,
"requests the gmail.compose scope")
# 3. validation
try:
gmail_send.send_via_gmail("x@x", [], "s", "b"); ok = False
except ValueError:
ok = True
check(ok, "empty recipients -> ValueError")
try:
gmail_send.send_via_gmail("", ["a@x"], "s", "b"); ok = False
except ValueError:
ok = True
check(ok, "missing sender -> ValueError")
# 4. transport() selection
orig_avail = gmail_send.gmail_available
orig_smtp_cfg = smtp_send.smtp_configured
try:
gmail_send.gmail_available = lambda: True
check(digest_mailer.transport() == "gmail-dwd", "transport prefers gmail-dwd")
gmail_send.gmail_available = lambda: False
smtp_send.smtp_configured = lambda: True
check(digest_mailer.transport() == "smtp", "transport falls back to smtp")
smtp_send.smtp_configured = lambda: False
check(digest_mailer.transport() is None, "transport None when neither configured")
finally:
gmail_send.gmail_available = orig_avail
smtp_send.smtp_configured = orig_smtp_cfg
# 5. send_digest routes to gmail + honors CRM_DIGEST_SENDER
orig_send = gmail_send.send_via_gmail
sent = {}
try:
gmail_send.gmail_available = lambda: True
def fake_send(sender, to, subj, body, conn=None):
sent.update(sender=sender, to=to)
return {"sent_to": to, "from": sender, "message_id": "m"}
gmail_send.send_via_gmail = fake_send
os.environ["CRM_DIGEST_SENDER"] = "digest@ten31.xyz"
res = digest_mailer.send_digest(None, ["a@ten31.xyz"], "S", "B")
check(res["transport"] == "gmail-dwd", "send_digest tags transport gmail-dwd")
check(sent["sender"] == "digest@ten31.xyz", "send_digest uses CRM_DIGEST_SENDER")
finally:
gmail_send.send_via_gmail = orig_send
gmail_send.gmail_available = orig_avail
os.environ.pop("CRM_DIGEST_SENDER", None)
# 6. send_digest raises NoTransport when neither is available
try:
gmail_send.gmail_available = lambda: False
smtp_send.smtp_configured = lambda: False
try:
digest_mailer.send_digest(None, ["a@x"], "S", "B"); ok = False
except digest_mailer.NoTransport:
ok = True
check(ok, "send_digest raises NoTransport when no transport")
finally:
gmail_send.gmail_available = orig_avail
smtp_send.smtp_configured = orig_smtp_cfg
print()
if FAILS:
print(f"FAILED ({len(FAILS)}):")
for f in FAILS:
print(" -", f)
raise SystemExit(1)
print("ALL PASS (gmail_send + digest_mailer)")
if __name__ == "__main__":
main()
+3 -2
View File
@@ -40,8 +40,9 @@ export const PACKAGE_TITLE = 'Ten31 Database'
// * 0.1.0:72 (stage v2.0 reserve-asset thesis spine as Workshop candidates) // * 0.1.0:72 (stage v2.0 reserve-asset thesis spine as Workshop candidates)
// * 0.1.0:73 (replace old settlement spine with v2.0 reserve-asset spine across Architect + outreach prompts, seed constants, and docs; promote v2.0 to the working approved spine + soft-retire old settlement nodes, reversibly, node-level only) // * 0.1.0:73 (replace old settlement spine with v2.0 reserve-asset spine across Architect + outreach prompts, seed constants, and docs; promote v2.0 to the working approved spine + soft-retire old settlement nodes, reversibly, node-level only)
// * 0.1.0:74 (security/privacy hardening — full-eval P0+2×P1: close /assets/ path traversal, add NER backstop to the outreach redaction boundary, filter deleted_at on get-by-id) // * 0.1.0:74 (security/privacy hardening — full-eval P0+2×P1: close /assets/ path traversal, add NER backstop to the outreach redaction boundary, filter deleted_at on get-by-id)
// * Current: 0.1.0:75 (Phase-A digest SMTP: per-package "Configure Digest SMTP" action writes /data/secrets/smtp/*; entrypoint exports SMTP_*; backend smtp_send.py + admin "send test email" endpoint + Settings→Admin "Send Test Digest Email" button) // * 0.1.0:75 (Phase-A digest SMTP: per-package "Configure Digest SMTP" action writes /data/secrets/smtp/*; entrypoint exports SMTP_*; backend smtp_send.py + admin "send test email" endpoint + Settings→Admin "Send Test Digest Email" button)
export const PACKAGE_VERSION = '0.1.0:75' // * Current: 0.1.0:76 (digest send via Gmail DWD: backend/email_integration/gmail_send.py uses the existing service account's gmail.compose scope for users.messages.send; digest_mailer prefers Gmail DWD and falls back to SMTP; the admin test endpoint + Settings button route through it — no app password needed when Gmail is enabled)
export const PACKAGE_VERSION = '0.1.0:76'
export const DATA_MOUNT_PATH = '/data' export const DATA_MOUNT_PATH = '/data'
export const WEB_PORT = 8080 export const WEB_PORT = 8080
+3 -2
View File
@@ -36,8 +36,9 @@ import { v_0_1_0_72 } from './v0.1.0.72'
import { v_0_1_0_73 } from './v0.1.0.73' import { v_0_1_0_73 } from './v0.1.0.73'
import { v_0_1_0_74 } from './v0.1.0.74' import { v_0_1_0_74 } from './v0.1.0.74'
import { v_0_1_0_75 } from './v0.1.0.75' import { v_0_1_0_75 } from './v0.1.0.75'
import { v_0_1_0_76 } from './v0.1.0.76'
export const versionGraph = VersionGraph.of({ export const versionGraph = VersionGraph.of({
current: v_0_1_0_75, current: v_0_1_0_76,
other: [v_0_1_0_39, v_0_1_0_40, v_0_1_0_41, v_0_1_0_42, v_0_1_0_43, v_0_1_0_44, v_0_1_0_45, v_0_1_0_46, v_0_1_0_47, v_0_1_0_48, v_0_1_0_49, v_0_1_0_50, v_0_1_0_51, v_0_1_0_52, v_0_1_0_53, v_0_1_0_54, v_0_1_0_55, v_0_1_0_56, v_0_1_0_57, v_0_1_0_58, v_0_1_0_59, v_0_1_0_60, v_0_1_0_61, v_0_1_0_62, v_0_1_0_63, v_0_1_0_64, v_0_1_0_65, v_0_1_0_66, v_0_1_0_67, v_0_1_0_68, v_0_1_0_69, v_0_1_0_70, v_0_1_0_71, v_0_1_0_72, v_0_1_0_73, v_0_1_0_74], other: [v_0_1_0_39, v_0_1_0_40, v_0_1_0_41, v_0_1_0_42, v_0_1_0_43, v_0_1_0_44, v_0_1_0_45, v_0_1_0_46, v_0_1_0_47, v_0_1_0_48, v_0_1_0_49, v_0_1_0_50, v_0_1_0_51, v_0_1_0_52, v_0_1_0_53, v_0_1_0_54, v_0_1_0_55, v_0_1_0_56, v_0_1_0_57, v_0_1_0_58, v_0_1_0_59, v_0_1_0_60, v_0_1_0_61, v_0_1_0_62, v_0_1_0_63, v_0_1_0_64, v_0_1_0_65, v_0_1_0_66, v_0_1_0_67, v_0_1_0_68, v_0_1_0_69, v_0_1_0_70, v_0_1_0_71, v_0_1_0_72, v_0_1_0_73, v_0_1_0_74, v_0_1_0_75],
}) })
+25
View File
@@ -0,0 +1,25 @@
import { VersionInfo } from '@start9labs/start-sdk'
// Digest send path via Gmail domain-wide delegation. Code-only, no schema change
// (migrations are no-ops):
// * The DWD grant on this deployment already includes the gmail.compose scope
// (verified 2026-06-15: token mint + a live messages.send both succeed), and
// gmail.compose authorizes users.messages.send. So CRM-originated mail can go
// through the existing service account — no SMTP account / app password.
// * backend/email_integration/gmail_send.py: send_via_gmail() impersonates a
// domain user and POSTs users.messages.send (mirrors compose.py's REST path).
// * backend/digest_mailer.py: send_digest() prefers Gmail DWD when enabled and
// falls back to smtp_send when not. The admin POST /api/admin/digest/test-email
// and the Settings "Send Test Digest Email" button route through it.
// * Sender for the DWD path is CRM_DIGEST_SENDER, else the first active admin.
export const v_0_1_0_76 = VersionInfo.of({
version: '0.1.0:76',
releaseNotes: {
en_US: [
'The daily-digest mailer can now send through the Gmail account the CRM already uses for email',
'capture (domain-wide delegation) — no separate SMTP account or app password needed. SMTP stays',
'available as a fallback.',
].join(' '),
},
migrations: { up: async () => {}, down: async () => {} },
})