Add Gmail-DWD send path for the digest mailer (v0.1.0:76)
The box's existing service-account domain-wide-delegation grant already includes gmail.compose, which authorizes users.messages.send — verified 2026-06-15 by a token-mint probe and a live messages.send to grant. So CRM-originated mail can send through the account that already powers email capture: no SMTP account, no app password, no admin change. - backend/email_integration/gmail_send.py: send_via_gmail() impersonates a domain user and POSTs users.messages.send (reuses credentials.py + the compose scope; mirrors compose.py's REST pattern). - backend/digest_mailer.py: send_digest() prefers Gmail DWD when enabled, falls back to smtp_send otherwise. Sender = CRM_DIGEST_SENDER else first active admin. - server.py: the admin test endpoint now routes through digest_mailer (so the Settings button sends via DWD on the box with zero SMTP config). Recipient restriction to the admin set and no-leak error handling preserved. - test_gmail_send.py: build/send + transport routing (provider + urlopen faked). 19/19 backend green; s9pk typechecks. SMTP (v75) stays as the fallback transport. Send-path decision + scope finding recorded in ROADMAP.md and AGENTS.md.
This commit is contained in:
Keysat
+8
-1
@@ -20,7 +20,14 @@ X_API_KEY=
|
||||
CRM_DB_PATH=./data/crm.db
|
||||
CRM_DEV_DB_PATH=./data/crm_dev.db
|
||||
|
||||
# ── Daily-digest outbound SMTP (dev override of the per-package mailbox) ──
|
||||
# ── Daily-digest sender ──
|
||||
# The digest mailer prefers Gmail domain-wide delegation (the service account that
|
||||
# already powers email capture; its grant includes gmail.compose, which can send) and
|
||||
# falls back to SMTP below. For the Gmail/DWD path it sends impersonating this domain
|
||||
# user; if unset, it uses the first active admin's email.
|
||||
CRM_DIGEST_SENDER=
|
||||
|
||||
# ── Daily-digest outbound SMTP fallback (dev override of the per-package mailbox) ──
|
||||
# On the Start9 box these are set by the "Configure Digest SMTP" action (written
|
||||
# to /data/secrets/smtp/* and exported by docker_entrypoint.sh). For dev, set them
|
||||
# here. SMTP_SECURITY is one of: starttls (587) | tls (465) | none.
|
||||
|
||||
@@ -100,14 +100,15 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
|
||||
|
||||
## Current state
|
||||
|
||||
_Phase 0 substrate + Phase 1 thesis/outreach are built; **box and repo both at v0.1.0:75** (deployed & verified live 2026-06-15). Longer-term backlog: `ROADMAP.md`._
|
||||
_Phase 0 substrate + Phase 1 thesis/outreach are built; **box at v0.1.0:75, repo at v0.1.0:76** (the Gmail-DWD digest-send path; committed, redeploy pending). Longer-term backlog: `ROADMAP.md`._
|
||||
|
||||
- **Working (all draft-only):** CRM + ingest (chunk→embed→Qdrant + retrieval) + redaction boundary; Gmail capture (DWD) + email-activity propose→approve; Thesis Workshop + Architect (Claude) with dual-approval gate; Outreach Draft Assistant + follow-up radar + per-user voice + Tier-B in-thread Gmail draft creation.
|
||||
- **Deployed & verified live (2026-06-15): v0.1.0:75** on the box (`$START9_BOX_HOST` / immense-voyage.local). `start-cli package installed-version ten-database` → `0.1.0:75`; boot log shows the v75-only entrypoint line `[entrypoint] Digest SMTP: not configured` and the server healthy on `:8080`. Shipped two batches at once: (a) the post-v74 **list-view soft-delete aggregate fix** (`server.py`: org `contact_count`/`total_funded`, contacts `comm_count`/`last_contact_date` now filter `deleted_at`) + 3 regression tests + aggregate runner; (b) **daily-digest Phase A** — `configureDigestSmtp` action writes a per-package SMTP account to `/data/secrets/smtp/*` (password over stdin; independent of any StartOS system SMTP), `docker_entrypoint.sh` exports `SMTP_*`, `backend/smtp_send.py` (stdlib smtplib), admin **`POST /api/admin/digest/test-email`** (recipients restricted to the active-admin set — not an open relay), and a **Settings → Admin "Send Test Digest Email" button**. **SMTP not yet configured on the box** — Grant runs the action + restart + test to light it up.
|
||||
- **Deployed & verified live (2026-06-15): v0.1.0:75** on the box (`$START9_BOX_HOST` / immense-voyage.local). `start-cli package installed-version ten-database` → `0.1.0:75`; boot log shows the v75-only entrypoint line `[entrypoint] Digest SMTP: not configured` and the server healthy on `:8080`. Shipped two batches at once: (a) the post-v74 **list-view soft-delete aggregate fix** (`server.py`: org `contact_count`/`total_funded`, contacts `comm_count`/`last_contact_date` now filter `deleted_at`) + 3 regression tests + aggregate runner; (b) **daily-digest Phase A** — `configureDigestSmtp` action writes a per-package SMTP account to `/data/secrets/smtp/*` (password over stdin; independent of any StartOS system SMTP), `docker_entrypoint.sh` exports `SMTP_*`, `backend/smtp_send.py` (stdlib smtplib), admin **`POST /api/admin/digest/test-email`** (recipients restricted to the active-admin set — not an open relay), and a **Settings → Admin "Send Test Digest Email" button**.
|
||||
- **Repo ahead (v0.1.0:76 — committed, redeploy pending):** the digest send path now prefers **Gmail domain-wide delegation** over SMTP. The box's DWD grant **includes `gmail.compose`** (send-capable; the narrow `gmail.send` is *not* granted) — verified 2026-06-15 by a token-mint probe **and a live `messages.send` to grant** (both succeeded). `backend/email_integration/gmail_send.py` impersonates a domain user and calls `users.messages.send` (reuses `credentials.py` + the compose scope, mirrors `compose.py`); `backend/digest_mailer.py` routes **Gmail-DWD → SMTP fallback**; the admin test endpoint + Settings button go through it. Sender = `CRM_DIGEST_SENDER` else the first active admin. **Net: no app password needed** — once v76 is redeployed, the test button sends via DWD with zero SMTP config. SMTP (v75) remains the fallback.
|
||||
- **Live since v74 (2026-06-13):** login works; `/assets/` traversal 404s (plain + URL-encoded), root health 200. On boot, `ensure_thesis_v2_promoted` makes the v2.0 reserve-asset spine the working *approved* spine (node-level, reversible).
|
||||
- **Shipped in v0.1.0:74** (security/privacy hardening from the 2026-06-12 full-eval; report in `EVALUATION.md`): closed a pre-auth `/assets/` path traversal (could read crm.db / JWT secret / Gmail key); wired the local-Qwen NER backstop into the outreach redaction boundary (free-prose email bodies were reaching Claude with unknown names in the clear); added `deleted_at IS NULL` to every get-by-id + nested sub-select read path. Verified locally (py_compile, query exec, redaction/outreach tests, containment logic) + two reviewer passes.
|
||||
- **Tests (2026-06-15):** **18/18 backend tests green** via `python3 backend/run_tests.py` (+`test_smtp_send.py`/`test_smtp_endpoint.py` this session). `py_compile` clean; the s9pk TypeScript typechecks (`cd start9/0.4 && npm run check`, deps installed); `docker_entrypoint.sh` passes `sh -n`. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`).
|
||||
- **Tests (2026-06-15):** **19/19 backend tests green** via `python3 backend/run_tests.py` (+`test_smtp_send.py`/`test_smtp_endpoint.py`/`test_gmail_send.py` this session). `py_compile` clean; the s9pk TypeScript typechecks (`cd start9/0.4 && npm run check`, deps installed); `docker_entrypoint.sh` passes `sh -n`. The 2 stale thesis tests stay fixed (seed structure in `docs/guides/thesis.md`).
|
||||
- **Decided, not yet built:** CRM as canonical thesis backbone with the signal-engine reading from it (reconciliation unwired); reply-all for Tier-B drafts (drafts currently reply to the LP only).
|
||||
- **Known debt (P2, not deploy-blocking):** the **reports subsystem** (`handle_dashboard_report`/`handle_pipeline_report`/`handle_lp_breakdown_report`, ~16 aggregate queries over contacts/opportunities/communications/lp_profiles) still counts soft-deleted rows — the list/detail aggregates were fixed (v74 + the org/contacts list-view follow-up) but the reports were not; needs its own pass + report-endpoint tests; `?limit=abc` crashes the request thread (authenticated list path); scrub-gateway TLS verify off; `cryptography==42.0.5`; unpkg/no-SRI frontend; stale user-visible `start9/0.4/assets/ABOUT.md`; hardcoded Spark/Qdrant IPs in the s9pk; the 5.4k-line `server.py` monolith. P3 batch + full list in `EVALUATION.md`.
|
||||
- **Other gaps:** the v2.0 spine is the *working* spine but **not a canonical `thesis_version`** (needs Grant + Jonathan dual sign-off); Appendix-A conviction/exposure (incl. ~40% Strike) stay Grant's working read, not canonical, not fed to the engine; live features (Claude/Qdrant/Gmail) unverified on the box.
|
||||
- **Next:** 1) **on the box: configure the digest mailbox** — run **Configure Digest SMTP** (dedicated SMTP creds), restart, then Settings→Admin **Send Test Digest Email** to confirm a real message lands; 2) **digest Phase B** — daily scheduler + per-user→per-investor activity query (`deleted_at IS NULL`) + **Spark-narrative** summary (never Claude) → email all admins (decisions locked in `ROADMAP.md`); 3) **reports-subsystem soft-delete sweep** (~16 aggregates still leak; fix + tests); 4) `?limit=abc` crash (P2); 5) Grant + Jonathan freeze v2.0 canonical; 6) build reply-all; 7) confirm Appendix-A + Maple/OpenSecret/Primal, then promote.
|
||||
- **Next:** 1) **redeploy v0.1.0:76**, then Settings→Admin **Send Test Digest Email** — it should send via **Gmail DWD** to all admins with no SMTP setup (the live DWD send is already proven; this confirms it through the app path); 2) **digest Phase B** — daily scheduler + per-user→per-investor activity query (`deleted_at IS NULL`) + **Spark-narrative** summary (never Claude) → email all admins (decisions locked in `ROADMAP.md`); 3) **reports-subsystem soft-delete sweep** (~16 aggregates still leak; fix + tests); 4) `?limit=abc` crash (P2); 5) Grant + Jonathan freeze v2.0 canonical; 6) build reply-all; 7) confirm Appendix-A + Maple/OpenSecret/Primal, then promote.
|
||||
|
||||
+4
-2
@@ -87,11 +87,13 @@
|
||||
## Backlog (post-Phase-1 agentic)
|
||||
|
||||
### Daily activity digest (email to the team)
|
||||
*Requested 2026-06-15. **Phase A built in v0.1.0:75** (outbound SMTP send capability + admin test-email endpoint; not yet deployed). Phase B (digest content + Spark summarization + daily scheduler) remains.*
|
||||
*Requested 2026-06-15. **Phase A built + deployed** (v0.1.0:75 live on the box; send path then moved to **Gmail DWD** in v0.1.0:76, redeploy pending). Phase B (digest content + Spark summarization + daily scheduler) remains.*
|
||||
|
||||
**Decisions (locked 2026-06-15):** recipients = **all active admins**; summarization = **Spark-LLM narrative** (never Claude — un-anonymized substance stays local); granularity = **grouped by user** (→ per investor).
|
||||
|
||||
**Phase A — DONE (v0.1.0:75):** `configureDigestSmtp` Start9 action writes a per-package SMTP account to `/data/secrets/smtp/*`; `docker_entrypoint.sh` exports `SMTP_*`; `backend/smtp_send.py` (stdlib smtplib) + admin `POST /api/admin/digest/test-email` (recipient-restricted to the admin set — not an open relay). Tests: `test_smtp_send.py`, `test_smtp_endpoint.py`.
|
||||
**Send transport — DECIDED 2026-06-15: Gmail domain-wide delegation** (not SMTP). The box's existing service-account grant (which powers email capture) **includes `gmail.compose`**, which authorizes `users.messages.send` — verified by a token-mint probe **and a live `messages.send` to grant**. So the digest sends through the account the CRM already uses: **no app password, no new account, no admin change.** The narrow `gmail.send` scope is *not* granted, so the sender must request `gmail.compose`.
|
||||
|
||||
**Phase A — DONE:** (v0.1.0:75) `configureDigestSmtp` Start9 action + `docker_entrypoint.sh` `SMTP_*` export + `backend/smtp_send.py` + admin `POST /api/admin/digest/test-email` (recipient-restricted to the admin set — not an open relay) + Settings button. (v0.1.0:76, redeploy pending) `backend/email_integration/gmail_send.py` (`users.messages.send` via DWD/compose) + `backend/digest_mailer.py` (**Gmail-DWD preferred, SMTP fallback**); the endpoint + button route through it; sender = `CRM_DIGEST_SENDER` else first active admin. Tests: `test_smtp_send.py`, `test_smtp_endpoint.py`, `test_gmail_send.py`.
|
||||
|
||||
**Phase B — TODO:** daily scheduler (co-locate with `email_integration/scheduler.py`); per-user→per-investor activity query (`deleted_at IS NULL` throughout); Spark-narrative summary of captured email substance; compose + send to all admins.
|
||||
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
"""Transport selection for CRM-originated email (daily digest, admin test sends).
|
||||
|
||||
Prefers Gmail-over-DWD — it reuses the service account that already powers email
|
||||
capture (the grant includes gmail.compose, which can send), so there's no extra
|
||||
credential to manage — and falls back to SMTP (`smtp_send`) when DWD isn't
|
||||
available. One entry point so the digest and the admin test endpoint share the
|
||||
same routing. Stdlib only.
|
||||
"""
|
||||
import os
|
||||
|
||||
|
||||
class NoTransport(Exception):
|
||||
"""Neither Gmail DWD nor SMTP is configured."""
|
||||
|
||||
|
||||
def transport():
|
||||
"""Return the active transport: 'gmail-dwd', 'smtp', or None."""
|
||||
try:
|
||||
from email_integration import gmail_send
|
||||
if gmail_send.gmail_available():
|
||||
return "gmail-dwd"
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
import smtp_send
|
||||
if smtp_send.smtp_configured():
|
||||
return "smtp"
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def default_sender(conn):
|
||||
"""Domain user to send as for the DWD path. `CRM_DIGEST_SENDER` if set, else
|
||||
the first active admin's email."""
|
||||
s = os.environ.get("CRM_DIGEST_SENDER", "").strip()
|
||||
if s:
|
||||
return s
|
||||
if conn is None:
|
||||
return None
|
||||
row = conn.execute(
|
||||
"SELECT email FROM users WHERE role='admin' AND is_active=1 "
|
||||
"AND email IS NOT NULL AND TRIM(email)!='' ORDER BY created_at LIMIT 1"
|
||||
).fetchone()
|
||||
return row["email"].strip() if row and row["email"] else None
|
||||
|
||||
|
||||
def send_digest(conn, to_addrs, subject, body, sender_email=None):
|
||||
"""Send via the active transport. Returns the transport's result dict with a
|
||||
'transport' key added; raises NoTransport if neither is configured."""
|
||||
t = transport()
|
||||
if t == "gmail-dwd":
|
||||
from email_integration import gmail_send
|
||||
sender = sender_email or default_sender(conn)
|
||||
result = gmail_send.send_via_gmail(sender, to_addrs, subject, body, conn=conn)
|
||||
result["transport"] = "gmail-dwd"
|
||||
return result
|
||||
if t == "smtp":
|
||||
import smtp_send
|
||||
result = smtp_send.send_email(to_addrs, subject, body)
|
||||
result["transport"] = "smtp"
|
||||
return result
|
||||
raise NoTransport("No email transport configured: enable Gmail (DWD) or set "
|
||||
"SMTP via the 'Configure Digest SMTP' action.")
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Send an email via the Gmail API using the same domain-wide delegation that
|
||||
powers capture and draft creation.
|
||||
|
||||
The DWD grant on this deployment includes the `gmail.compose` scope (verified
|
||||
2026-06-15: token mint + a live messages.send both succeed), and `gmail.compose`
|
||||
authorizes `users.messages.send`. So CRM-originated mail (the daily digest) can
|
||||
send through the existing service account — no SMTP account, no app password, no
|
||||
admin change. Sends impersonating `sender_email`, which must be a Workspace user
|
||||
in the delegated domain. Mirrors the REST pattern in compose.py; stdlib only.
|
||||
"""
|
||||
import base64
|
||||
import email.message
|
||||
import json
|
||||
import os
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from . import config as _cfg
|
||||
from . import credentials as _creds
|
||||
|
||||
|
||||
def gmail_available():
|
||||
"""True when DWD send is usable: integration enabled, DWD auth, key present."""
|
||||
cfg = _cfg.CONFIG
|
||||
if not cfg.enabled or cfg.primary_auth != "dwd":
|
||||
return False
|
||||
return bool(cfg.dwd_key_path) and os.path.exists(cfg.dwd_key_path)
|
||||
|
||||
|
||||
def _build_raw(from_addr, to_addrs, subject, body):
|
||||
msg = email.message.EmailMessage()
|
||||
msg["From"] = from_addr
|
||||
msg["To"] = ", ".join(to_addrs)
|
||||
msg["Subject"] = subject or "(no subject)"
|
||||
msg.set_content(body or "")
|
||||
return base64.urlsafe_b64encode(msg.as_bytes()).decode("ascii")
|
||||
|
||||
|
||||
def send_via_gmail(sender_email, to_addrs, subject, body, conn=None):
|
||||
"""Send one message as `sender_email` to `to_addrs` via the Gmail API (DWD).
|
||||
Returns {'sent_to', 'from', 'message_id'}; raises on failure."""
|
||||
if isinstance(to_addrs, str):
|
||||
to_addrs = [to_addrs]
|
||||
to_addrs = [a for a in (str(x).strip() for x in to_addrs) if a]
|
||||
if not to_addrs:
|
||||
raise ValueError("no recipients")
|
||||
if not sender_email:
|
||||
raise ValueError("no sender_email (DWD impersonation needs a domain user)")
|
||||
|
||||
provider = _creds.build_provider(lambda: conn)
|
||||
token = provider.access_token_for(sender_email, _creds.GMAIL_COMPOSE_SCOPE).token
|
||||
raw = _build_raw(sender_email, to_addrs, subject, body)
|
||||
url = ("https://gmail.googleapis.com/gmail/v1/users/"
|
||||
f"{urllib.parse.quote(sender_email)}/messages/send")
|
||||
req = urllib.request.Request(
|
||||
url, data=json.dumps({"raw": raw}).encode("utf-8"), method="POST",
|
||||
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as resp:
|
||||
result = json.loads(resp.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
detail = e.read().decode("utf-8", "replace")[:300]
|
||||
raise RuntimeError(f"Gmail API send failed (HTTP {e.code}): {detail}")
|
||||
return {"sent_to": to_addrs, "from": sender_email, "message_id": result.get("id")}
|
||||
+40
-45
@@ -4237,63 +4237,58 @@ class CRMHandler(BaseHTTPRequestHandler):
|
||||
})
|
||||
|
||||
def handle_admin_send_test_email(self, user, body):
|
||||
"""Send a test email through the configured digest SMTP account, to prove
|
||||
the outbound pipe before the daily digest itself is built (Phase A)."""
|
||||
"""Send a test email through the active transport (Gmail DWD preferred,
|
||||
SMTP fallback) to prove the outbound pipe before the daily digest (Phase B)."""
|
||||
if not require_admin(user):
|
||||
return self.send_error_json("Admin only", 403)
|
||||
|
||||
from smtp_send import send_email, smtp_configured, SMTPNotConfigured
|
||||
if not smtp_configured():
|
||||
return self.send_error_json(
|
||||
"SMTP is not configured. Set it via the 'Configure Digest SMTP' "
|
||||
"Start9 action (or SMTP_* in .env for dev), then restart.", 400)
|
||||
import digest_mailer
|
||||
|
||||
# Recipients are restricted to the active-admin set (the real digest
|
||||
# audience). An explicit `to` may NARROW to specific admins but can never
|
||||
# introduce an outside address — this endpoint is not an open mail relay.
|
||||
conn = get_db()
|
||||
try:
|
||||
# Recipients are restricted to the active-admin set (the real digest
|
||||
# audience). An explicit `to` may NARROW to specific admins but can never
|
||||
# introduce an outside address — this endpoint is not an open mail relay.
|
||||
rows = conn.execute(
|
||||
"SELECT email FROM users WHERE role = 'admin' AND is_active = 1 "
|
||||
"AND email IS NOT NULL AND TRIM(email) != ''"
|
||||
).fetchall()
|
||||
admin_emails = [str(r['email']).strip() for r in rows if str(r['email']).strip()]
|
||||
admin_lower = {e.lower() for e in admin_emails}
|
||||
|
||||
to = body.get('to')
|
||||
if to:
|
||||
requested = [to] if isinstance(to, str) else list(to)
|
||||
requested = [str(e).strip() for e in requested if str(e).strip()]
|
||||
outside = [e for e in requested if e.lower() not in admin_lower]
|
||||
if outside:
|
||||
return self.send_error_json(
|
||||
"Test email may only go to an active admin address; "
|
||||
f"not allowed: {', '.join(outside)}", 400)
|
||||
recipients = requested
|
||||
else:
|
||||
recipients = admin_emails
|
||||
if not recipients:
|
||||
return self.send_error_json(
|
||||
"No recipient: give an active admin an email address first.", 400)
|
||||
|
||||
subject = "Ten31 CRM — test digest email"
|
||||
email_body = (
|
||||
"This is a test message from the Ten31 CRM daily-digest mailer.\n\n"
|
||||
f"Triggered by {user.get('full_name') or user.get('user_id')} at {now()}.\n\n"
|
||||
"If you received this, outbound email works and the daily digest can "
|
||||
"be delivered to this address."
|
||||
)
|
||||
result = digest_mailer.send_digest(conn, recipients, subject, email_body)
|
||||
except digest_mailer.NoTransport as exc:
|
||||
return self.send_error_json(str(exc), 400)
|
||||
except Exception as exc:
|
||||
# Never echo the exception to the client — an auth error can carry a
|
||||
# credential or token. Log it server-side instead.
|
||||
print(f"[digest] test send failed: {type(exc).__name__}: {exc}", file=sys.stderr)
|
||||
return self.send_error_json("Send failed — see server logs for details.", 502)
|
||||
finally:
|
||||
conn.close()
|
||||
admin_emails = [str(r['email']).strip() for r in rows if str(r['email']).strip()]
|
||||
admin_lower = {e.lower() for e in admin_emails}
|
||||
|
||||
to = body.get('to')
|
||||
if to:
|
||||
requested = [to] if isinstance(to, str) else list(to)
|
||||
requested = [str(e).strip() for e in requested if str(e).strip()]
|
||||
outside = [e for e in requested if e.lower() not in admin_lower]
|
||||
if outside:
|
||||
return self.send_error_json(
|
||||
"Test email may only go to an active admin address; "
|
||||
f"not allowed: {', '.join(outside)}", 400)
|
||||
recipients = requested
|
||||
else:
|
||||
recipients = admin_emails
|
||||
if not recipients:
|
||||
return self.send_error_json(
|
||||
"No recipient: give an active admin an email address first.", 400)
|
||||
|
||||
subject = "Ten31 CRM — test digest email"
|
||||
email_body = (
|
||||
"This is a test message from the Ten31 CRM daily-digest mailer.\n\n"
|
||||
f"Triggered by {user.get('full_name') or user.get('user_id')} at {now()}.\n\n"
|
||||
"If you received this, outbound SMTP works and the daily digest can be "
|
||||
"delivered to this address."
|
||||
)
|
||||
try:
|
||||
result = send_email(recipients, subject, email_body)
|
||||
except SMTPNotConfigured as exc:
|
||||
return self.send_error_json(f"SMTP not configured: {exc}", 400)
|
||||
except Exception as exc:
|
||||
# Never echo the exception to the client — an SMTP auth error can carry
|
||||
# the server's reply (and potentially the credential). Log it instead.
|
||||
print(f"[smtp] test send failed: {type(exc).__name__}: {exc}", file=sys.stderr)
|
||||
return self.send_error_json("Send failed — see server logs for details.", 502)
|
||||
|
||||
return self.send_json({"data": {"status": "sent", **result}})
|
||||
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for gmail_send (DWD Gmail-API send) + digest_mailer (transport routing).
|
||||
|
||||
No network: the credential provider and urllib.request.urlopen are monkeypatched.
|
||||
Run directly or via backend/run_tests.py.
|
||||
"""
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from email_integration import gmail_send
|
||||
import digest_mailer
|
||||
import smtp_send
|
||||
|
||||
FAILS = []
|
||||
|
||||
|
||||
def check(cond, msg):
|
||||
print((" ok " if cond else " XX ") + msg)
|
||||
if not cond:
|
||||
FAILS.append(msg)
|
||||
|
||||
|
||||
class _Tok:
|
||||
def __init__(self, t):
|
||||
self.token = t
|
||||
|
||||
|
||||
class _Provider:
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
|
||||
def access_token_for(self, email, scope):
|
||||
self.calls.append((email, scope))
|
||||
return _Tok("tok-" + email)
|
||||
|
||||
|
||||
class _Resp:
|
||||
def __init__(self, payload):
|
||||
self._p = payload
|
||||
|
||||
def read(self):
|
||||
return json.dumps(self._p).encode()
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *a):
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
# 1. _build_raw round-trips
|
||||
raw = gmail_send._build_raw("a@x", ["b@x", "c@x"], "Subj", "Hello body")
|
||||
decoded = base64.urlsafe_b64decode(raw).decode()
|
||||
check("From: a@x" in decoded and "Subject: Subj" in decoded
|
||||
and "b@x, c@x" in decoded and "Hello body" in decoded,
|
||||
"_build_raw round-trips From/To/Subject/body")
|
||||
|
||||
# 2. send_via_gmail posts correctly (fake provider + fake urlopen)
|
||||
captured = {}
|
||||
orig_open = urllib.request.urlopen
|
||||
orig_build = gmail_send._creds.build_provider
|
||||
prov = _Provider()
|
||||
gmail_send._creds.build_provider = lambda factory: prov
|
||||
|
||||
def fake_open(req, timeout=None):
|
||||
captured["url"] = req.full_url
|
||||
captured["auth"] = req.headers.get("Authorization")
|
||||
captured["body"] = json.loads(req.data.decode())
|
||||
return _Resp({"id": "msg123", "threadId": "thr123"})
|
||||
|
||||
urllib.request.urlopen = fake_open
|
||||
try:
|
||||
res = gmail_send.send_via_gmail("grant@ten31.xyz",
|
||||
["a@ten31.xyz", "b@ten31.xyz"], "S", "B")
|
||||
finally:
|
||||
urllib.request.urlopen = orig_open
|
||||
gmail_send._creds.build_provider = orig_build
|
||||
|
||||
check(res["message_id"] == "msg123", "send returns message_id")
|
||||
check(res["from"] == "grant@ten31.xyz", "send reports from")
|
||||
check(res["sent_to"] == ["a@ten31.xyz", "b@ten31.xyz"], "send reports recipients")
|
||||
check(captured["url"].endswith("/users/grant%40ten31.xyz/messages/send"),
|
||||
"posts to messages/send for the impersonated user")
|
||||
check(captured["auth"] == "Bearer tok-grant@ten31.xyz", "uses the compose-scoped token")
|
||||
check("raw" in captured["body"] and "message" not in captured["body"],
|
||||
"send body is {raw:...} (not nested under message, unlike drafts)")
|
||||
check(prov.calls and prov.calls[0][1] == gmail_send._creds.GMAIL_COMPOSE_SCOPE,
|
||||
"requests the gmail.compose scope")
|
||||
|
||||
# 3. validation
|
||||
try:
|
||||
gmail_send.send_via_gmail("x@x", [], "s", "b"); ok = False
|
||||
except ValueError:
|
||||
ok = True
|
||||
check(ok, "empty recipients -> ValueError")
|
||||
try:
|
||||
gmail_send.send_via_gmail("", ["a@x"], "s", "b"); ok = False
|
||||
except ValueError:
|
||||
ok = True
|
||||
check(ok, "missing sender -> ValueError")
|
||||
|
||||
# 4. transport() selection
|
||||
orig_avail = gmail_send.gmail_available
|
||||
orig_smtp_cfg = smtp_send.smtp_configured
|
||||
try:
|
||||
gmail_send.gmail_available = lambda: True
|
||||
check(digest_mailer.transport() == "gmail-dwd", "transport prefers gmail-dwd")
|
||||
gmail_send.gmail_available = lambda: False
|
||||
smtp_send.smtp_configured = lambda: True
|
||||
check(digest_mailer.transport() == "smtp", "transport falls back to smtp")
|
||||
smtp_send.smtp_configured = lambda: False
|
||||
check(digest_mailer.transport() is None, "transport None when neither configured")
|
||||
finally:
|
||||
gmail_send.gmail_available = orig_avail
|
||||
smtp_send.smtp_configured = orig_smtp_cfg
|
||||
|
||||
# 5. send_digest routes to gmail + honors CRM_DIGEST_SENDER
|
||||
orig_send = gmail_send.send_via_gmail
|
||||
sent = {}
|
||||
try:
|
||||
gmail_send.gmail_available = lambda: True
|
||||
|
||||
def fake_send(sender, to, subj, body, conn=None):
|
||||
sent.update(sender=sender, to=to)
|
||||
return {"sent_to": to, "from": sender, "message_id": "m"}
|
||||
|
||||
gmail_send.send_via_gmail = fake_send
|
||||
os.environ["CRM_DIGEST_SENDER"] = "digest@ten31.xyz"
|
||||
res = digest_mailer.send_digest(None, ["a@ten31.xyz"], "S", "B")
|
||||
check(res["transport"] == "gmail-dwd", "send_digest tags transport gmail-dwd")
|
||||
check(sent["sender"] == "digest@ten31.xyz", "send_digest uses CRM_DIGEST_SENDER")
|
||||
finally:
|
||||
gmail_send.send_via_gmail = orig_send
|
||||
gmail_send.gmail_available = orig_avail
|
||||
os.environ.pop("CRM_DIGEST_SENDER", None)
|
||||
|
||||
# 6. send_digest raises NoTransport when neither is available
|
||||
try:
|
||||
gmail_send.gmail_available = lambda: False
|
||||
smtp_send.smtp_configured = lambda: False
|
||||
try:
|
||||
digest_mailer.send_digest(None, ["a@x"], "S", "B"); ok = False
|
||||
except digest_mailer.NoTransport:
|
||||
ok = True
|
||||
check(ok, "send_digest raises NoTransport when no transport")
|
||||
finally:
|
||||
gmail_send.gmail_available = orig_avail
|
||||
smtp_send.smtp_configured = orig_smtp_cfg
|
||||
|
||||
print()
|
||||
if FAILS:
|
||||
print(f"FAILED ({len(FAILS)}):")
|
||||
for f in FAILS:
|
||||
print(" -", f)
|
||||
raise SystemExit(1)
|
||||
print("ALL PASS (gmail_send + digest_mailer)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -40,8 +40,9 @@ export const PACKAGE_TITLE = 'Ten31 Database'
|
||||
// * 0.1.0:72 (stage v2.0 reserve-asset thesis spine as Workshop candidates)
|
||||
// * 0.1.0:73 (replace old settlement spine with v2.0 reserve-asset spine across Architect + outreach prompts, seed constants, and docs; promote v2.0 to the working approved spine + soft-retire old settlement nodes, reversibly, node-level only)
|
||||
// * 0.1.0:74 (security/privacy hardening — full-eval P0+2×P1: close /assets/ path traversal, add NER backstop to the outreach redaction boundary, filter deleted_at on get-by-id)
|
||||
// * Current: 0.1.0:75 (Phase-A digest SMTP: per-package "Configure Digest SMTP" action writes /data/secrets/smtp/*; entrypoint exports SMTP_*; backend smtp_send.py + admin "send test email" endpoint + Settings→Admin "Send Test Digest Email" button)
|
||||
export const PACKAGE_VERSION = '0.1.0:75'
|
||||
// * 0.1.0:75 (Phase-A digest SMTP: per-package "Configure Digest SMTP" action writes /data/secrets/smtp/*; entrypoint exports SMTP_*; backend smtp_send.py + admin "send test email" endpoint + Settings→Admin "Send Test Digest Email" button)
|
||||
// * Current: 0.1.0:76 (digest send via Gmail DWD: backend/email_integration/gmail_send.py uses the existing service account's gmail.compose scope for users.messages.send; digest_mailer prefers Gmail DWD and falls back to SMTP; the admin test endpoint + Settings button route through it — no app password needed when Gmail is enabled)
|
||||
export const PACKAGE_VERSION = '0.1.0:76'
|
||||
|
||||
export const DATA_MOUNT_PATH = '/data'
|
||||
export const WEB_PORT = 8080
|
||||
|
||||
@@ -36,8 +36,9 @@ import { v_0_1_0_72 } from './v0.1.0.72'
|
||||
import { v_0_1_0_73 } from './v0.1.0.73'
|
||||
import { v_0_1_0_74 } from './v0.1.0.74'
|
||||
import { v_0_1_0_75 } from './v0.1.0.75'
|
||||
import { v_0_1_0_76 } from './v0.1.0.76'
|
||||
|
||||
export const versionGraph = VersionGraph.of({
|
||||
current: v_0_1_0_75,
|
||||
other: [v_0_1_0_39, v_0_1_0_40, v_0_1_0_41, v_0_1_0_42, v_0_1_0_43, v_0_1_0_44, v_0_1_0_45, v_0_1_0_46, v_0_1_0_47, v_0_1_0_48, v_0_1_0_49, v_0_1_0_50, v_0_1_0_51, v_0_1_0_52, v_0_1_0_53, v_0_1_0_54, v_0_1_0_55, v_0_1_0_56, v_0_1_0_57, v_0_1_0_58, v_0_1_0_59, v_0_1_0_60, v_0_1_0_61, v_0_1_0_62, v_0_1_0_63, v_0_1_0_64, v_0_1_0_65, v_0_1_0_66, v_0_1_0_67, v_0_1_0_68, v_0_1_0_69, v_0_1_0_70, v_0_1_0_71, v_0_1_0_72, v_0_1_0_73, v_0_1_0_74],
|
||||
current: v_0_1_0_76,
|
||||
other: [v_0_1_0_39, v_0_1_0_40, v_0_1_0_41, v_0_1_0_42, v_0_1_0_43, v_0_1_0_44, v_0_1_0_45, v_0_1_0_46, v_0_1_0_47, v_0_1_0_48, v_0_1_0_49, v_0_1_0_50, v_0_1_0_51, v_0_1_0_52, v_0_1_0_53, v_0_1_0_54, v_0_1_0_55, v_0_1_0_56, v_0_1_0_57, v_0_1_0_58, v_0_1_0_59, v_0_1_0_60, v_0_1_0_61, v_0_1_0_62, v_0_1_0_63, v_0_1_0_64, v_0_1_0_65, v_0_1_0_66, v_0_1_0_67, v_0_1_0_68, v_0_1_0_69, v_0_1_0_70, v_0_1_0_71, v_0_1_0_72, v_0_1_0_73, v_0_1_0_74, v_0_1_0_75],
|
||||
})
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
import { VersionInfo } from '@start9labs/start-sdk'
|
||||
|
||||
// Digest send path via Gmail domain-wide delegation. Code-only, no schema change
|
||||
// (migrations are no-ops):
|
||||
// * The DWD grant on this deployment already includes the gmail.compose scope
|
||||
// (verified 2026-06-15: token mint + a live messages.send both succeed), and
|
||||
// gmail.compose authorizes users.messages.send. So CRM-originated mail can go
|
||||
// through the existing service account — no SMTP account / app password.
|
||||
// * backend/email_integration/gmail_send.py: send_via_gmail() impersonates a
|
||||
// domain user and POSTs users.messages.send (mirrors compose.py's REST path).
|
||||
// * backend/digest_mailer.py: send_digest() prefers Gmail DWD when enabled and
|
||||
// falls back to smtp_send when not. The admin POST /api/admin/digest/test-email
|
||||
// and the Settings "Send Test Digest Email" button route through it.
|
||||
// * Sender for the DWD path is CRM_DIGEST_SENDER, else the first active admin.
|
||||
export const v_0_1_0_76 = VersionInfo.of({
|
||||
version: '0.1.0:76',
|
||||
releaseNotes: {
|
||||
en_US: [
|
||||
'The daily-digest mailer can now send through the Gmail account the CRM already uses for email',
|
||||
'capture (domain-wide delegation) — no separate SMTP account or app password needed. SMTP stays',
|
||||
'available as a fallback.',
|
||||
].join(' '),
|
||||
},
|
||||
migrations: { up: async () => {}, down: async () => {} },
|
||||
})
|
||||
Reference in New Issue
Block a user