The fundraising grid (canonical) now drives the classic opportunities
Pipeline board, instead of the board being a disconnected second data-entry
surface. An "Add to Pipeline" row action creates a durably-linked opportunity
via the new opportunities.fundraising_investor_id (migration 0005, additive +
reversible), reusing the grid's already-synced contact — retiring the
POST /api/contacts side-door — and mapping the grid lead to the opp owner.
Ownership is split so the two stay reconciled: the grid owns whether the link
exists and the seed; the board owns stage/probability/owner. The link endpoint
is idempotent (one live opp per investor; a re-link never reseeds funnel
fields). "Is in pipeline?"/"what stage?" are derived from a live opp join and
injected as read-only grid columns on read, stripped on write, so they never
persist or dirty the autosave. Remove-from-pipeline soft-deletes the opp and
leaves the grid row fully intact; deleting an investor from the grid archives
its orphaned opp.
Also fixes the standing soft-delete leak in handle_pipeline_report and the
dashboard pipeline aggregates, which counted tombstoned opportunities.
Tests: backend/test_grid_pipeline_link.py (link/idempotent/round-trip/guards/
unlink-intact/re-link/orphan/aggregates); 28/28 suite green, render-smoke green.
Local-smoke found "jonathan is chatting with wyoming" extracted the teammate, not
the prospect. Feed the parser an optional team roster (INTAKE_TEAM_ROSTER) via a
build_system(roster) outreach frame: roster names/initials are the people doing
outreach and are never extracted; the other party is the investor/prospect. Same
framing on the revise leg. Unset roster = prior behavior.
Bot now runs as a docker-compose service on the Spark (verified live, listening). Docs (matrix-intake guide ops, ROADMAP, AGENTS Current state) updated. Also logs the live-smoke parse bug (teammate read as investor -> team-roster fix), the spark-control dashboard-card handoff, and the long-term dedicated-repo extraction.
Turn the bot from a bare nohup process (silently dies on a Spark reboot) into a docker-compose service. Dockerfile bundles backend/matrix_intake + the stdlib backend/ingest Spark client it reuses; .env is mounted read-only at runtime, never baked. The existing repo-root .dockerignore (shared with the s9pk build) already keeps data/ and .env out of context. Also adds a handoff doc for wiring a spark-control dashboard card in a later session.
Box installed to 0.1.0:86 (migration chain ...85->86 clean, candidates endpoint verified live); bot pulled + restarted on the Spark. Only the Matrix live-smoke remains.
Close the two locked post-deploy enhancements for the Matrix intake bot.
Fuzzy matching (server-side, ships in the s9pk): new find_intake_candidates in
server.py returns ranked deterministic near-matches (difflib name similarity +
token-set Jaccard, legal-suffix-aware, + email Levenshtein <= 2); GET
/api/intake/match now returns {match, candidates}. The bot surfaces a numbered
shortlist so a near-duplicate (Charlie/Charles, Acme Capital vs Acme Capital LLC,
a one-char email typo) is confirmed by a human instead of silently creating a
second investor. Exact match still auto-attaches; fuzzy candidates are never
auto-attached. The optional LLM-judge re-rank is deferred.
Conversational edits (bot-side, ships on the Spark): any in-thread reply that
isn't yes/no/edit field=value is treated as a natural-language revision and
re-run through local Qwen (parse.revise). Email integrity is preserved -- a
changed address must literally appear in the instruction; the model's email
field is structurally unreachable. No-op revisions re-prompt.
Docs/current-state brought current; 27/27 backend tests green.
The grid note line was "YYYY-MM-DD [type] Contact: summary"; for the default
"note" type the tag is noise. Omit it for "note"; keep it for informative
types (call, meeting, …). Shared by the Matrix intake bot and grid-UI logging.
Built + installed to the box (installed-version 0.1.0:85, clean 84->85
migration). No schema change.
Four bot-side UX fixes surfaced by the live smoke:
- Post a brief pointer in the main timeline (a reply to the user's message)
alongside the in-thread proposal card, so proposals aren't missed inside a
thread. Pointer only — approvals still happen in the thread, where the note
is visible (you can't make an informed yes/no without seeing it).
- A bare yes/no typed in the main timeline while a proposal is pending now
gets a "reply in the thread" redirect instead of "couldn't tell what to record."
- Clearer commit confirmations: "Created a new grid entry for X" vs
"Logged a note on X (existing grid entry)."
- Send a blank communication subject when a note is present so the grid's
one-line note summary shows the note text, not the "(Matrix)" label
(provenance stays in source="matrix_intake").
The intake bot's server-side dependencies — GET /api/intake/match (new-vs-
existing lookup) and `source` provenance on log-communication — shipped in
source with 7ad0ee7 but were never packaged. The box ran v83 (pre-7ad0ee7),
so the bot's match calls 404'd: a note on an existing investor would have
created a duplicate, and writes weren't tagged matrix_intake. Bump + build +
install verified live (installed-version 0.1.0:84, clean 83->84 migration,
match endpoint now resolves by name and email). No schema change.
Also log the conversational (LLM-mediated) edit enhancement in ROADMAP.
normalize()'s email regex matched non-@/non-space runs, so "Name <addr>"
(the most common contact format) yielded "<addr"; only trailing punctuation
was stripped, never leading. Tighten the regex to standard local@domain.tld
so the bare address is extracted from <…>, (…), and trailing-period forms.
Found via the live-deploy pre-flight. Add a regression test.
Also log two intake backlog items in ROADMAP: the scoped service-credential
auth path (deferred; bot uses a member login for now) and fuzzy match +
in-thread confirm (post-deploy).
New backend/matrix_intake/ runs as its own process (matrix-nio isolated from the
stdlib CRM): local-Qwen parse via Spark Control → in-thread human approval
(yes/edit/no) → write through the CRM's own log-communication endpoint, tagged
source=matrix_intake. Adds read-only GET /api/intake/match (returns grid row id,
no-duplicate contract); threads provenance through handle_log_fundraising_communication.
Reviewer-passed: pop-before-commit closes a double-approve race; edit-grammar fix.
Text-only v1; business-card photo (M3) deferred (no Spark vision model).
26/26 tests green; live Matrix smoke pending deploy.
Fix AGENTS.md access wording (ClearNet/StartTunnel + app-level auth, not
LAN/Tailscale). Add the StartOS icon bug to Known debt and the email-sync-status
error as Next #8. Add ROADMAP sections for Matrix-bridge grid intake (next,
high priority) and an admin-vs-all-users UI audit.
Communications tab (search/query roadmap items 1 & 2):
- Fix the investor dropdown: the facet only listed grid investors, so it
came back empty whenever email matched a classic contact or org domain
(no grid id — the common case). It now mirrors the email list, resolving
each link to a typed identity (fund:/org:/contact:/addr:) with precedence
grid -> org -> contact -> address; investor_id accepts the typed key
(bare id = fund: for back-compat) and an unknown prefix matches nothing.
- Add a date-range filter and a click-to-expand full-body view
(GET /api/email/detail, admin, soft-delete-gated; body_text only, never
raw remote HTML).
- Add a "Search content" mode: GET /api/email/search wraps the ingest
hybrid_search over the Qdrant email index (doc_type=email), hydrated and
soft-delete-filtered against SQLite (canonical), 503 if Spark/Qdrant down.
Daily digest:
- Settings -> Admin builds a digest over a chosen window (last 24h or since
a date) as an in-app preview before sending (POST /api/admin/digest/preview),
so the local-Spark summarizer can be verified on demand even on a quiet day.
Manual send uses the same window; neither advances the daily cursor, so a
preview never suppresses the scheduled digest.
Code-only, migrations no-op. 22/22 backend tests, render-smoke pass.
Record the v82 vendor+SRI + render-smoke work in durable docs: packaging guide
gains the verified-build gate + re-vendor instructions; Current state rewritten
and compressed for v82; ROADMAP logs the deferred pre-compile-JSX alternative.
React/ReactDOM/Babel were loaded from the unpkg CDN at runtime — react@18
and react-dom@18 weren't even exact-pinned, and none had SRI. A CDN swap (or
react auto-resolving a new 18.x) could blank the whole app with no change on
our side: exactly the v78/v79 blank-screen class. It also made the self-hosted
box depend on outbound internet to render.
Vendor the three libs into frontend/assets/vendor/ (React 18.3.1, ReactDOM
18.3.1, @babel/standalone 7.29.7) and load them same-origin with sha384
integrity attributes. They now ship inside the s9pk (Dockerfile already COPYs
frontend/; server.py serves /assets/* with the path-containment check), so a
CDN can never swap prod deps again and no outbound fetch is needed at runtime.
Add start9/0.4/render-smoke.mjs: a jsdom render smoke check that (1) runs the
shipped Babel over the app's inline JSX and asserts a classic, non-module,
parseable script (the v79 ESM-import regression), and (2) mounts the app in
jsdom and asserts the login UI renders (the v78 blank-screen class). Wired into
the default `make` goal so every package build is gated on the frontend
actually rendering — closing the "verified live via curl only" gap. jsdom is a
build-time devDependency, not shipped in the image.
The email-activity panel surfaced every captured message, including cold/
unknown-sender email with no investor association. Gate query_email_activity
on EXISTS(email_investor_links) so the panel shows only email tied to a known
investor/contact. Capture is unchanged — unmatched email is still stored
(metadata-only) and will appear automatically if its sender is later added as
an investor; this is a read-side filter only.
Graveyard investors are unaffected (their email has a link), so they remain
visible/searchable as an audit surface, hidden only from the filter picker.
The Communications tab is now an admin-only search over captured Gmail
(email_* tables), part of consolidating on the fundraising grid + email
capture as the canonical system of record.
- New GET /api/email/activity (admin-enforced server-side): filter by
investor / mailbox / direction with free-text search over subject,
snippet, and sender. Query logic in db.query_email_activity.
- Soft-delete honored on the per-mailbox sighting (emails carry no
deleted_at; deletion lives on email_account_messages).
- Direction decided at the email level (outbound if the sender is one of
our mailboxes), mirroring digest_builder.
- Graveyard investors are hidden from the filter dropdown (CRM-wide
graveyard=0 convention) but their email stays visible in the list and
findable by free-text search — this is an audit surface.
- Communications page rewritten to render the panel; the classic manual
"Log Communication" form is retired (the grid context menu remains the
manual-log path). Nav item + page are admin-only.
- Tests: email_integration/test_email_activity_panel.py (filters,
per-sighting soft-delete, roll-ups, graveyard handling, route 401/403);
full suite 22/22. Frontend render verified via a jsdom mount smoke test
plus the pinned classic-runtime Babel transform.
Code-only, no schema migration (version migrations are no-ops).
Record the Babel-pin fix + root cause, the 3 newly admin-gated GET endpoints, the corrected deploy-verification convention (browser render, not curl/health), and the re-ordered Next list (vendor+SRI, auth regression test, email-activity panel in the admin-only Communications tab).
The web UI rendered a blank screen for every user. Root cause: the page
loaded @babel/standalone from unpkg with no version pin, so the CDN silently
served Babel 8.0.0. Babel 8 defaults @babel/preset-react to the automatic JSX
runtime, which prepends `import {jsx} from "react/jsx-runtime"` to the compiled
output. An ESM import is illegal in this classic (non-module) inline <script>,
so the browser rejected the whole bundle and React never mounted — hence the
blank screen. The prior "verified live" checks were server-up/curl, which can't
catch a browser-render failure.
- Pin @babel/standalone@7.29.7 (its preset-react defaults to the classic
React.createElement runtime). Verified via headless render: app mounts, login
screen renders, no console error. Follow-up: vendor + SRI-pin the CDN libs so
a third party can't swap our front-end deps in production again.
- Close three server-side admin gaps surfaced by a permissions audit — endpoints
that were UI-hidden from members but not API-enforced: GET /api/users,
/api/email/status, /api/email/accounts now require_admin. Removed the now-dead
non-admin mailbox-row filter. 21/21 backend tests green; py_compile clean.
The fundraising grid + email capture is the canonical system of record. lp_profiles
was a superseded single-fund model with no reachable create/edit path, and the LP
Tracker page was already orphaned (no nav entry + a redirect bouncing it to the grid).
- Remove /api/lp-profiles* endpoints + handlers, the unused lp-breakdown report,
the contact-dossier LP section, the demo-seed LP block, and (frontend) the
LPTrackerPage component + its lp-tracker->fundraising-grid redirect.
- Dashboard "Total Committed" now sums fundraising_investors.total_invested
(graveyarded investors excluded) instead of the orphaned lp_profiles table, which
read ~$0. "Total Funded" dropped: the grid tracks commitments, not a funded amount,
and the frontend never rendered it.
- Leave the empty lp_profiles table/index, the contact-delete soft-delete cascade,
and the --reset-all-data clear in place (never-hard-delete).
- Tests: add test_dashboard_report.py; update test_soft_delete_reads.py. 21/21 green.
Docs-only: packaging guide notes start-cli install is silent on success (verify
with installed-version/logs); AGENTS.md adds the operational-toggles-in-the-admin-
panel convention and tightens the digest Current state.
installed-version on the box -> 0.1.0:77; migration chain ran through 76->77;
server up on :8080 with the digest scheduler running (policy-controlled, auto-send
off by default). Docs-only.
Sends a once-a-day internal email to all active admins summarizing each team
member's email activity per investor, plus a team-wide by-investor view
(inbound + outbound, deduped). Narratives are generated on the LOCAL Spark
model, never Claude — the digest is intentionally un-anonymized, so substance
stays on Ten31 infra. This is an internal ops email, exempt from the
'agents draft, humans send' rule (which governs outward LP contact).
- backend/digest_builder.py: per-user + per-investor activity queries
(soft-delete filtered), per-user Spark narrative with a deterministic
fallback, two-section plain-text body, and the DB-backed policy resolver.
- backend/email_integration/digest_scheduler.py: always-on daily thread that
re-reads the policy each cycle and sends once/day; window cursor in
app_settings so a missed day rolls forward.
- server.py: POST /api/admin/digest/send-now and GET/PATCH
/api/admin/digest/policy; scheduler wired into main().
- Control lives in Settings -> Admin (enable toggle + send-time dropdown),
not StartOS actions; env vars only seed the first-boot default.
- Tests: backend/test_digest_builder.py.
Extend docs/guides/email.md paths: frontmatter (and its AGENTS.md index entry) to
include backend/digest_mailer.py and backend/smtp_send.py, so the guide auto-loads
when editing the outbound-digest send path — not just backend/email_integration/**.
Portability-checker: compliant.
- docs/guides/email.md: new "Outbound mail — the daily digest" section (Gmail-DWD
primary → SMTP fallback; gmail.compose send capability; the internal-digest
exemption from the agents-draft rule).
- AGENTS.md: add digest env names (CRM_DIGEST_SENDER, SMTP_*); consolidate the
v75/v76 deploy bullets into one current bullet; drop finished v74 narrative.
Non-blocking items from the v76 reviewer pass. No redeploy needed — the box runs
v76 and its happy path is unaffected; these ride the next build:
- digest_mailer.send_digest: when Gmail is enabled but no sender resolves
(CRM_DIGEST_SENDER unset and no admin email), raise NoTransport so the caller
returns a clear 400 instead of a generic 502.
- gmail_send.send_via_gmail: wrap OSError/URLError (timeout/DNS) as a RuntimeError
("Gmail API unreachable: ...") to match the HTTPError handling; include the
sender in the HTTPError message for debuggability.
- credentials.py: correct the now-stale GMAIL_COMPOSE_SCOPE comment (the digest
mailer sends with this scope; only outreach drafts never send).
- test_gmail_send.py: add the HTTPError->RuntimeError branch, default_sender DB
fallback (+None case + env override), and the send_digest SMTP-tag path.
19/19 backend tests green.
The box's existing service-account domain-wide-delegation grant already includes
gmail.compose, which authorizes users.messages.send — verified 2026-06-15 by a
token-mint probe and a live messages.send to grant. So CRM-originated mail can
send through the account that already powers email capture: no SMTP account, no
app password, no admin change.
- backend/email_integration/gmail_send.py: send_via_gmail() impersonates a
domain user and POSTs users.messages.send (reuses credentials.py + the compose
scope; mirrors compose.py's REST pattern).
- backend/digest_mailer.py: send_digest() prefers Gmail DWD when enabled, falls
back to smtp_send otherwise. Sender = CRM_DIGEST_SENDER else first active admin.
- server.py: the admin test endpoint now routes through digest_mailer (so the
Settings button sends via DWD on the box with zero SMTP config). Recipient
restriction to the admin set and no-leak error handling preserved.
- test_gmail_send.py: build/send + transport routing (provider + urlopen faked).
19/19 backend green; s9pk typechecks.
SMTP (v75) stays as the fallback transport. Send-path decision + scope finding
recorded in ROADMAP.md and AGENTS.md.
Surface the digest test-send endpoint as a clickable admin control so it can be
exercised on the box without curl. Calls POST /api/admin/digest/test-email and
toasts the result (or a 'configure SMTP first' hint). JSX parse-checked.
Groundwork for the daily activity digest: give the CRM an outbound mail path.
Today nothing leaves the box (Gmail capture + drafts only), so this adds a
dedicated, per-package SMTP account independent of any StartOS system-wide SMTP.
- configureDigestSmtp Start9 action: writes host/port/from/username/password/
security to /data/secrets/smtp/* (password piped over stdin, never argv/env;
per-field files, owner-only) — mirrors the setAnthropicApiKey pattern.
- docker_entrypoint.sh reads those at boot and exports SMTP_* (operator env wins).
- backend/smtp_send.py: stdlib smtplib wrapper reading SMTP_* (one code path for
dev .env and the box); starttls/tls/none modes.
- POST /api/admin/digest/test-email (admin-only): proves the pipe. Recipients are
restricted to the active-admin set — an arbitrary `to` is rejected, so the
endpoint is not an open relay; send failures are logged, not echoed (an SMTP
auth error can carry the credential).
- Tests: test_smtp_send.py (sender), test_smtp_endpoint.py (gating + relay
restriction + no-leak). 18/18 backend green; s9pk typechecks.
Analysis/summarization for the digest body (Phase B) will run on Spark, never
Claude — the digest is deliberately un-anonymized. Decisions + Phase B plan in
ROADMAP.md.
Verification against published tarballs confirms ^0.4.0-beta.66 resolves to
0.4.0-beta.66 and its SMTP API surface (getSystemSmtp/SmtpValue,
inputSpecConstants.smtpInputSpec, smtpShape, smtpPrefill) is byte-identical to
the 1.5.3 reference packages. Build against beta.66 as-is; no SDK bump needed.
Research confirms StartOS 0.4 supports per-package SMTP credentials fully
independent of the server's system account: the "custom" branch of the
manage-smtp action (gitea-startos and vaultwarden-startos both) never calls
getSystemSmtp and works on a box with no system SMTP. Record this as the likely
fit (a digest-only mailbox), built on sdk.inputSpecConstants.smtpInputSpec, plus
the SDK version caveat (our ^0.4.0-beta.66 pin vs the references' 1.x).
Fold the research into the daily-digest item: outbound mail uses the StartOS
0.4 system SMTP account via sdk.getSystemSmtp(effects).const(), delivered to the
Python process as env vars from the daemon exec block (gitea-startos pattern,
mirroring the existing setAnthropicApiKey action). Notes the action+storeJson
config model that replaced the 0.3 manifest Config/Properties spec.
Records the requested per-user daily digest (who emailed which investors +
email substance) as a post-Phase-1 backlog item. Notes the three build-shaping
constraints: the new outbound-SMTP dependency (no current send path), the
Spark-only / never-Claude analysis rule for the un-anonymized substance, and
the internal-digest exemption from the "agents draft, humans send" guardrail.
thesis-seed v1–v4 are superseded by v5 (the version seeded by
thesis_seed.py) and had no inbound references. refresh_seed.sh and
seed/README.md are 0.3.5-era seed-snapshot helpers the 0.4 entrypoint
no longer uses (DEPLOY_040 labels both LEGACY). data/test_write was a
stray 0-byte write-probe. Folder-rename housekeeping; no runtime change.
Cross-repo git-hygiene audit remediation: surface ~/Projects/standards/INBOX.md items at session start, and switch .gitignore to the deny-by-default .claude/* block (shared wiring allow-listed) plus the canonical secrets/env lines — per standards/portability.md.
Lock in the three v0.1.0:74 security/privacy fixes with regression tests, and
fix a same-class soft-delete leak surfaced while writing them.
- backend/test_assets_traversal.py: boots the real server, proves /assets/
path-traversal vectors (incl. a real decoy file and the live crm.db, plain
and URL-encoded) 404 and leak nothing, while a legit asset still serves 200.
- backend/test_soft_delete_reads.py: get-by-id 404s soft-deleted rows and
nested + list-view aggregates exclude soft-deleted children.
- backend/mcp/test_outreach_redaction.py: an unknown free-prose name is
tokenized away from the Claude payload but re-hydrated locally, and the path
fails closed (no Claude call) when the local NER model is down.
- backend/run_tests.py: aggregate runner (each backend/**/test_*.py in its own
subprocess); replaces the manual for-loop. 16/16 green.
A reviewer pass on the tests confirmed the soft-delete filter was missing from
list-view aggregate sub-selects: org contact_count/total_funded and contacts
comm_count/last_contact_date counted soft-deleted rows. Add `deleted_at IS NULL`
to those four (server.py) and regression-cover them.
The reports subsystem (dashboard/pipeline/LP-breakdown, ~16 aggregate queries)
has the same leak and is logged as P2 for a dedicated pass. Not yet built or
deployed — bump the package version before the next s9pk build.
The crm-preview launch config hardcoded an absolute machine path for
CRM_FRONTEND_DIR, breaking the preview on any other clone path. Derive
it from $PWD at launch time so the repo is self-contained. Clears the
sole blocker from the portability-standard audit.
ensure_positioning_framings adds 5 Architect framings to the core
positioning variant group alongside Option A/B, so the group holds 7
candidates and choose_variant retires 6. The two thesis tests still
asserted the pre-framings count of 2 — the tests were stale, not the
seed. Realign them, document the 2+5=7 seed structure in the thesis
guide, and refresh AGENTS.md Current state (13/13 tests green).
Keysat