ten31-database/ROADMAP.md

Venture CRM Roadmap (Airtable Replacement)

Current status

• Premium Airtable-like frontend grid exists and is actively iterating.
• Backend now has production-grade APIs for:
  • GET /api/fundraising/state
  • PUT /api/fundraising/state (with optimistic version check)
  • GET /api/fundraising/export
  • POST /api/fundraising/backup
  • POST /api/fundraising/restore-preview
  • POST /api/fundraising/restore
  • GET /api/fundraising/backups
  • GET/PATCH /api/fundraising/backup-policy
  • GET /api/fundraising/relational-summary
  • GET /api/feature-requests
  • POST /api/feature-requests
  • PATCH /api/feature-requests/:id
• New DB tables:
  • fundraising_state
  • fundraising_investors
  • fundraising_contacts
  • fundraising_funds
  • fundraising_commitments
  • fundraising_views
  • feature_requests
  • app_settings
• Grid saves/restores now sync into relational fundraising tables automatically.
• Formula engine is now sandboxed (no eval/new Function) with expanded function support.
• Automation engine v1 added:
  • Rule table + toggle API
  • List memberships (main, follow_up, graveyard, longshot, all)
  • Automation run log
• Collaboration/reliability additions:
  • Unified activity feed API (audit + automation + backup)
  • Backup integrity verification API
  • Better version-conflict metadata (updated_at, updated_by)
• Security hardening additions:
  • Basic IP rate limiting (login and write APIs)
  • Configurable CORS origin (CRM_CORS_ORIGIN)
  • Production secret enforcement (CRM_ENV=production requires CRM_SECRET_KEY)
  • Security status API + go-live checklist (SECURITY.md)

Phase 1 (Production foundation)

1. Persist grid + views on backend

• Wire frontend fundraising grid reads/writes to /api/fundraising/state.
• Keep localStorage only as emergency fallback.
• Add autosave debounce and conflict handling (expected_version).

2. Admin-invite auth model

• Disable self-register for non-admin users.
• Add admin-only invite/create-user endpoint.
• Keep role model: admin, member.

3. Deployment and remote access

• Add docker-compose for one-command launch.
• Reverse proxy + TLS option (Caddy/Traefik) for non-Tailscale deployments.
• Recommended for your use case: Tailscale private access to laptop host.

4. Data safety and operations

• Automated nightly SQLite backups and restore test script.
• Add /api/fundraising/export for JSON snapshot export.
• Add health/readiness checks.

Phase 2 (Airtable parity)

1. Advanced views

• Multi-condition filter groups (AND/OR groups)
• Multi-column sorting
• Pinned/frozen columns
• Personal vs shared views

2. Formula engine v2

• Add functions: SUM, MIN, MAX, ROUND, ABS, CONCAT (done)
• Type-aware formulas and better errors
• Dependency graph and recalculation rules

3. Activity + audit

• Record-level change history in UI
• Last modified by / at fields
• Restore archived rows

Phase 3 (Team workflow and automation)

1. Tasks/reminders tied to investors/contacts
2. Automation rules (graveyard/follow-up triggers)
3. Email/communication integrations (optional)
4. Granular permissions (if team grows)

Backlog (post-Phase-1 agentic)

Daily activity digest (email to the team)

Requested 2026-06-15. Phase A deployed (v0.1.0:76, live on the box). Phase B built 2026-06-15 (digest content + Spark summarization + daily scheduler + on-demand send) — in the repo, awaiting the next build/deploy.

Decisions (locked 2026-06-15): recipients = all active admins; summarization = Spark-LLM narrative (never Claude — un-anonymized substance stays local); granularity = grouped by user (→ per investor).

Send transport — DECIDED 2026-06-15: Gmail domain-wide delegation (not SMTP). The box's existing service-account grant (which powers email capture) includes gmail.compose, which authorizes users.messages.send — verified by a token-mint probe and a live messages.send to grant. So the digest sends through the account the CRM already uses: no app password, no new account, no admin change. The narrow gmail.send scope is not granted, so the sender must request gmail.compose.

Phase A — DONE: (v0.1.0:75) configureDigestSmtp Start9 action + docker_entrypoint.sh SMTP_* export + backend/smtp_send.py + admin POST /api/admin/digest/test-email (recipient-restricted to the admin set — not an open relay) + Settings button. (v0.1.0:76, redeploy pending) backend/email_integration/gmail_send.py (users.messages.send via DWD/compose) + backend/digest_mailer.py (Gmail-DWD preferred, SMTP fallback); the endpoint + button route through it; sender = CRM_DIGEST_SENDER else first active admin. Tests: test_smtp_send.py, test_smtp_endpoint.py, test_gmail_send.py.

Phase B — DONE (2026-06-15/16): backend/digest_builder.py builds two sections — by team member (per-user Spark narrative + both directions, with a deterministic fallback) and by investor (team-wide, inbound + outbound, deduped per email, structured). Soft-delete filtered throughout. backend/email_integration/digest_scheduler.py is an always-on daily thread that re-reads a DB-backed policy each cycle and sends once/day at the configured hour to all active admins (window cursor in app_settings). Control moved out of env into the admin panel: app_settings.digest_policy + GET/PATCH /api/admin/digest/policy + a Settings → Admin enable toggle + send-time dropdown (env vars only seed the first-boot default). Plus admin POST /api/admin/digest/send-now + a "Send Digest Now" button. Decisions settled: 6 PM default, always-send (empty-day note), per-user narrative + by-investor section, in-app control (not StartOS). Tests: backend/test_digest_builder.py. Detail: docs/guides/email.md.

Have the CRM send a daily digest email summarizing each registered user's activity — primarily who emailed which investors and the substance of those emails — to the fund principal (and eventually other admins). Scales with the synced-user count: 2 users synced today, ~5 eventually.

• Source data: the captured email-activity already flowing through the Gmail DWD propose→approve pipeline (backend/email_integration/), keyed per registered user → per investor/contact. Optionally fold in other CRM activity (audit feed, automation runs, new opportunities) later.
• Send path is NEW capability. Today nothing leaves the box — the system only captures Gmail and creates drafts. This needs outbound SMTP. StartOS 0.4 has a system-wide SMTP account (since v0.4.0-beta.9): the user configures it once for the whole server and services read it via sdk.getSystemSmtp(effects).const(), which returns a T.SmtpValue (host, port, from, username, password, security). Wire the digest sender to that rather than hardcoding any account. Implementation path (researched 2026-06-15, our SDK pin ^0.4.0-beta.66): model a manageSMTP action on gitea-startos / vaultwarden-startos — a three-way selection (system / custom / disabled) built on sdk.inputSpecConstants.smtpInputSpec, persisted to storeJson, with main.ts injecting SMTP_HOST/PORT/USER/PASS/FROM/SECURITY env vars into the daemon exec block (same shape as the existing setAnthropicApiKey.ts action). The Python sender reads them via os.environ and opens smtplib.SMTP/SMTP_SSL. "Custom SMTP" is a dedicated per-package account, fully independent of the server's system SMTP — the custom branch never calls getSystemSmtp, so the digest can send through its own provider even on a box with no system account configured (confirmed in both reference packages). This is the likely fit here: a digest-only mailbox separate from anyone's Gmail. Note StartOS 0.4 dropped the old Config/Properties manifest spec — SMTP config is an action + storeJson, not a manifest config field. SDK note (verified 2026-06-15): our pin ^0.4.0-beta.66 resolves to exactly 0.4.0-beta.66 (caret on a prerelease stays within the 0.4.0 tuple), whose SMTP surface — getSystemSmtp → T.SmtpValue {host, port, from, username, password, security}, inputSpecConstants.smtpInputSpec (providers gmail/ses/sendgrid/mailgun/protonmail/other; selection disabled/system/custom), smtpShape, smtpPrefill — is byte-identical to the 1.5.3 reference packages (verified from published tarballs; repo node_modules is absent). Build against beta.66 as-is — no SDK bump needed (moving to 1.x is a major-track change with broad blast radius across startos/, and nothing about SMTP justifies it).
• Analysis runs on Spark, never Claude. The digest is deliberately un-anonymized (real LP names + email substance), so any summarization/analysis must go through Spark Control to local models — this is the one path that intentionally bypasses the scrub→Claude→re-hydrate boundary, because keeping the substance local is the whole point. Never route digest content to Claude.
• Exempt from "agents draft, humans send." That rule governs outward LP/prospect contact. This is an internal ops digest to the team's own inboxes — a different category — so an automated daily send here does not violate the draft-only guardrail. State this explicitly at build time.
• Scheduling: a daily cron, naturally co-located with the existing backend/email_integration/scheduler.py sync cadence.
• Soft-delete: every aggregate/read in the digest must filter deleted_at IS NULL (see the standing soft-delete rule).

Open design questions (settled at build time): send time = 6 PM box-local (configurable in the admin panel), covering the ~24h window up to send; empty days = always send with a "no activity" note; summary granularity = one per-user narrative plus a by-investor structured section (inbound + outbound, team-wide) added 2026-06-16; enable/time live in the admin panel (DB-backed), not StartOS actions.

Definition of done for "Airtable substitute" v1

• Team can manage all investors in one master table
• Saved views replicate current Airtable workflows
• CSV import from Airtable is reliable and repeatable
• Data persists safely and supports multi-user access
• Auth is invite-only and backups are automated