Phase 0 foundation: canonical schema, ingest pipeline, CRM MCP server
Workstream A–C substrate for the Ten31 agentic system: - A1: docs/crm-overview.md; CLAUDE.md conventions + guardrail #9 - A2: additive/reversible core migration (canonical_entities, entity_links, interaction_log, relationship_edges, soft-delete) + ledgered runner - B1/B3: chunking + deterministic entity resolution (backend/ingest) - B2: dense (bge-m3) + BM25 sparse ingest to Qdrant crm_chunks - C: CRM MCP server (reads, retrieval modes, logged writes) — no outbound tools - docs: redaction/re-hydration, Gmail enablement runbook - synthetic test data; .env.example; housekeeping (.gitignore, untrack crm.db, drop legacy files + start9/0.3.5) Verified end-to-end on synthetic data + live Sparks (hybrid > dense on entity queries). Real backfill runs on Ten31 infra; index holds synthetic data only. Branch snapshot also captures pre-existing working-tree changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,58 @@
|
||||
import { VersionInfo } from '@start9labs/start-sdk'
|
||||
|
||||
// Gmail integration — Phase 1.
|
||||
//
|
||||
// Background: the CRM previously had no ingestion path for email
|
||||
// activity. Contacts were logged manually; correspondence history lived
|
||||
// only in our mailboxes. This release adds a one-way capture pipeline
|
||||
// that ingests sent and received mail for every Workspace user at
|
||||
// ten31.xyz, matches messages against existing investor records, and
|
||||
// records metadata (+ bodies and attachments for matched threads) into
|
||||
// the CRM database.
|
||||
//
|
||||
// Auth model: domain-wide delegation via a Google service account. The
|
||||
// service-account JSON key is stored on the /data volume at
|
||||
// /data/secrets/gmail-service-account.json (chmod 600, operator-dropped).
|
||||
// The integration is self-disabling: if the key file is absent, the
|
||||
// scheduler doesn't start and /api/email/* routes return 503. No key →
|
||||
// no behavior change from 0.1.0:41.
|
||||
//
|
||||
// When the key IS present, docker_entrypoint.sh auto-enables the
|
||||
// integration and sets sensible defaults (3-hour sync interval, domain
|
||||
// ten31.xyz, DWD auth). All defaults can still be overridden via env.
|
||||
//
|
||||
// Database: migration 0001 adds eight new tables under the email_
|
||||
// namespace (emails, email_accounts, email_recipients,
|
||||
// email_account_messages, email_attachments, email_threads,
|
||||
// email_investor_links, email_sync_runs). All CREATE TABLE IF NOT EXISTS,
|
||||
// so the migration is safely idempotent — re-applying is a no-op.
|
||||
//
|
||||
// Backend: wholly isolated under backend/email_integration/. Three tiny,
|
||||
// feature-flag-guarded hooks in server.py (migration call, scheduler
|
||||
// startup, /api/email/* route dispatch). Removing or disabling the
|
||||
// integration leaves server behavior identical to 0.1.0:41.
|
||||
//
|
||||
// New Python dep: cryptography==42.0.5 (required for RS256 JWT signing
|
||||
// in DWD bearer token exchange). Now installed in the image.
|
||||
//
|
||||
// No data migration code needed — new tables, additive only.
|
||||
export const v_0_1_0_42 = VersionInfo.of({
|
||||
version: '0.1.0:42',
|
||||
releaseNotes: {
|
||||
en_US: [
|
||||
'Adds a Gmail capture pipeline. When a Google Workspace',
|
||||
"service-account key is dropped into the server's /data/secrets",
|
||||
'folder, the CRM begins pulling sent and received mail for every',
|
||||
'ten31.xyz user on a 3-hour cycle, matching messages against',
|
||||
'existing investor records and storing metadata (plus bodies and',
|
||||
'attachments for matched threads) in the database. With no key',
|
||||
'present the feature is dormant and this release behaves',
|
||||
'identically to 0.1.0:41. Eight new email_* tables are added',
|
||||
'additively; no existing data is touched.',
|
||||
].join(' '),
|
||||
},
|
||||
migrations: {
|
||||
up: async () => {},
|
||||
down: async () => {},
|
||||
},
|
||||
})
|
||||
Reference in New Issue
Block a user