Phase 0 foundation: canonical schema, ingest pipeline, CRM MCP server

Workstream A–C substrate for the Ten31 agentic system: - A1: docs/crm-overview.md; CLAUDE.md conventions + guardrail #9 - A2: additive/reversible core migration (canonical_entities, entity_links, interaction_log, relationship_edges, soft-delete) + ledgered runner - B1/B3: chunking + deterministic entity resolution (backend/ingest) - B2: dense (bge-m3) + BM25 sparse ingest to Qdrant crm_chunks - C: CRM MCP server (reads, retrieval modes, logged writes) — no outbound tools - docs: redaction/re-hydration, Gmail enablement runbook - synthetic test data; .env.example; housekeeping (.gitignore, untrack crm.db, drop legacy files + start9/0.3.5) Verified end-to-end on synthetic data + live Sparks (hybrid > dense on entity queries). Real backfill runs on Ten31 infra; index holds synthetic data only. Branch snapshot also captures pre-existing working-tree changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 08:11:28 -05:00
parent 7027efd777
commit c7ce44d963
99 changed files with 10676 additions and 7817 deletions
@@ -0,0 +1,61 @@
+#!/bin/sh
+# ═══════════════════════════════════════════════════════════════
+#  Ten31 Database container entrypoint (StartOS 0.4 wrapper)
+# ═══════════════════════════════════════════════════════════════
+#
+#  Responsibilities:
+#    1. Ensure the mounted /data volume directories exist.
+#    2. Ensure a persistent CRM_SECRET_KEY exists so issued JWTs
+#       survive container restarts.
+#    3. Launch the Python backend server.
+#
+#  Note: This entrypoint NO LONGER seeds /data from a baked-in
+#  snapshot. The 0.3.5 → 0.4 migration is complete; from 0.1.0:40
+#  forward the live /data volume on the StartOS host is the sole
+#  source of truth. StartOS preserves /data across sideloads, so
+#  upgrades will not disturb live data.
+# ═══════════════════════════════════════════════════════════════
+
+set -eu
+
+DATA_DIR="${CRM_DATA_DIR:-/data}"
+SECRET_FILE="$DATA_DIR/.crm-secret"
+SECRETS_DIR="$DATA_DIR/secrets"
+EMAIL_ATTACHMENTS_DIR="$DATA_DIR/email_attachments"
+GMAIL_SA_KEY="$SECRETS_DIR/gmail-service-account.json"
+
+mkdir -p "$DATA_DIR" "$DATA_DIR/backups" "$SECRETS_DIR" "$EMAIL_ATTACHMENTS_DIR"
+# /data/secrets holds the Gmail service-account key; lock it down so only
+# the container user can read the directory. chmod on the file itself is
+# the operator's responsibility when they drop the key in.
+chmod 700 "$SECRETS_DIR" 2>/dev/null || true
+
+# ── Persistent JWT secret ───────────────────────────────────────
+if [ -z "${CRM_SECRET_KEY:-}" ]; then
+  if [ -f "$SECRET_FILE" ]; then
+    CRM_SECRET_KEY="$(cat "$SECRET_FILE")"
+  else
+    CRM_SECRET_KEY="$(head -c 48 /dev/urandom | base64 | tr -d '\n' | tr '/+' 'ab')"
+    printf '%s' "$CRM_SECRET_KEY" > "$SECRET_FILE"
+    chmod 600 "$SECRET_FILE"
+  fi
+  export CRM_SECRET_KEY
+fi
+
+# ── Gmail integration env vars ──────────────────────────────────
+# The integration is enabled only if the service-account key file is
+# actually present on the /data volume. This makes the package
+# self-disabling on fresh installs until an operator drops the key in.
+if [ -f "$GMAIL_SA_KEY" ]; then
+  export CRM_GMAIL_INTEGRATION_ENABLED="${CRM_GMAIL_INTEGRATION_ENABLED:-true}"
+  export CRM_GMAIL_AUTH_METHOD="${CRM_GMAIL_AUTH_METHOD:-dwd}"
+  export CRM_GMAIL_SA_KEY_PATH="${CRM_GMAIL_SA_KEY_PATH:-$GMAIL_SA_KEY}"
+  export CRM_GMAIL_WORKSPACE_DOMAIN="${CRM_GMAIL_WORKSPACE_DOMAIN:-ten31.xyz}"
+  export CRM_GMAIL_SYNC_INTERVAL_MIN="${CRM_GMAIL_SYNC_INTERVAL_MIN:-180}"
+  echo "[entrypoint] Gmail integration: ENABLED (key at $GMAIL_SA_KEY)"
+else
+  echo "[entrypoint] Gmail integration: DISABLED (no key at $GMAIL_SA_KEY)"
+fi
+
+# ── Launch the app ──────────────────────────────────────────────
+exec python3 /app/backend/server.py