Files
ten31-database/docs/guides/migrations.md
T
Keysat 1564c087bf Remove Instructions/Feedback + lp_profiles; sync retry, purge, mobile fixes (v0.1.0:104)
Removals (net -570 lines):
- Delete the Instructions and Feedback (feature_requests) pages + backend.
- Retire lp_profiles + investor_type across server, ingest, and seeds; migration
  0008 drops both empty tables (a sanctioned one-off exception to
  never-hard-delete). 0001's lp_profiles ALTER is removed so a fresh DB doesn't
  break the migration chain (live DBs already applied it).

Fixes:
- Email sync: a transient timeout no longer terminally parks a mailbox; the
  scheduler retries 'retrying' each cycle and re-includes errored accounts on an
  hourly backoff, so stuck mailboxes self-heal.
- Mobile Contacts: page through the full directory (server caps 500/page) -- one
  fetch silently truncated at 720, hiding people from the list and from search.
- Mobile email review: clock icon to set a reminder inline; approval cards show
  date/time.

New:
- Admin-only purge of soft-deleted rows (Settings -> Admin; type-to-confirm,
  refuses any row still linked to live data).

Tests: 45/45 (adds test_sync_ready + test_purge_soft_deleted). Reviewer pass
applied (NULL reminders.contact_id on contact purge). Bumped to v0.1.0:104.
2026-06-20 20:06:11 -05:00

2.6 KiB

paths
paths
backend/migrations/**
backend/core_migrations.py
backend/thesis_seed.py

Migrations & seeders

Read this before adding or editing a schema migration or a one-time seed/backfill.

How they run

  • Migrations apply automatically at startup via backend/core_migrations.py, reading backend/migrations/NNNN_*.sql in order and tracking applied files in a schema_migrations ledger.
  • One-time seeds/backfills live in backend/thesis_seed.py (the ensure_* functions), wired into server.init_db() and run on every boot.

Rules

  • Additive + reversible only. Numbered NNNN_*.sql with a paired NNNN_*.down.sql — ship the .down.sql with every new migration. SQLite ALTER is add-column / rename only; no drop-column, no type change.
  • Seeds/backfills must be idempotent via interaction_log sentinels (the ensure_* pattern) — safe to re-run on every boot.
  • Make migrations/seeders deployment-state-invariant. Target rows structurally, not by transient text the same change mutates; capture prior state so a revert is exact.
    • Learned the hard way: matching old nodes by a body string the same changeset deleted broke fresh DBs. A migration must produce the same end state whether the box is empty, mid-version, or fully seeded.
  • Soft-delete onlydeleted_at and/or status='retired'; never hard-delete CRM records or thesis history.
  • Dropping a table is forbidden by default — it needs explicit sign-off (never-hard-delete). 0008_drop_retired_tables (lp_profiles + feature_requests, v0.1.0:104) is the one sanctioned exception, for empty retired tables only. To actually drop one: (1) remove its CREATE TABLE from init_db()init_db() runs every boot, so leaving it there re-creates the table right after the drop migration runs; (2) add a DROP TABLE IF EXISTS forward migration + a .down.sql recreating the empty shell; (3) remove any ALTER TABLE <dropped_table> line from an earlier historical migration — once init_db() stops creating the table, that ALTER fails no such table on a fresh DB and aborts the whole chain (it was the actual bug here). Editing that old migration is safe and deployment-state-invariant: live DBs already applied it before the drop, so the edit only affects fresh DBs, which converge to the same end state. DROP TABLE IF EXISTS is a no-op on a fresh DB and removes the table on the live box.

Verify before shipping

  • python3 -m py_compile the edited Python.
  • For any DB logic, run the change against a copy of data/crm.db, never production. Confirm the paired .down.sql cleanly reverts.