Remove Instructions/Feedback + lp_profiles; sync retry, purge, mobile fixes (v0.1.0:104)
Removals (net -570 lines): - Delete the Instructions and Feedback (feature_requests) pages + backend. - Retire lp_profiles + investor_type across server, ingest, and seeds; migration 0008 drops both empty tables (a sanctioned one-off exception to never-hard-delete). 0001's lp_profiles ALTER is removed so a fresh DB doesn't break the migration chain (live DBs already applied it). Fixes: - Email sync: a transient timeout no longer terminally parks a mailbox; the scheduler retries 'retrying' each cycle and re-includes errored accounts on an hourly backoff, so stuck mailboxes self-heal. - Mobile Contacts: page through the full directory (server caps 500/page) -- one fetch silently truncated at 720, hiding people from the list and from search. - Mobile email review: clock icon to set a reminder inline; approval cards show date/time. New: - Admin-only purge of soft-deleted rows (Settings -> Admin; type-to-confirm, refuses any row still linked to live data). Tests: 45/45 (adds test_sync_ready + test_purge_soft_deleted). Reviewer pass applied (NULL reminders.contact_id on contact purge). Bumped to v0.1.0:104.
This commit is contained in:
@@ -8,7 +8,7 @@
|
||||
|
||||
- **One Python file, no framework.** The whole backend is `backend/server.py` (~4,530 lines): a stdlib `http.server.ThreadingHTTPServer` with a hand-written `CRMHandler(BaseHTTPRequestHandler)` and manual path dispatch. `requirements.txt` lists FastAPI/SQLAlchemy/Alembic/Pydantic but **none are imported** — they are vestigial.
|
||||
- **Storage is one SQLite file** (`data/crm.db`), WAL mode, opened fresh per request. Schema is created idempotently in-code at boot. There is no Alembic; "migrations" are `CREATE TABLE IF NOT EXISTS` + best-effort `ALTER TABLE ADD COLUMN`.
|
||||
- **Two parallel investor data models** coexist with no shared key: (1) the *classic* `contacts / organizations / opportunities / communications / lp_profiles` CRM, and (2) the *newer, actively-used* `fundraising_*` collaborative grid. They are bridged only by fuzzy name/email matching. **This duality is the central entity-resolution problem for Phase 0.**
|
||||
- **Two parallel investor data models** coexist with no shared key: (1) the *classic* `contacts / organizations / opportunities / communications` CRM, and (2) the *newer, actively-used* `fundraising_*` collaborative grid. They are bridged only by fuzzy name/email matching. **This duality is the central entity-resolution problem for Phase 0.** *(Note: `lp_profiles`, formerly part of the classic model, was dropped in v0.1.0:104 — migration `0008`; the grid is now canonical for commitments. The `lp_profiles` mentions later in this doc are retained as historical context for the entity-resolution discussion.)*
|
||||
- **A real Gmail subsystem** (`backend/email_integration/`) stores threaded correspondence in `crm.db` and matches emails to investors — but is **self-disabling** (off unless a service-account key is present).
|
||||
- **Auth is a single scheme:** username/password → HS256 JWT (Bearer header), re-validated against the `users` table each request; two roles (`admin`/`member`). The `X_API_KEY` named in `CLAUDE.md`/`PHASE_0.md` **does not exist in the code** — it is aspirational.
|
||||
- **Guardrail flags:** all deletes are **hard deletes** (violates guardrail #3 as written); a destructive `POST /api/admin/reset-all-data` exists; `audit_log` is mutation-only and is *not* the append-only interaction log Phase 0 wants.
|
||||
@@ -73,7 +73,7 @@ PKs are **8-char truncated UUIDs** (`generate_id()` = `str(uuid.uuid4())[:8]`, `
|
||||
| `organizations` | weak parent of contacts/opps | `name` (not unique), `type` (free-text, default `other`), `tags` JSON, `description`. (`backend/server.py:104`) |
|
||||
| `contacts` | **the hub** | `first_name`/`last_name` (req), `organization_id` (FK SET NULL), `contact_type` (free-text; load-bearing values `prospect`/`investor`), `status` (default `active`), `source`, `tags` JSON, `notes`, `linkedin_url`. (`backend/server.py:123`) |
|
||||
| `opportunities` | deal pipeline | `contact_id` (req, FK **CASCADE**), `stage` (allowlist `PIPELINE_STAGES` at `backend/server.py:1380`, enforced **only** on the stage endpoint), `commitment_amount`, `expected_amount`, `fund_name`, `owner_id`, `lost_reason`. (`backend/server.py:148`) |
|
||||
| `lp_profiles` | closed-LP extension | 1:1 with a contact (`contact_id` UNIQUE, FK CASCADE). Holds `commitment_amount`, `funded_amount`, `accredited` (bare 0/1), `legal_docs_signed`, `wire_received`, `k1_sent`, `investor_type` (free-text). (`backend/server.py:186`) |
|
||||
| `lp_profiles` | **RETIRED** | Dropped in v0.1.0:104 (migration `0008_drop_retired_tables`) — the table was empty; the `fundraising_*` grid is the canonical commitment record. Formerly a 1:1 closed-LP extension of a contact holding commitment/funded/accreditation fields. |
|
||||
| `custom_fields` / `custom_field_values` | EAV custom fields | **Dead**: schema exists but has **no routes/handlers**; only ever wiped by reset. Do not build on this. (`backend/server.py:206`) |
|
||||
| `tags` | global tag palette | `name` UNIQUE + `color`. Not FK-linked to the per-row `tags` JSON arrays; just an autocomplete source. (`backend/server.py:237`) |
|
||||
| `audit_log` | mutation diff trail | `user_id`, `entity_type`, `entity_id`, `action`, `changes` JSON. **Mutation-only**, no reads, no actor/agent dimension. (`backend/server.py:227`) |
|
||||
@@ -140,11 +140,9 @@ Full REST verbs exist (mutations are **not** tunneled through POST): `do_GET` (1
|
||||
| GET/POST · GET/PUT/DELETE | `/api/opportunities[/{id}]` | Opp CRUD | Bearer |
|
||||
| PATCH | `/api/opportunities/{id}/stage` | Move pipeline stage (validated) | Bearer |
|
||||
| GET/POST · GET/PUT/DELETE | `/api/communications[/{id}]` | Comms CRUD | Bearer |
|
||||
| GET/POST · GET/PUT | `/api/lp-profiles[/{id}]` | LP-profile CRUD (no delete route) | Bearer |
|
||||
| GET | `/api/reports/{dashboard,pipeline,lp-breakdown,activity}` | Aggregates | Bearer |
|
||||
| GET | `/api/reports/{dashboard,pipeline,activity}` | Aggregates | Bearer |
|
||||
| GET | `/api/export/contacts` | Export **all** contacts (returns JSON, not CSV) | Bearer |
|
||||
| POST | `/api/import/csv` | Bulk import from JSON rows | Bearer |
|
||||
| GET/POST · PATCH | `/api/feature-requests[/{id}]` | Feature-request tracker | Bearer |
|
||||
| GET | `/api/users` | List users (no hashes) | Bearer |
|
||||
| POST · PATCH | `/api/admin/users[/{id}]` | Create / update user | **Admin** |
|
||||
| POST | `/api/admin/reset-all-data` | ⚠️ Wipe CRM (confirm phrase `RESET ALL DATA`) | **Admin** |
|
||||
|
||||
@@ -21,6 +21,7 @@ Read this before adding or editing a schema migration or a one-time seed/backfil
|
||||
- **Make migrations/seeders deployment-state-invariant.** Target rows **structurally**, not by transient text the same change mutates; capture prior state so a revert is exact.
|
||||
- *Learned the hard way:* matching old nodes by a body string the same changeset deleted broke fresh DBs. A migration must produce the same end state whether the box is empty, mid-version, or fully seeded.
|
||||
- **Soft-delete only** — `deleted_at` and/or `status='retired'`; never hard-delete CRM records or thesis history.
|
||||
- **Dropping a table is forbidden by default — it needs explicit sign-off** (never-hard-delete). `0008_drop_retired_tables` (lp_profiles + feature_requests, v0.1.0:104) is the one sanctioned exception, for **empty** retired tables only. To actually drop one: (1) **remove its `CREATE TABLE` from `init_db()`** — `init_db()` runs every boot, so leaving it there re-creates the table right after the drop migration runs; (2) add a `DROP TABLE IF EXISTS` forward migration + a `.down.sql` recreating the empty shell; (3) **remove any `ALTER TABLE <dropped_table>` line from an earlier historical migration** — once `init_db()` stops creating the table, that ALTER fails `no such table` on a *fresh* DB and aborts the whole chain (it was the actual bug here). Editing that old migration is safe and deployment-state-invariant: live DBs already applied it before the drop, so the edit only affects fresh DBs, which converge to the same end state. `DROP TABLE IF EXISTS` is a no-op on a fresh DB and removes the table on the live box.
|
||||
|
||||
## Verify before shipping
|
||||
|
||||
|
||||
Reference in New Issue
Block a user