Email-proposal review over Matrix + a bot role (v0.1.0:89)
The email-capture "proposed grid notes" gain two review surfaces:
1. Inline source email — each proposed-note card on the Email Capture page
gets a "View email" toggle that lazily fetches the existing
GET /api/email/detail and shows from/to/cc/date/subject + scrollable body,
so a reviewer can judge the note against the email it was drafted from.
2. CRM->Matrix review bridge — the CRM (box, stdlib, no matrix-nio) can't post
to Matrix, so the intake bot (Spark) PULLS: GET /api/intake/email-proposals
returns to_post/open/to_close work-lists; the bot posts a review card
(metadata + snippet + draft note) to a dedicated review room
(MATRIX_EMAIL_REVIEW_ROOM) and relays in-thread yes / no / NL-edit
(POST .../{id}/decide, note revised via local Qwen). Decisions sync both
ways: web decide -> bot announces + closes the thread; Matrix decide -> the
web panel's ~25s poll clears the card. State lives CRM-side in the new
email_proposal_matrix side row (email-integration migration 0003, additive
+ idempotent CREATE TABLE IF NOT EXISTS), so it survives a bot restart.
Adds a 'bot' role (authenticated, never admin; require_bot_or_admin) to gate
the email-proposal endpoints rather than handing the bot full admin — the
principled base for the coming agentic capabilities. Role controls reach;
the draft->approve gate still controls autonomy (a human approves every write).
Deploy split: endpoints + migration + role + frontend ship in the s9pk; the
bot poll loop + review-room handling ship on the Spark. The bot's CRM user
must be flipped member->bot and joined to the review room (one-time).
Tests: backend/test_email_proposal_matrix.py + matrix_intake/test_email_proposals.py
(30/30 suite green, render-smoke green, migration verified twice on a DB copy).
This commit is contained in:
@@ -12,7 +12,7 @@ Read this before editing Gmail capture or draft creation.
|
||||
## What it does
|
||||
|
||||
- `backend/email_integration/` captures Gmail via **domain-wide delegation** (`credentials.py`, `matcher.py`, `parser.py`, `db.py`, `sync.py`, `scheduler.py`, `routes.py`) and creates Tier-B in-thread drafts (`compose.py`). It has its own `migrations/`.
|
||||
- Captured email becomes CRM activity through a **propose → approve** flow — nothing lands on a contact record until a human approves the proposal.
|
||||
- Captured email becomes CRM activity through a **propose → approve** flow — nothing lands on a contact record until a human approves the proposal. The proposed grid notes show on the **Email Capture** page (admin-only): each card has a **View email** toggle that fetches `GET /api/email/detail?id=` and shows the source email inline (from/to/cc/date/subject + scrollable body) so you can judge the note against it. The same proposals can also be reviewed/approved/edited from a **dedicated Matrix room**, kept in sync with this panel (decide on either surface; the other reflects it) — that CRM→Matrix bridge lives in the **review bot**, see `docs/guides/matrix-intake.md`. The proposal model itself (`email_activity_proposals` + the `propose_email_activity_notes` drafter + the decide path) lives in `backend/server.py`, not this package.
|
||||
|
||||
## Hard rule
|
||||
|
||||
|
||||
@@ -90,6 +90,42 @@ rows ≥ `min_score` (0.62), ranked, capped at 5:
|
||||
only the shortlist, never the whole LP list — intentionally NOT built in this pass, because the
|
||||
deterministic filter already surfaces every duplicate the human then resolves.
|
||||
|
||||
## Email-activity proposal review (the CRM→Matrix bridge, v0.1.0:89)
|
||||
|
||||
A second, separate flow runs alongside intake: reviewing the **proposed grid notes** the CRM
|
||||
drafts from newly-matched email (`server.propose_email_activity_notes`, surfaced on the web Email
|
||||
Capture panel). The bot lets the team approve/dismiss/edit those on mobile, kept **in sync** with
|
||||
the web panel. The CRM (box, stdlib, no matrix-nio) can't post to Matrix, so the bot **pulls**.
|
||||
|
||||
- **Dedicated room** (`MATRIX_EMAIL_REVIEW_ROOM`, see *Config*) — separate from the intake room
|
||||
so high-volume email proposals don't drown the conversational intake. Unset → the whole leg is
|
||||
off (the bot just does intake). The bot must be a **member** of this room.
|
||||
- **Poll loop** (`bot.poll_email_proposals`, every `EMAIL_POLL_SEC`=20s) calls `crm_client.
|
||||
list_email_proposals` → `GET /api/intake/email-proposals`, which returns three work-lists:
|
||||
- **to_post** — pending, not yet posted → the bot posts a review card (metadata + a short email
|
||||
**snippet** + the drafted note; the full body is the web popup's job, kept compact for mobile),
|
||||
then records the thread-root event id via `POST .../{id}/matrix {event_id}`.
|
||||
- **open** — pending, posted, not closed → the bot rebuilds its `event_id → proposal` routing map
|
||||
from these on **every poll**, so replies still route **after a bot restart** (unlike intake's
|
||||
in-memory-only store — the state lives CRM-side in `email_proposal_matrix`).
|
||||
- **to_close** — decided on the **web** while a thread was open → the bot posts a "decided on the
|
||||
web — thread closed" line and `POST .../{id}/matrix {closed:true}`.
|
||||
- **In-thread replies** (`bot.handle_email_reply`, `email_proposals.interpret`): `yes` →
|
||||
`POST .../{id}/decide {decision:"approve", note}` (appends the note to the grid, source='matrix',
|
||||
closes the thread atomically); `no` → dismiss; **anything else → NL revision of the note** via
|
||||
local Qwen (`email_proposals.revise_note`, no Claude/scrub) — re-rendered for re-approval, so the
|
||||
draft→approve gate holds. A no-op/empty revision re-prompts instead of saying "Updated".
|
||||
- **Two surfaces, one source of truth.** Decide on the web → the bot announces + closes the thread;
|
||||
decide on Matrix → the web panel polls `/api/activity/proposals` (~25s) and the card clears.
|
||||
`email_proposal_matrix` (1:1 side row, migration `0003`) carries `event_id`/`posted_at`/`closed_at`;
|
||||
a matrix decision sets `closed_at` in the same txn so it's never re-announced via `to_close`.
|
||||
- **Pure logic is `email_proposals.py`** (card render, reply grammar, note revision) — unit-tested
|
||||
offline in `test_email_proposals.py`; the async poll/post wiring is in `bot.py` (live-smoke only).
|
||||
- **Known minors (low-likelihood, ~5-person team):** if the CRM is unreachable *between* posting a
|
||||
card and recording its event id, the next poll re-posts a duplicate card (the orphan's replies
|
||||
won't route — re-send/decide the recorded one). A mid-revise bot restart loses the in-memory
|
||||
revised note (rebuilt from `open` = the original `proposed_note`; still a valid proposal).
|
||||
|
||||
## Rules / gotchas
|
||||
|
||||
- **Module-name collision:** the intake config module is `settings.py`, **not** `config.py`,
|
||||
@@ -161,7 +197,11 @@ rows ≥ `min_score` (0.62), ranked, capped at 5:
|
||||
- **Server-side endpoints ship in the s9pk, not the bot.** `GET /api/intake/match` and the
|
||||
`source` provenance on `log-communication` live in `backend/server.py`, so they reach the box
|
||||
only via an **s9pk build + install** — a bot restart won't deliver them. (Missed in v83: the
|
||||
box 404'd `/api/intake/match` until **v0.1.0:84**.)
|
||||
box 404'd `/api/intake/match` until **v0.1.0:84**.) **Same split for the email-review bridge
|
||||
(v0.1.0:89):** the `/api/intake/email-proposals*` endpoints + the `email_proposal_matrix`
|
||||
migration (`0003`) + the `bot` role ship in the **s9pk**; the poll loop + review-room handling
|
||||
ship on the **Spark** (git pull + restart). A bot restart against a pre-v89 box returns nothing
|
||||
useful (404/empty), so install the s9pk first, then set the bot user's role + the review room.
|
||||
- **`CRM_API_BASE` is the box over the LAN, not localhost** (bot on the Spark, CRM on the box).
|
||||
`https://immense-voyage.local` (443) is the **StartOS dashboard**, not the CRM — the CRM has
|
||||
its own interface address (the URL you open in a browser); container port 8080 isn't
|
||||
@@ -174,6 +214,19 @@ All in `.env` (names in `.env.example`): `MATRIX_HOMESERVER`, `MATRIX_USER`,
|
||||
`CRM_BOT_USERNAME`, `CRM_BOT_PASSWORD`, `CRM_API_VERIFY_TLS`. Spark settings are inherited from
|
||||
the ingest client (`SPARK_CONTROL_URL`, `CRM_CHAT_MODEL`).
|
||||
|
||||
- **`MATRIX_EMAIL_REVIEW_ROOM`** (optional) — the dedicated room for the email-activity proposal
|
||||
review leg (above). Unset/empty disables that leg entirely (the bot does intake only). The bot
|
||||
must be invited to + joined in this room. Read once at startup, like the room/roster.
|
||||
- **Bot CRM user needs role `bot`.** The email-proposal endpoints (`/api/intake/email-proposals*`)
|
||||
are gated to `require_bot_or_admin` because they expose LP email content (the proposals are
|
||||
admin-only on the web). The `bot` role is **authenticated-but-not-admin** — it passes these
|
||||
endpoints + the auth-only ones the bot already uses (login, `/api/intake/match`,
|
||||
`log-communication`), but **never** `require_admin` (no user-management/settings/security reach).
|
||||
One-time flip of the existing service account (kept out of the invite UI's member/admin dropdown
|
||||
— provision deliberately): an admin `PATCH /api/users/<id> {"role":"bot"}`, or on the box
|
||||
`UPDATE users SET role='bot' WHERE username='<CRM_BOT_USERNAME>';`. Role controls *reach*; the
|
||||
draft→approve gate (a human still approves every write) controls *autonomy* — two separate axes.
|
||||
|
||||
- **`INTAKE_TEAM_ROSTER`** (optional, comma-separated) — Ten31 team-member names that frame the
|
||||
parse (see *Flow* step 1). Use the **first names as actually typed in the room** ("Grant,
|
||||
Jonathan, …"). Read once at startup by `settings.team_roster()`, so **a roster change needs a
|
||||
|
||||
Reference in New Issue
Block a user