Email-proposal review over Matrix + a bot role (v0.1.0:89)
The email-capture "proposed grid notes" gain two review surfaces:
1. Inline source email — each proposed-note card on the Email Capture page
gets a "View email" toggle that lazily fetches the existing
GET /api/email/detail and shows from/to/cc/date/subject + scrollable body,
so a reviewer can judge the note against the email it was drafted from.
2. CRM->Matrix review bridge — the CRM (box, stdlib, no matrix-nio) can't post
to Matrix, so the intake bot (Spark) PULLS: GET /api/intake/email-proposals
returns to_post/open/to_close work-lists; the bot posts a review card
(metadata + snippet + draft note) to a dedicated review room
(MATRIX_EMAIL_REVIEW_ROOM) and relays in-thread yes / no / NL-edit
(POST .../{id}/decide, note revised via local Qwen). Decisions sync both
ways: web decide -> bot announces + closes the thread; Matrix decide -> the
web panel's ~25s poll clears the card. State lives CRM-side in the new
email_proposal_matrix side row (email-integration migration 0003, additive
+ idempotent CREATE TABLE IF NOT EXISTS), so it survives a bot restart.
Adds a 'bot' role (authenticated, never admin; require_bot_or_admin) to gate
the email-proposal endpoints rather than handing the bot full admin — the
principled base for the coming agentic capabilities. Role controls reach;
the draft->approve gate still controls autonomy (a human approves every write).
Deploy split: endpoints + migration + role + frontend ship in the s9pk; the
bot poll loop + review-room handling ship on the Spark. The bot's CRM user
must be flipped member->bot and joined to the review room (one-time).
Tests: backend/test_email_proposal_matrix.py + matrix_intake/test_email_proposals.py
(30/30 suite green, render-smoke green, migration verified twice on a DB copy).
This commit is contained in:
@@ -57,6 +57,13 @@ MATRIX_USER=@intake-bot:<homeserver>
|
|||||||
MATRIX_ACCESS_TOKEN=
|
MATRIX_ACCESS_TOKEN=
|
||||||
MATRIX_DEVICE_ID=ten31-intake-bot
|
MATRIX_DEVICE_ID=ten31-intake-bot
|
||||||
MATRIX_INTAKE_ROOM=!<roomid>:<homeserver>
|
MATRIX_INTAKE_ROOM=!<roomid>:<homeserver>
|
||||||
|
# Dedicated room for reviewing CRM-drafted email-activity proposals (the proposed grid notes the
|
||||||
|
# Email Capture panel shows). The bot posts a review card per pending proposal here and relays the
|
||||||
|
# in-thread yes/no/edit back to the CRM, in sync with the web panel. Separate from the intake room
|
||||||
|
# so high-volume email proposals don't drown the conversational intake. Leave empty to disable the
|
||||||
|
# whole email-review poll loop. The bot must be a member of this room. Needs the server side in the
|
||||||
|
# s9pk (≥ v0.1.0:89) and the bot's CRM user set to role 'bot' (see docs/guides/matrix-intake.md).
|
||||||
|
MATRIX_EMAIL_REVIEW_ROOM=!<roomid>:<homeserver>
|
||||||
# CRM write-back: the bot logs in as a DEDICATED service user (admin-created CRM user;
|
# CRM write-back: the bot logs in as a DEDICATED service user (admin-created CRM user;
|
||||||
# the CRM has no service-key path, so it uses normal Bearer-JWT auth).
|
# the CRM has no service-key path, so it uses normal Bearer-JWT auth).
|
||||||
CRM_API_BASE=http://127.0.0.1:8080
|
CRM_API_BASE=http://127.0.0.1:8080
|
||||||
|
|||||||
@@ -103,7 +103,9 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
|
|||||||
|
|
||||||
## Current state
|
## Current state
|
||||||
|
|
||||||
_Phase 0 + Phase 1 built; **box and repo at v0.1.0:88** (deployed & verified live 2026-06-18 — chain …86→88 clean, `0005_grid_pipeline_link.sql` applied on the box, server up; the full Pipeline **+Pipeline → board → advance-stage → remove** round-trip is live-smoked on the box). **The fundraising grid + email capture is the canonical system of record** (2026-06-16) — vestigial classic-CRM surfaces get pruned/repurposed. Deploy/feature history lives in git log + `start9/0.4/startos/versions/`; longer-term backlog + debt in `ROADMAP.md` / `EVALUATION.md`._
|
_Phase 0 + Phase 1 built; **box at v0.1.0:88, repo at v0.1.0:89 (built, NOT yet deployed)**. v88 deployed & verified live 2026-06-18 — chain …86→88 clean, `0005_grid_pipeline_link.sql` applied on the box, server up; the full Pipeline **+Pipeline → board → advance-stage → remove** round-trip is live-smoked on the box. **The fundraising grid + email capture is the canonical system of record** (2026-06-16) — vestigial classic-CRM surfaces get pruned/repurposed. Deploy/feature history lives in git log + `start9/0.4/startos/versions/`; longer-term backlog + debt in `ROADMAP.md` / `EVALUATION.md`._
|
||||||
|
|
||||||
|
- **Email-proposal review over Matrix + a `bot` role — BUILT v0.1.0:89, NOT yet deployed (2026-06-18).** Two asks on the email-capture "proposed grid notes": (1) **inline source-email popup** — each proposed-note card on the Email Capture page gets a **View email** toggle that lazily fetches the existing **`GET /api/email/detail`** and renders from/to/cc/date/subject + scrollable body inline, so you can judge the note against the email (frontend-only, reuses the Communications detail pattern). (2) **CRM→Matrix review bridge** — the CRM (box, no matrix-nio) can't push, so the **intake bot (Spark) PULLS**: new **`GET /api/intake/email-proposals`** returns three work-lists (`to_post`/`open`/`to_close`); the bot posts a review card (metadata + snippet + draft note) to a **dedicated review room** (`MATRIX_EMAIL_REVIEW_ROOM`), records the thread root (`POST .../{id}/matrix`), and relays in-thread **yes / no / NL-edit** (`POST .../{id}/decide`, NL-revise via local Qwen). **Bidirectional sync:** decide on the web → bot announces + closes the thread; decide on Matrix → the web panel's ~25s poll clears the card. State is CRM-side in **`email_proposal_matrix`** (1:1 side row, email-integration migration **`0003`**, additive + idempotent `CREATE TABLE IF NOT EXISTS` — the email runner re-runs every boot via `executescript`, so no `ALTER`), so it survives a bot restart. New **`bot` role** (authenticated, never admin — `require_bot_or_admin`) gates the email-proposal endpoints; the bot's CRM user must be flipped `member→bot` (one-time, kept out of the invite UI). **Deploy split:** endpoints + migration + role + frontend → **s9pk v89 build+install**; poll loop + review-room handling → **Spark git pull + restart**. Tests: `backend/test_email_proposal_matrix.py` + `backend/matrix_intake/test_email_proposals.py`; **30/30 suite green**, render-smoke green, migration verified twice on a copy of `data/crm.db`. **Next: deploy v89 to the box (auth required), set the bot user's role + the review room + invite the bot, then live-smoke both legs** (popup; web↔Matrix round-trip). Guide: `docs/guides/matrix-intake.md` "Email-activity proposal review".
|
||||||
|
|
||||||
- **Adopt the Pipeline — grid drives the deal board — DEPLOYED & live-smoked 2026-06-18 (v0.1.0:88; the full +Pipeline → board → advance-stage → remove round-trip is verified on the box). v88 (frontend-only): retired the Pipeline page's "+ New Opportunity" button + its create-by-contact modal** — opportunities are now born **only** from a grid investor row (matches how the team works; the board is view + stage-management; button replaced with a muted "Add deals from the Fundraising Grid" hint). An **"Add to Pipeline"** row action on the fundraising grid opens a seed modal (primary contact / target fund / expected amount / stage / probability) and creates a durably-linked `opportunities` row via the new **`opportunities.fundraising_investor_id`** (migration 0005, additive + reversible). **Grid owns the link + seed; the board owns stage/probability/owner** — a grid save never reseeds a live opp (`POST /api/fundraising/pipeline/link` is idempotent, one live opp/investor). Contact is **reused from the grid's synced `fundraising_contacts.contact_id`** (the `POST /api/contacts` side-door is gone); grid `lead`→owner. Two **read-only** grid columns (Pipeline action + Pipeline Stage) are **injected on read** from the live opp and **stripped on write** (never persisted, never dirty the autosave). **Remove from pipeline** (`POST .../unlink`) **soft-deletes the opp; the grid row stays fully intact**; deleting an investor from the grid archives its orphaned opp (`reconcile_grid_pipeline_links`, after `sync_fundraising_relational`). **Folded in:** the standing P2 soft-delete leak in `handle_pipeline_report` + dashboard pipeline aggregates (archived opps no longer counted). Tests: `backend/test_grid_pipeline_link.py`; 28/28 suite green, render-smoke green; migration verified on a copy of `data/crm.db` and **applied clean on the box**. **Next: live-smoke on the box — add an investor to the pipeline, confirm it lands on the board, advance a stage, and remove (opp archived, grid row intact).** Detail + locked decisions in `ROADMAP.md` "Adopt the Pipeline".
|
- **Adopt the Pipeline — grid drives the deal board — DEPLOYED & live-smoked 2026-06-18 (v0.1.0:88; the full +Pipeline → board → advance-stage → remove round-trip is verified on the box). v88 (frontend-only): retired the Pipeline page's "+ New Opportunity" button + its create-by-contact modal** — opportunities are now born **only** from a grid investor row (matches how the team works; the board is view + stage-management; button replaced with a muted "Add deals from the Fundraising Grid" hint). An **"Add to Pipeline"** row action on the fundraising grid opens a seed modal (primary contact / target fund / expected amount / stage / probability) and creates a durably-linked `opportunities` row via the new **`opportunities.fundraising_investor_id`** (migration 0005, additive + reversible). **Grid owns the link + seed; the board owns stage/probability/owner** — a grid save never reseeds a live opp (`POST /api/fundraising/pipeline/link` is idempotent, one live opp/investor). Contact is **reused from the grid's synced `fundraising_contacts.contact_id`** (the `POST /api/contacts` side-door is gone); grid `lead`→owner. Two **read-only** grid columns (Pipeline action + Pipeline Stage) are **injected on read** from the live opp and **stripped on write** (never persisted, never dirty the autosave). **Remove from pipeline** (`POST .../unlink`) **soft-deletes the opp; the grid row stays fully intact**; deleting an investor from the grid archives its orphaned opp (`reconcile_grid_pipeline_links`, after `sync_fundraising_relational`). **Folded in:** the standing P2 soft-delete leak in `handle_pipeline_report` + dashboard pipeline aggregates (archived opps no longer counted). Tests: `backend/test_grid_pipeline_link.py`; 28/28 suite green, render-smoke green; migration verified on a copy of `data/crm.db` and **applied clean on the box**. **Next: live-smoke on the box — add an investor to the pipeline, confirm it lands on the board, advance a stage, and remove (opp archived, grid row intact).** Detail + locked decisions in `ROADMAP.md` "Adopt the Pipeline".
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,30 @@
|
|||||||
|
-- ============================================================================
|
||||||
|
-- email_proposal_matrix — Matrix-review state for an email_activity_proposal,
|
||||||
|
-- kept 1:1 with the proposal (proposal_id PK). The CRM runs on the box and has
|
||||||
|
-- no matrix-nio, so it cannot post to Matrix itself: the intake bot (on the Spark)
|
||||||
|
-- PULLS pending proposals, posts a review card to the dedicated Matrix review room,
|
||||||
|
-- and writes the thread-root event_id back here. Persisting it CRM-side (not just in
|
||||||
|
-- the bot's memory) keeps both surfaces in sync and survives a bot restart.
|
||||||
|
--
|
||||||
|
-- A SIDE TABLE rather than new columns on email_activity_proposals because the
|
||||||
|
-- email-integration migration runner (email_integration/db.py:apply_migrations)
|
||||||
|
-- re-runs every .sql file on every boot via executescript with no ledger — so
|
||||||
|
-- CREATE TABLE IF NOT EXISTS is idempotent, whereas ALTER ... ADD COLUMN would throw
|
||||||
|
-- "duplicate column" on the second boot and abort startup. Reversal: DROP TABLE
|
||||||
|
-- (this runner has no .down.sql convention; cf. 0001/0002).
|
||||||
|
--
|
||||||
|
-- posted_at — set once the bot has posted the review card (event_id = thread root).
|
||||||
|
-- closed_at — set when the thread is resolved: either the bot decided in-thread, OR
|
||||||
|
-- the bot announced a web-side decision. A posted+decided proposal with
|
||||||
|
-- closed_at NULL is exactly the bot's signal to post "decided on the web"
|
||||||
|
-- into the thread and then close it.
|
||||||
|
-- ============================================================================
|
||||||
|
CREATE TABLE IF NOT EXISTS email_proposal_matrix (
|
||||||
|
proposal_id TEXT PRIMARY KEY,
|
||||||
|
event_id TEXT, -- Matrix thread-root event id of the posted review card
|
||||||
|
posted_at TEXT,
|
||||||
|
closed_at TEXT,
|
||||||
|
created_at TEXT DEFAULT (datetime('now')),
|
||||||
|
FOREIGN KEY(proposal_id) REFERENCES email_activity_proposals(id) ON DELETE CASCADE
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_email_proposal_matrix_event ON email_proposal_matrix(event_id);
|
||||||
@@ -14,6 +14,7 @@ import asyncio
|
|||||||
from nio import AsyncClient, MatrixRoom, RoomMessageText
|
from nio import AsyncClient, MatrixRoom, RoomMessageText
|
||||||
|
|
||||||
import crm_client
|
import crm_client
|
||||||
|
import email_proposals
|
||||||
import matrix_io
|
import matrix_io
|
||||||
import parse
|
import parse
|
||||||
import proposals
|
import proposals
|
||||||
@@ -25,6 +26,8 @@ UNCLEAR_HELP = (
|
|||||||
"or a note like `Note for Acme Capital: wants the Q3 deck, follow up next week`."
|
"or a note like `Note for Acme Capital: wants the Q3 deck, follow up next week`."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
EMAIL_POLL_SEC = 20 # how often the bot polls the CRM for new/decided email-activity proposals
|
||||||
|
|
||||||
|
|
||||||
async def main():
|
async def main():
|
||||||
mx = settings.matrix_settings()
|
mx = settings.matrix_settings()
|
||||||
@@ -37,6 +40,8 @@ async def main():
|
|||||||
roster = settings.team_roster() # frames the parse: teammates do outreach, aren't prospects
|
roster = settings.team_roster() # frames the parse: teammates do outreach, aren't prospects
|
||||||
if roster:
|
if roster:
|
||||||
print(f"matrix-intake: team roster loaded ({len(roster)} names)", flush=True)
|
print(f"matrix-intake: team roster loaded ({len(roster)} names)", flush=True)
|
||||||
|
review_room = settings.email_review_room() # CRM-drafted email proposals (empty → feature off)
|
||||||
|
email_threads = {} # Matrix thread-root event_id -> {id, investor_name, note} for an email proposal
|
||||||
|
|
||||||
async def handle_intake(room_id, root, text):
|
async def handle_intake(room_id, root, text):
|
||||||
# A bare yes/no/approve typed in the MAIN timeline (not inside a proposal's thread) is
|
# A bare yes/no/approve typed in the MAIN timeline (not inside a proposal's thread) is
|
||||||
@@ -157,15 +162,103 @@ async def main():
|
|||||||
store.put(root, proposal)
|
store.put(root, proposal)
|
||||||
await say(room_id, "I didn't catch that.\n\n" + proposals.render_disambiguation(proposal), root)
|
await say(room_id, "I didn't catch that.\n\n" + proposals.render_disambiguation(proposal), root)
|
||||||
|
|
||||||
|
async def handle_email_reply(room_id, root, text):
|
||||||
|
"""An in-thread reply to a CRM-drafted email-proposal card: yes commits, no dismisses, and
|
||||||
|
anything else is a natural-language revision of the note (re-drafted by local Qwen; the
|
||||||
|
human still approves the revised note, so the draft→approve gate holds)."""
|
||||||
|
item = email_threads.get(root)
|
||||||
|
if item is None:
|
||||||
|
return # a threaded reply we don't own (or already resolved)
|
||||||
|
decision = email_proposals.interpret(text)
|
||||||
|
if decision == "approve":
|
||||||
|
# Claim before the await (double-approve guard, like the intake commit path).
|
||||||
|
email_threads.pop(root, None)
|
||||||
|
try:
|
||||||
|
await asyncio.to_thread(crm_client.decide_email_proposal, item["id"], "approve", item.get("note"))
|
||||||
|
except Exception as exc:
|
||||||
|
email_threads[root] = item # restore for retry
|
||||||
|
await say(room_id, f"⚠️ couldn't add it ({str(exc)[:200]}). Reply **yes** to retry, **no** to dismiss.", root)
|
||||||
|
return
|
||||||
|
await say(room_id, f"✅ Added to the grid for **{item.get('investor_name') or 'the investor'}**.", root)
|
||||||
|
elif decision == "reject":
|
||||||
|
email_threads.pop(root, None)
|
||||||
|
try:
|
||||||
|
await asyncio.to_thread(crm_client.decide_email_proposal, item["id"], "dismiss")
|
||||||
|
except Exception as exc:
|
||||||
|
email_threads[root] = item
|
||||||
|
await say(room_id, f"⚠️ couldn't dismiss it ({str(exc)[:200]}). Try again.", root)
|
||||||
|
return
|
||||||
|
await say(room_id, "🗑️ Dismissed — nothing added to the grid.", root)
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
new_note = await asyncio.to_thread(email_proposals.revise_note, item.get("note") or "", text)
|
||||||
|
except Exception as exc:
|
||||||
|
await say(room_id, f"⚠️ couldn't revise that ({str(exc)[:200]}). Reply **yes** to add as-is, "
|
||||||
|
"**no** to dismiss, or rephrase.", root)
|
||||||
|
return
|
||||||
|
if not new_note:
|
||||||
|
await say(room_id, "I didn't catch a change. Reply **yes** to add the note as-is, **no** to "
|
||||||
|
"dismiss, or tell me how to change it.", root)
|
||||||
|
return
|
||||||
|
item["note"] = new_note
|
||||||
|
email_threads[root] = item
|
||||||
|
await say(room_id, f"✏️ Updated draft note:\n\n{new_note}\n\nReply **yes** to add it, **no** to "
|
||||||
|
"dismiss, or refine again.", root)
|
||||||
|
|
||||||
|
async def poll_email_proposals():
|
||||||
|
"""Poll the CRM for email-activity proposals: post a review card for each new one, rebuild
|
||||||
|
the reply-routing map from already-posted threads (so replies still route after a restart),
|
||||||
|
and announce+close any decided on the web. One failing cycle logs and retries next tick."""
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
lists = await asyncio.to_thread(crm_client.list_email_proposals)
|
||||||
|
for it in lists["open"]: # rebuild routing for threads posted before (e.g. a restart)
|
||||||
|
ev = it.get("event_id")
|
||||||
|
if ev and ev not in email_threads:
|
||||||
|
email_threads[ev] = {"id": it["id"], "investor_name": it.get("investor_name"),
|
||||||
|
"note": it.get("proposed_note") or ""}
|
||||||
|
for it in lists["to_post"]:
|
||||||
|
try:
|
||||||
|
resp = await client.room_send(
|
||||||
|
review_room, "m.room.message",
|
||||||
|
matrix_io.thread_content(email_proposals.render_card(it), None))
|
||||||
|
ev = getattr(resp, "event_id", None)
|
||||||
|
if not ev:
|
||||||
|
print(f"matrix-intake: card send returned no event_id for {it['id']}", flush=True)
|
||||||
|
continue
|
||||||
|
await asyncio.to_thread(crm_client.mark_email_proposal_posted, it["id"], ev)
|
||||||
|
email_threads[ev] = {"id": it["id"], "investor_name": it.get("investor_name"),
|
||||||
|
"note": it.get("proposed_note") or ""}
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"matrix-intake: failed to post email proposal {it.get('id')}: {exc}", flush=True)
|
||||||
|
for it in lists["to_close"]: # decided on the web → announce in-thread, then close
|
||||||
|
ev = it.get("event_id")
|
||||||
|
if not ev:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
await say(review_room, email_proposals.closure_line(it.get("status")), ev)
|
||||||
|
await asyncio.to_thread(crm_client.mark_email_proposal_closed, it["id"])
|
||||||
|
email_threads.pop(ev, None)
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"matrix-intake: failed to close email proposal {it.get('id')}: {exc}", flush=True)
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"matrix-intake: email-proposal poll error: {exc}", flush=True)
|
||||||
|
await asyncio.sleep(EMAIL_POLL_SEC)
|
||||||
|
|
||||||
async def on_message(room: MatrixRoom, event: RoomMessageText):
|
async def on_message(room: MatrixRoom, event: RoomMessageText):
|
||||||
if event.sender == mx["user_id"]:
|
if event.sender == mx["user_id"]:
|
||||||
return # never react to our own messages (we post in-thread — this prevents loops)
|
return # never react to our own messages (we post in-thread — this prevents loops)
|
||||||
if room.room_id != intake_room:
|
|
||||||
return
|
|
||||||
text = (event.body or "").strip()
|
text = (event.body or "").strip()
|
||||||
if not text:
|
if not text:
|
||||||
return
|
return
|
||||||
root = matrix_io.thread_root_of(event)
|
root = matrix_io.thread_root_of(event)
|
||||||
|
# Email-proposal review room: only a threaded reply to a card we posted is actionable.
|
||||||
|
if review_room and room.room_id == review_room:
|
||||||
|
if root and root in email_threads:
|
||||||
|
await handle_email_reply(room.room_id, root, text)
|
||||||
|
return
|
||||||
|
if room.room_id != intake_room:
|
||||||
|
return
|
||||||
if root and store.has(root):
|
if root and store.has(root):
|
||||||
await handle_reply(room.room_id, root, text)
|
await handle_reply(room.room_id, root, text)
|
||||||
elif root:
|
elif root:
|
||||||
@@ -180,8 +273,12 @@ async def main():
|
|||||||
client.add_event_callback(on_message, RoomMessageText)
|
client.add_event_callback(on_message, RoomMessageText)
|
||||||
who = await client.whoami()
|
who = await client.whoami()
|
||||||
print(f"matrix-intake: listening as {who.user_id} in room {intake_room}", flush=True)
|
print(f"matrix-intake: listening as {who.user_id} in room {intake_room}", flush=True)
|
||||||
|
tasks = [asyncio.create_task(client.sync_forever(timeout=30000))]
|
||||||
|
if review_room:
|
||||||
|
tasks.append(asyncio.create_task(poll_email_proposals()))
|
||||||
|
print(f"matrix-intake: reviewing email proposals in room {review_room} (every {EMAIL_POLL_SEC}s)", flush=True)
|
||||||
try:
|
try:
|
||||||
await client.sync_forever(timeout=30000)
|
await asyncio.gather(*tasks)
|
||||||
finally:
|
finally:
|
||||||
await client.close()
|
await client.close()
|
||||||
|
|
||||||
|
|||||||
@@ -98,6 +98,47 @@ def match(proposal):
|
|||||||
return {"match": match_out, "candidates": candidates}
|
return {"match": match_out, "candidates": candidates}
|
||||||
|
|
||||||
|
|
||||||
|
def list_email_proposals():
|
||||||
|
"""Pull the email-activity review work-lists for the poll loop: {to_post, open, to_close}.
|
||||||
|
to_post = pending, un-posted (post a card); open = posted, awaiting a decision (rebuild the
|
||||||
|
reply-routing map after a restart); to_close = decided on the web (announce in-thread + close)."""
|
||||||
|
status, data = _authed("GET", "/api/intake/email-proposals")
|
||||||
|
if status != 200:
|
||||||
|
raise RuntimeError(f"email-proposals list failed ({status}): {data.get('error') or data}")
|
||||||
|
payload = data.get("data") or {}
|
||||||
|
return {k: (payload.get(k) or []) for k in ("to_post", "open", "to_close")}
|
||||||
|
|
||||||
|
|
||||||
|
def mark_email_proposal_posted(proposal_id, event_id):
|
||||||
|
"""Record the Matrix thread-root event id so the proposal's review state survives a restart."""
|
||||||
|
status, data = _authed("POST", f"/api/intake/email-proposals/{proposal_id}/matrix",
|
||||||
|
{"event_id": event_id})
|
||||||
|
if status != 200:
|
||||||
|
raise RuntimeError(f"mark posted failed ({status}): {data.get('error') or data}")
|
||||||
|
return data.get("data") or {}
|
||||||
|
|
||||||
|
|
||||||
|
def mark_email_proposal_closed(proposal_id):
|
||||||
|
"""Mark the review thread resolved after announcing a web-side decision in it."""
|
||||||
|
status, data = _authed("POST", f"/api/intake/email-proposals/{proposal_id}/matrix",
|
||||||
|
{"closed": True})
|
||||||
|
if status != 200:
|
||||||
|
raise RuntimeError(f"mark closed failed ({status}): {data.get('error') or data}")
|
||||||
|
return data.get("data") or {}
|
||||||
|
|
||||||
|
|
||||||
|
def decide_email_proposal(proposal_id, decision, note=None):
|
||||||
|
"""Relay an in-thread approve/dismiss (with the possibly-revised note) to the CRM. The server
|
||||||
|
appends the note to the grid on approve, tags source='matrix', and closes the thread."""
|
||||||
|
body = {"decision": decision}
|
||||||
|
if note is not None:
|
||||||
|
body["note"] = note
|
||||||
|
status, data = _authed("POST", f"/api/intake/email-proposals/{proposal_id}/decide", body)
|
||||||
|
if status not in (200, 201):
|
||||||
|
raise RuntimeError(f"email-proposal decide failed ({status}): {data.get('error') or data}")
|
||||||
|
return data.get("data") or {}
|
||||||
|
|
||||||
|
|
||||||
def build_commit_payload(proposal):
|
def build_commit_payload(proposal):
|
||||||
"""Pure: map a proposal to the /api/fundraising/log-communication request body.
|
"""Pure: map a proposal to the /api/fundraising/log-communication request body.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,85 @@
|
|||||||
|
"""Email-activity proposal review over Matrix — the CRM→Matrix leg of the email-capture flow.
|
||||||
|
|
||||||
|
The CRM (on the box) drafts a proposed grid note per newly-matched email (local model, no Claude)
|
||||||
|
and queues it for human review. The CRM is stdlib-only and can't post to Matrix itself, so this
|
||||||
|
bot PULLS the pending proposals (crm_client.list_email_proposals), posts a review card to the
|
||||||
|
dedicated review room, and relays the human's in-thread reply back to the CRM. Same draft→approve
|
||||||
|
discipline as the intake bot: nothing is appended to the grid until a human approves — here OR on
|
||||||
|
the web Email Capture panel, the two surfaces kept in sync via the CRM's email_proposal_matrix row.
|
||||||
|
|
||||||
|
This module is the PURE logic (card rendering, reply grammar, note revision) so it's unit-tested
|
||||||
|
offline; the async poll/post/reply wiring lives in bot.py (network + Matrix, live-smoke only).
|
||||||
|
"""
|
||||||
|
import spark
|
||||||
|
|
||||||
|
_YES = {"yes", "y", "approve", "approved", "ok", "confirm", "go", "add", "👍", "✅"}
|
||||||
|
_NO = {"no", "n", "cancel", "discard", "reject", "skip", "stop", "👎", "❌"}
|
||||||
|
|
||||||
|
_SNIPPET_MAX = 400 # email snippet shown on the card; the full body is in the web popup
|
||||||
|
|
||||||
|
|
||||||
|
def _truncate(s, n):
|
||||||
|
s = (s or "").strip()
|
||||||
|
return s if len(s) <= n else s[:n].rstrip() + "…"
|
||||||
|
|
||||||
|
|
||||||
|
def render_card(item):
|
||||||
|
"""The review card posted to the Matrix review room: who/when + a short email snippet + the
|
||||||
|
drafted note. Deliberately compact for mobile — the full scrollable body is in the web Email
|
||||||
|
Capture popup (this is the metadata+snippet+note choice)."""
|
||||||
|
name = item.get("investor_name") or "Unknown investor"
|
||||||
|
direction = "Sent" if item.get("direction") == "sent" else "Received"
|
||||||
|
frm = item.get("from_name") or item.get("from_email") or "?"
|
||||||
|
lines = [f"📧 Proposed **grid note** for **{name}** ({direction})"]
|
||||||
|
if item.get("email_subject"):
|
||||||
|
lines.append(f"· Subject: {item['email_subject']}")
|
||||||
|
if item.get("email_date"):
|
||||||
|
lines.append(f"· Date: {item['email_date']}")
|
||||||
|
lines.append(f"· From: {frm}")
|
||||||
|
snippet = _truncate(item.get("snippet"), _SNIPPET_MAX)
|
||||||
|
if snippet:
|
||||||
|
lines.append(f"· Email: {snippet}")
|
||||||
|
lines.append("")
|
||||||
|
lines.append(f"📝 Draft note: {item.get('proposed_note') or '(empty)'}")
|
||||||
|
lines.append("")
|
||||||
|
lines.append("Reply **yes** to add it to the grid, **no** to dismiss, or just tell me how to "
|
||||||
|
"change the note (e.g. *say we discussed the Q3 raise*).")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def closure_line(status):
|
||||||
|
"""Posted in-thread when a proposal was decided on the WEB while its Matrix thread was open."""
|
||||||
|
verb = "approved ✅ and added to the grid" if status == "approved" else "dismissed 🗑️"
|
||||||
|
return f"This was {verb} on the web — nothing more to do here. Thread closed."
|
||||||
|
|
||||||
|
|
||||||
|
def interpret(text):
|
||||||
|
"""Classify an in-thread reply: 'approve' | 'reject' | 'revise' (anything else → revise the note)."""
|
||||||
|
t = (text or "").strip().lower()
|
||||||
|
if t in _YES:
|
||||||
|
return "approve"
|
||||||
|
if t in _NO:
|
||||||
|
return "reject"
|
||||||
|
return "revise"
|
||||||
|
|
||||||
|
|
||||||
|
REVISE_SYSTEM = (
|
||||||
|
"You revise a single CRM note from a short instruction a venture-fund team member typed. "
|
||||||
|
"You are given the CURRENT note and an INSTRUCTION. Apply the instruction and reply with "
|
||||||
|
"ONLY a JSON object of the form {\"note\": \"<the full revised note>\"}. Keep it to one or two "
|
||||||
|
"factual sentences, no preamble. Output JSON only."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def revise_note(note, instruction, parse_fn=spark.parse_json):
|
||||||
|
"""Re-draft the note via local Qwen from a free-form instruction (no Claude, no scrub — same
|
||||||
|
local-only basis as the intake parse). Returns the new note text, or None if the model gave
|
||||||
|
nothing usable / unchanged, in which case the caller re-prompts. `parse_fn` is injectable for
|
||||||
|
tests."""
|
||||||
|
prompt = "CURRENT:\n" + (note or "") + "\n\nINSTRUCTION:\n" + (instruction or "").strip()
|
||||||
|
raw = parse_fn(prompt, system=REVISE_SYSTEM, max_tokens=400) or {}
|
||||||
|
new = raw.get("note") if isinstance(raw, dict) else None
|
||||||
|
new = (new or "").strip()
|
||||||
|
if not new or new == (note or "").strip():
|
||||||
|
return None
|
||||||
|
return new
|
||||||
@@ -61,3 +61,10 @@ def crm_settings():
|
|||||||
# unset/empty just means no roster framing, i.e. the prior behavior.
|
# unset/empty just means no roster framing, i.e. the prior behavior.
|
||||||
def team_roster():
|
def team_roster():
|
||||||
return [n.strip() for n in os.environ.get("INTAKE_TEAM_ROSTER", "").split(",") if n.strip()]
|
return [n.strip() for n in os.environ.get("INTAKE_TEAM_ROSTER", "").split(",") if n.strip()]
|
||||||
|
|
||||||
|
|
||||||
|
# Dedicated room for reviewing CRM-drafted email-activity proposals (the CRM→Matrix push leg).
|
||||||
|
# Separate from the intake room so high-volume email proposals don't drown the conversational
|
||||||
|
# intake flow. Unset/empty disables the whole email-review poll loop (the bot just does intake).
|
||||||
|
def email_review_room():
|
||||||
|
return os.environ.get("MATRIX_EMAIL_REVIEW_ROOM", "").strip()
|
||||||
|
|||||||
@@ -0,0 +1,72 @@
|
|||||||
|
"""Offline tests for the email-proposal review logic (card render, reply grammar, note revision).
|
||||||
|
The network/Matrix wiring lives in bot.py (live-smoke only); this covers the pure functions."""
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
|
||||||
|
import email_proposals # noqa: E402
|
||||||
|
|
||||||
|
ITEM = {
|
||||||
|
"id": "p1", "investor_name": "Acme Capital", "direction": "received",
|
||||||
|
"from_name": "Jane Doe", "from_email": "jane@acme.com",
|
||||||
|
"email_subject": "Re: Fund III", "email_date": "2026-06-02",
|
||||||
|
"snippet": "thanks for the deck — one question on terms", "proposed_note": "✉ Received: asked about terms",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_interpret_yes_no_else():
|
||||||
|
assert email_proposals.interpret("yes") == "approve"
|
||||||
|
assert email_proposals.interpret(" Y ") == "approve"
|
||||||
|
assert email_proposals.interpret("✅") == "approve"
|
||||||
|
assert email_proposals.interpret("no") == "reject"
|
||||||
|
assert email_proposals.interpret("skip") == "reject"
|
||||||
|
# anything that isn't a clear yes/no is treated as a revision instruction
|
||||||
|
assert email_proposals.interpret("say we discussed the Q3 raise") == "revise"
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_card_has_context_note_and_actions():
|
||||||
|
card = email_proposals.render_card(ITEM)
|
||||||
|
assert "Acme Capital" in card and "Received" in card
|
||||||
|
assert "Jane Doe" in card
|
||||||
|
assert "Re: Fund III" in card and "2026-06-02" in card
|
||||||
|
assert "thanks for the deck" in card
|
||||||
|
assert "✉ Received: asked about terms" in card
|
||||||
|
assert "yes" in card.lower() and "no" in card.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_card_sent_direction():
|
||||||
|
assert "(Sent)" in email_proposals.render_card(dict(ITEM, direction="sent"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_card_truncates_long_snippet():
|
||||||
|
card = email_proposals.render_card(dict(ITEM, snippet="x" * 1000))
|
||||||
|
assert "…" in card and len(card) < 1000
|
||||||
|
|
||||||
|
|
||||||
|
def test_revise_note_applies_model_output():
|
||||||
|
out = email_proposals.revise_note(
|
||||||
|
"old note", "make it about the Q3 raise",
|
||||||
|
parse_fn=lambda prompt, system=None, max_tokens=400: {"note": "Discussed the Q3 raise."})
|
||||||
|
assert out == "Discussed the Q3 raise."
|
||||||
|
|
||||||
|
|
||||||
|
def test_revise_note_noop_or_empty_returns_none():
|
||||||
|
# model echoes the same note unchanged -> None so the caller re-prompts (not "Updated")
|
||||||
|
assert email_proposals.revise_note("same", "x", parse_fn=lambda *a, **k: {"note": "same"}) is None
|
||||||
|
# model returns nothing usable -> None
|
||||||
|
assert email_proposals.revise_note("n", "y", parse_fn=lambda *a, **k: {}) is None
|
||||||
|
assert email_proposals.revise_note("n", "y", parse_fn=lambda *a, **k: None) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_closure_line_reflects_status():
|
||||||
|
assert "approved" in email_proposals.closure_line("approved").lower()
|
||||||
|
assert "dismiss" in email_proposals.closure_line("dismissed").lower()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
fns = [v for k, v in sorted(globals().items()) if k.startswith("test_") and callable(v)]
|
||||||
|
for fn in fns:
|
||||||
|
fn()
|
||||||
|
print(f"ok {fn.__name__}")
|
||||||
|
print(f"\n{len(fns)} passed")
|
||||||
+153
-7
@@ -619,6 +619,14 @@ def parse_iso_utc(ts):
|
|||||||
def require_admin(user):
|
def require_admin(user):
|
||||||
return bool(user and user.get('role') == 'admin')
|
return bool(user and user.get('role') == 'admin')
|
||||||
|
|
||||||
|
def require_bot_or_admin(user):
|
||||||
|
"""Gate for agent/bot-facing endpoints: a dedicated 'bot' service account OR an admin
|
||||||
|
(admins keep parity for debugging/curl). The 'bot' role is authenticated-but-not-admin —
|
||||||
|
it never passes require_admin, so a bot credential cannot reach user-management, security,
|
||||||
|
or settings. Reach (which endpoints) is controlled here; autonomy (acting without a human)
|
||||||
|
stays governed by the per-feature draft->approve gate, independent of role."""
|
||||||
|
return bool(user and user.get('role') in ('admin', 'bot'))
|
||||||
|
|
||||||
def log_audit(conn, user_id, entity_type, entity_id, action, changes=None):
|
def log_audit(conn, user_id, entity_type, entity_id, action, changes=None):
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO audit_log (id, user_id, entity_type, entity_id, action, changes) VALUES (?, ?, ?, ?, ?, ?)",
|
"INSERT INTO audit_log (id, user_id, entity_type, entity_id, action, changes) VALUES (?, ?, ?, ?, ?, ?)",
|
||||||
@@ -2068,6 +2076,9 @@ class CRMHandler(BaseHTTPRequestHandler):
|
|||||||
# Matrix intake bot — new-vs-existing lookup for its in-thread proposal
|
# Matrix intake bot — new-vs-existing lookup for its in-thread proposal
|
||||||
if path == '/api/intake/match':
|
if path == '/api/intake/match':
|
||||||
return self.handle_intake_match(user, params)
|
return self.handle_intake_match(user, params)
|
||||||
|
# Matrix review bot — email-activity proposal work-lists (to_post/open/to_close)
|
||||||
|
if path == '/api/intake/email-proposals':
|
||||||
|
return self.handle_list_bot_email_proposals(user)
|
||||||
|
|
||||||
# Users
|
# Users
|
||||||
if path == '/api/users':
|
if path == '/api/users':
|
||||||
@@ -2187,6 +2198,10 @@ class CRMHandler(BaseHTTPRequestHandler):
|
|||||||
return self.handle_decide_activity_proposal(user, path.split('/')[-2], 'approve', body)
|
return self.handle_decide_activity_proposal(user, path.split('/')[-2], 'approve', body)
|
||||||
if re.match(r'^/api/activity/proposals/[^/]+/dismiss$', path):
|
if re.match(r'^/api/activity/proposals/[^/]+/dismiss$', path):
|
||||||
return self.handle_decide_activity_proposal(user, path.split('/')[-2], 'dismiss', body)
|
return self.handle_decide_activity_proposal(user, path.split('/')[-2], 'dismiss', body)
|
||||||
|
if re.match(r'^/api/intake/email-proposals/[^/]+/matrix$', path):
|
||||||
|
return self.handle_mark_email_proposal_matrix(user, path.split('/')[-2], body)
|
||||||
|
if re.match(r'^/api/intake/email-proposals/[^/]+/decide$', path):
|
||||||
|
return self.handle_decide_email_proposal_matrix(user, path.split('/')[-2], body)
|
||||||
if re.match(r'^/api/thesis/nodes/[^/]+/choose$', path):
|
if re.match(r'^/api/thesis/nodes/[^/]+/choose$', path):
|
||||||
return self.handle_choose_variant(user, path.split('/')[-2])
|
return self.handle_choose_variant(user, path.split('/')[-2])
|
||||||
if re.match(r'^/api/thesis/lines/[^/]+/approve$', path):
|
if re.match(r'^/api/thesis/lines/[^/]+/approve$', path):
|
||||||
@@ -3964,6 +3979,59 @@ class CRMHandler(BaseHTTPRequestHandler):
|
|||||||
return self.send_error_json(res["error"], code)
|
return self.send_error_json(res["error"], code)
|
||||||
return self.send_json({"data": res})
|
return self.send_json({"data": res})
|
||||||
|
|
||||||
|
# ─── Matrix review-bot bridge for email-activity proposals (bot-or-admin) ───
|
||||||
|
def handle_list_bot_email_proposals(self, user):
|
||||||
|
"""The bot's poll endpoint: {to_post, open, to_close}. Bot-or-admin (the proposals
|
||||||
|
carry LP email content, so this stays off the member tier)."""
|
||||||
|
if not require_bot_or_admin(user):
|
||||||
|
return self.send_error_json("Bot or admin required", 403)
|
||||||
|
conn = get_db()
|
||||||
|
try:
|
||||||
|
return self.send_json({"data": list_bot_email_proposals(conn)})
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
def handle_mark_email_proposal_matrix(self, user, proposal_id, body):
|
||||||
|
"""Record Matrix thread state: {event_id} marks the card posted; {closed:true} marks
|
||||||
|
the thread resolved after the bot announced a web-side decision."""
|
||||||
|
if not require_bot_or_admin(user):
|
||||||
|
return self.send_error_json("Bot or admin required", 403)
|
||||||
|
body = body or {}
|
||||||
|
conn = get_db()
|
||||||
|
try:
|
||||||
|
if body.get("closed"):
|
||||||
|
res = mark_proposal_matrix_closed(conn, proposal_id)
|
||||||
|
else:
|
||||||
|
event_id = str(body.get("event_id") or "").strip()
|
||||||
|
if not event_id:
|
||||||
|
return self.send_error_json("event_id or closed is required")
|
||||||
|
res = mark_proposal_matrix_posted(conn, proposal_id, event_id)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
if res.get("error"):
|
||||||
|
return self.send_error_json(res["error"], 404 if res["error"] == "not_found" else 400)
|
||||||
|
return self.send_json({"data": res})
|
||||||
|
|
||||||
|
def handle_decide_email_proposal_matrix(self, user, proposal_id, body):
|
||||||
|
"""In-thread Matrix decision relayed by the bot: approve/dismiss (+ optional edited note),
|
||||||
|
tagged source='matrix' and closing the thread in the same transaction."""
|
||||||
|
if not require_bot_or_admin(user):
|
||||||
|
return self.send_error_json("Bot or admin required", 403)
|
||||||
|
body = body or {}
|
||||||
|
decision = str(body.get("decision") or "").strip()
|
||||||
|
if decision not in ("approve", "dismiss"):
|
||||||
|
return self.send_error_json("decision must be approve or dismiss")
|
||||||
|
conn = get_db()
|
||||||
|
try:
|
||||||
|
res = decide_email_activity_proposal(conn, proposal_id, decision, user['user_id'],
|
||||||
|
body.get('note'), source="matrix", close_matrix=True)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
if res.get("error"):
|
||||||
|
code = {"not_found": 404, "already_decided": 409}.get(res["error"], 400)
|
||||||
|
return self.send_error_json(res["error"], code)
|
||||||
|
return self.send_json({"data": res})
|
||||||
|
|
||||||
# ─── UI-triggered index jobs + entity-merge review (Phase 1) ───
|
# ─── UI-triggered index jobs + entity-merge review (Phase 1) ───
|
||||||
def handle_index_job(self, user, kind):
|
def handle_index_job(self, user, kind):
|
||||||
if not require_admin(user):
|
if not require_admin(user):
|
||||||
@@ -4357,8 +4425,10 @@ class CRMHandler(BaseHTTPRequestHandler):
|
|||||||
return self.send_error_json("password must be at least 8 characters")
|
return self.send_error_json("password must be at least 8 characters")
|
||||||
|
|
||||||
role = body.get('role', 'member')
|
role = body.get('role', 'member')
|
||||||
if role not in ('admin', 'member'):
|
# 'bot' is a deliberately-provisioned agent service account (kept out of the invite UI's
|
||||||
return self.send_error_json("role must be admin or member")
|
# member/admin dropdown) — authenticated but never an admin. See require_bot_or_admin.
|
||||||
|
if role not in ('admin', 'member', 'bot'):
|
||||||
|
return self.send_error_json("role must be admin, member, or bot")
|
||||||
|
|
||||||
conn = get_db()
|
conn = get_db()
|
||||||
existing = conn.execute(
|
existing = conn.execute(
|
||||||
@@ -4417,9 +4487,9 @@ class CRMHandler(BaseHTTPRequestHandler):
|
|||||||
|
|
||||||
if 'role' in body:
|
if 'role' in body:
|
||||||
role = str(body.get('role'))
|
role = str(body.get('role'))
|
||||||
if role not in ('admin', 'member'):
|
if role not in ('admin', 'member', 'bot'):
|
||||||
conn.close()
|
conn.close()
|
||||||
return self.send_error_json("role must be admin or member")
|
return self.send_error_json("role must be admin, member, or bot")
|
||||||
sets.append("role = ?")
|
sets.append("role = ?")
|
||||||
args.append(role)
|
args.append(role)
|
||||||
|
|
||||||
@@ -5728,8 +5798,14 @@ def list_email_activity_proposals(conn, status="pending", limit=200):
|
|||||||
return []
|
return []
|
||||||
|
|
||||||
|
|
||||||
def decide_email_activity_proposal(conn, proposal_id, decision, user_id, edited_note=None):
|
def decide_email_activity_proposal(conn, proposal_id, decision, user_id, edited_note=None,
|
||||||
"""Approve (optionally with an edited note -> append to grid) or dismiss a proposal."""
|
source="crm_ui", close_matrix=False):
|
||||||
|
"""Approve (optionally with an edited note -> append to grid) or dismiss a proposal.
|
||||||
|
|
||||||
|
`source` records the channel in the audit log ('crm_ui' for the web panel, 'matrix' for an
|
||||||
|
in-thread approval relayed by the review bot). `close_matrix` is set by the bot when the
|
||||||
|
decision was made in-thread: it stamps the Matrix side row closed in the same transaction so
|
||||||
|
the web->Matrix close path (list_bot_email_proposals.to_close) won't re-announce it."""
|
||||||
p = conn.execute("SELECT * FROM email_activity_proposals WHERE id=?", (proposal_id,)).fetchone()
|
p = conn.execute("SELECT * FROM email_activity_proposals WHERE id=?", (proposal_id,)).fetchone()
|
||||||
if not p:
|
if not p:
|
||||||
return {"error": "not_found"}
|
return {"error": "not_found"}
|
||||||
@@ -5747,15 +5823,85 @@ def decide_email_activity_proposal(conn, proposal_id, decision, user_id, edited_
|
|||||||
action, result = "email.activity_dismissed", {"status": "dismissed"}
|
action, result = "email.activity_dismissed", {"status": "dismissed"}
|
||||||
else:
|
else:
|
||||||
return {"error": "bad_decision"}
|
return {"error": "bad_decision"}
|
||||||
|
if close_matrix:
|
||||||
|
_mark_proposal_matrix(conn, proposal_id, closed_at=now())
|
||||||
conn.execute(
|
conn.execute(
|
||||||
"INSERT INTO interaction_log (id, ts, actor_type, actor_id, action, target_type, target_id, payload, source, created_at) "
|
"INSERT INTO interaction_log (id, ts, actor_type, actor_id, action, target_type, target_id, payload, source, created_at) "
|
||||||
"VALUES (?,?,?,?,?,?,?,?,?,?)",
|
"VALUES (?,?,?,?,?,?,?,?,?,?)",
|
||||||
(generate_id(), now(), "human", user_id, action, "fundraising_investor", p["investor_id"],
|
(generate_id(), now(), "human", user_id, action, "fundraising_investor", p["investor_id"],
|
||||||
json.dumps({"proposal_id": proposal_id}), "crm_ui", now()))
|
json.dumps({"proposal_id": proposal_id}), source, now()))
|
||||||
conn.commit()
|
conn.commit()
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ─── Matrix review-bot bridge for email-activity proposals (Features 2/3) ──────
|
||||||
|
# The CRM (box, stdlib, no matrix-nio) can't post to Matrix, so the intake bot (Spark) PULLS
|
||||||
|
# pending proposals via list_bot_email_proposals, posts a review card to the dedicated review
|
||||||
|
# room, and writes the thread-root event_id back here. State lives CRM-side (email_proposal_matrix)
|
||||||
|
# so both surfaces stay in sync and it survives a bot restart. All queries degrade to empty when
|
||||||
|
# the email integration tables are absent (OperationalError), mirroring list_email_activity_proposals.
|
||||||
|
_BOT_PROPOSAL_COLS = (
|
||||||
|
"SELECT p.id, p.investor_name, p.direction, p.summary, p.proposed_note, "
|
||||||
|
"p.email_subject, p.email_date, e.from_name, e.from_email, e.snippet, "
|
||||||
|
"m.event_id AS event_id, p.status AS status "
|
||||||
|
"FROM email_activity_proposals p "
|
||||||
|
"LEFT JOIN email_proposal_matrix m ON m.proposal_id = p.id "
|
||||||
|
"LEFT JOIN emails e ON e.id = p.email_id ")
|
||||||
|
|
||||||
|
|
||||||
|
def list_bot_email_proposals(conn, limit=100):
|
||||||
|
"""The three work-lists the Matrix review bot polls:
|
||||||
|
to_post — pending, not yet posted to Matrix -> bot posts a review card.
|
||||||
|
open — pending, posted, not closed -> live threads; the bot rebuilds its
|
||||||
|
event_id->proposal routing map from these after a restart.
|
||||||
|
to_close — decided on the WEB while a thread is open -> bot announces it in-thread, closes.
|
||||||
|
Each item carries the card content (investor, direction, subject, date, from, snippet, note)."""
|
||||||
|
try:
|
||||||
|
to_post = [dict(r) for r in conn.execute(
|
||||||
|
_BOT_PROPOSAL_COLS + "WHERE p.status='pending' AND (m.proposal_id IS NULL OR m.posted_at IS NULL) "
|
||||||
|
"ORDER BY p.email_date ASC, p.created_at ASC LIMIT ?", (limit,))]
|
||||||
|
open_threads = [dict(r) for r in conn.execute(
|
||||||
|
_BOT_PROPOSAL_COLS + "WHERE p.status='pending' AND m.posted_at IS NOT NULL AND m.closed_at IS NULL "
|
||||||
|
"ORDER BY p.email_date ASC, p.created_at ASC LIMIT ?", (limit,))]
|
||||||
|
to_close = [dict(r) for r in conn.execute(
|
||||||
|
_BOT_PROPOSAL_COLS + "WHERE p.status!='pending' AND m.posted_at IS NOT NULL AND m.closed_at IS NULL "
|
||||||
|
"ORDER BY p.decided_at ASC LIMIT ?", (limit,))]
|
||||||
|
except sqlite3.OperationalError:
|
||||||
|
return {"to_post": [], "open": [], "to_close": []}
|
||||||
|
return {"to_post": to_post, "open": open_threads, "to_close": to_close}
|
||||||
|
|
||||||
|
|
||||||
|
def _mark_proposal_matrix(conn, proposal_id, *, event_id=None, posted_at=None, closed_at=None):
|
||||||
|
"""Idempotent upsert of the 1:1 Matrix side row. Only the passed fields are written."""
|
||||||
|
cols, vals, sets = ["proposal_id"], [proposal_id], []
|
||||||
|
for name, val in (("event_id", event_id), ("posted_at", posted_at), ("closed_at", closed_at)):
|
||||||
|
if val is not None:
|
||||||
|
cols.append(name); vals.append(val); sets.append(f"{name}=excluded.{name}")
|
||||||
|
placeholders = ",".join("?" for _ in cols)
|
||||||
|
sql = f"INSERT INTO email_proposal_matrix ({','.join(cols)}) VALUES ({placeholders})"
|
||||||
|
if sets:
|
||||||
|
sql += " ON CONFLICT(proposal_id) DO UPDATE SET " + ",".join(sets)
|
||||||
|
conn.execute(sql, vals)
|
||||||
|
|
||||||
|
|
||||||
|
def mark_proposal_matrix_posted(conn, proposal_id, event_id):
|
||||||
|
"""Record that the bot posted a review card (thread root = event_id)."""
|
||||||
|
if not conn.execute("SELECT 1 FROM email_activity_proposals WHERE id=?", (proposal_id,)).fetchone():
|
||||||
|
return {"error": "not_found"}
|
||||||
|
_mark_proposal_matrix(conn, proposal_id, event_id=event_id, posted_at=now())
|
||||||
|
conn.commit()
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
|
def mark_proposal_matrix_closed(conn, proposal_id):
|
||||||
|
"""Mark the Matrix review thread resolved (the bot announced a web-side decision)."""
|
||||||
|
if not conn.execute("SELECT 1 FROM email_activity_proposals WHERE id=?", (proposal_id,)).fetchone():
|
||||||
|
return {"error": "not_found"}
|
||||||
|
_mark_proposal_matrix(conn, proposal_id, closed_at=now())
|
||||||
|
conn.commit()
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
# ─── Main Entry Point ────────────────────────────────────────────────────────
|
# ─── Main Entry Point ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
|
|||||||
@@ -0,0 +1,129 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Test the Matrix review-bot bridge for email-activity proposals (Features 2/3):
|
||||||
|
the bot work-lists (to_post / open / to_close), the Matrix side-row mark helpers, and an
|
||||||
|
in-thread (source='matrix') decision that closes the thread — plus the bot-or-admin role gate.
|
||||||
|
Synthetic data only (guardrail #9). The local model is stubbed.
|
||||||
|
Run: cd backend && python3 test_email_proposal_matrix.py
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sqlite3
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
os.environ["CRM_DB_PATH"] = os.path.join(tempfile.mkdtemp(), "crm.db")
|
||||||
|
os.environ.setdefault("CRM_DATA_DIR", os.path.dirname(os.environ["CRM_DB_PATH"]))
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
import server # noqa: E402
|
||||||
|
|
||||||
|
server._summarize_email_gist = lambda subject, body: "fundraising update; proposed a call"
|
||||||
|
|
||||||
|
FAILS = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(cond, msg):
|
||||||
|
print((" PASS " if cond else " FAIL ") + msg)
|
||||||
|
if not cond:
|
||||||
|
FAILS.append(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def setup():
|
||||||
|
conn = sqlite3.connect(os.environ["CRM_DB_PATH"])
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
conn.executescript("""
|
||||||
|
CREATE TABLE app_settings (key TEXT PRIMARY KEY, value_json TEXT, updated_at TEXT);
|
||||||
|
CREATE TABLE email_accounts (id TEXT, email_address TEXT, sync_enabled INT DEFAULT 1, sync_status TEXT, backfill_complete INT);
|
||||||
|
CREATE TABLE emails (id TEXT PRIMARY KEY, subject TEXT, body_text TEXT, snippet TEXT, from_name TEXT, from_email TEXT, sent_at TEXT, is_matched INT, match_status TEXT);
|
||||||
|
CREATE TABLE email_investor_links (id TEXT, email_id TEXT, fundraising_investor_id TEXT, organization_id TEXT, contact_id TEXT, match_confidence REAL);
|
||||||
|
CREATE TABLE email_activity_proposals (id TEXT PRIMARY KEY, email_id TEXT UNIQUE, investor_id TEXT, investor_name TEXT,
|
||||||
|
direction TEXT, summary TEXT, proposed_note TEXT, email_subject TEXT, email_date TEXT, status TEXT DEFAULT 'pending',
|
||||||
|
decided_by TEXT, decided_at TEXT, final_note TEXT, created_at TEXT);
|
||||||
|
CREATE TABLE email_proposal_matrix (proposal_id TEXT PRIMARY KEY, event_id TEXT, posted_at TEXT, closed_at TEXT, created_at TEXT);
|
||||||
|
CREATE TABLE users (id TEXT PRIMARY KEY, username TEXT);
|
||||||
|
CREATE TABLE fundraising_investors (id TEXT PRIMARY KEY, investor_name TEXT, notes TEXT);
|
||||||
|
CREATE TABLE fundraising_state (id TEXT PRIMARY KEY, grid_json TEXT, views_json TEXT, version INT,
|
||||||
|
updated_by TEXT REFERENCES users(id), updated_at TEXT);
|
||||||
|
CREATE TABLE interaction_log (id TEXT PRIMARY KEY, ts TEXT, actor_type TEXT, actor_id TEXT, action TEXT, target_type TEXT, target_id TEXT, payload TEXT, source TEXT, created_at TEXT);
|
||||||
|
""")
|
||||||
|
conn.execute("INSERT INTO users (id,username) VALUES ('user-1','grant')")
|
||||||
|
conn.execute("INSERT INTO app_settings VALUES ('email_activity_since', ?, ?)", (json.dumps("2026-01-01T00:00:00"), "x"))
|
||||||
|
conn.execute("INSERT INTO email_accounts (id,email_address) VALUES ('a','grant@ten31.xyz')")
|
||||||
|
conn.execute("INSERT INTO fundraising_investors (id,investor_name,notes) VALUES ('inv1','Harbor & Vine','existing note')")
|
||||||
|
grid = {"columns": [], "rows": [{"id": "inv1", "investor_name": "Harbor & Vine", "notes": "existing note"}]}
|
||||||
|
conn.execute("INSERT INTO fundraising_state (id,grid_json,views_json,version) VALUES ('main',?,?,1)", (json.dumps(grid), "[]"))
|
||||||
|
conn.executemany("INSERT INTO emails (id,subject,body_text,snippet,from_name,from_email,sent_at,is_matched,match_status) VALUES (?,?,?,?,?,?,?,1,'matched')", [
|
||||||
|
("e1", "Fund III", "Here is the update", "the quarterly update is attached", "Grant", "grant@ten31.xyz", "2026-06-01T10:00:00"),
|
||||||
|
("e2", "Re: Fund III", "Thanks, a question", "thanks — one question on terms", "LP Contact", "lp@harborvine.example", "2026-06-02T10:00:00"),
|
||||||
|
])
|
||||||
|
conn.executemany("INSERT INTO email_investor_links (id,email_id,fundraising_investor_id,match_confidence) VALUES (?,?, 'inv1', 1.0)",
|
||||||
|
[("l1", "e1"), ("l2", "e2")])
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
setup()
|
||||||
|
|
||||||
|
# role gate: bot passes the agent gate but is NOT an admin; member passes neither.
|
||||||
|
check(server.require_bot_or_admin({"role": "bot"}), "bot passes require_bot_or_admin")
|
||||||
|
check(server.require_bot_or_admin({"role": "admin"}), "admin passes require_bot_or_admin")
|
||||||
|
check(not server.require_bot_or_admin({"role": "member"}), "member does NOT pass require_bot_or_admin")
|
||||||
|
check(not server.require_admin({"role": "bot"}), "bot is NOT an admin (no user-mgmt/settings reach)")
|
||||||
|
|
||||||
|
check(server.propose_email_activity_notes().get("proposed") == 2, "drafts 2 proposals")
|
||||||
|
conn = server.get_db()
|
||||||
|
props = server.list_email_activity_proposals(conn, status="pending")
|
||||||
|
by_email = {p["email_id"]: p for p in props}
|
||||||
|
p_a, p_b = by_email["e1"], by_email["e2"]
|
||||||
|
|
||||||
|
# Both are pending + un-posted → both in to_post; card carries from/snippet/note context.
|
||||||
|
lists = server.list_bot_email_proposals(conn)
|
||||||
|
check(len(lists["to_post"]) == 2 and not lists["open"] and not lists["to_close"], "both proposals queued to_post")
|
||||||
|
card = next(it for it in lists["to_post"] if it["id"] == p_a["id"])
|
||||||
|
check(card.get("from_name") == "Grant" and "quarterly update" in (card.get("snippet") or ""), "card carries from_name + snippet")
|
||||||
|
check("✉" in (card.get("proposed_note") or ""), "card carries the drafted note")
|
||||||
|
|
||||||
|
# Post p_a to Matrix → it leaves to_post and becomes an open thread (event id recorded).
|
||||||
|
server.mark_proposal_matrix_posted(conn, p_a["id"], "evtA")
|
||||||
|
lists = server.list_bot_email_proposals(conn)
|
||||||
|
check(len(lists["to_post"]) == 1 and lists["to_post"][0]["id"] == p_b["id"], "posting p_a leaves only p_b to_post")
|
||||||
|
check(len(lists["open"]) == 1 and lists["open"][0]["id"] == p_a["id"] and lists["open"][0]["event_id"] == "evtA",
|
||||||
|
"posted p_a is an open thread carrying its event id")
|
||||||
|
|
||||||
|
# Decide p_a IN-THREAD on Matrix (approve + close in one transaction).
|
||||||
|
r = server.decide_email_activity_proposal(conn, p_a["id"], "approve", "user-1", source="matrix", close_matrix=True)
|
||||||
|
check(r.get("status") == "approved" and r.get("placed_in_grid") is True, "matrix approve appends to the grid")
|
||||||
|
lists = server.list_bot_email_proposals(conn)
|
||||||
|
check(not any(it["id"] == p_a["id"] for it in lists["open"] + lists["to_close"]),
|
||||||
|
"matrix-decided proposal is closed (not re-announced via to_close)")
|
||||||
|
src = conn.execute("SELECT source FROM interaction_log WHERE action='email.activity_approved'").fetchone()["source"]
|
||||||
|
check(src == "matrix", "matrix decision is audited source='matrix'")
|
||||||
|
|
||||||
|
# Web-decide path: post p_b, then dismiss it on the WEB (default source, no close) → the bot
|
||||||
|
# must see it in to_close so it can announce the web decision in-thread, then close.
|
||||||
|
server.mark_proposal_matrix_posted(conn, p_b["id"], "evtB")
|
||||||
|
server.decide_email_activity_proposal(conn, p_b["id"], "dismiss", "user-1") # web path: source crm_ui, no close
|
||||||
|
lists = server.list_bot_email_proposals(conn)
|
||||||
|
check(len(lists["to_close"]) == 1 and lists["to_close"][0]["id"] == p_b["id"] and lists["to_close"][0]["status"] == "dismissed",
|
||||||
|
"web-decided open thread surfaces in to_close")
|
||||||
|
src2 = conn.execute("SELECT source FROM interaction_log WHERE action='email.activity_dismissed'").fetchone()["source"]
|
||||||
|
check(src2 == "crm_ui", "web decision is audited source='crm_ui'")
|
||||||
|
|
||||||
|
server.mark_proposal_matrix_closed(conn, p_b["id"])
|
||||||
|
lists = server.list_bot_email_proposals(conn)
|
||||||
|
check(not lists["to_close"] and not lists["open"], "closing the thread clears the work-lists")
|
||||||
|
|
||||||
|
# Marking a non-existent proposal is a clean not_found, not a crash.
|
||||||
|
check(server.mark_proposal_matrix_posted(conn, "nope", "evtX").get("error") == "not_found", "mark posted on unknown id -> not_found")
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if FAILS:
|
||||||
|
print(f"\nFAILED ({len(FAILS)})")
|
||||||
|
for f in FAILS:
|
||||||
|
print(" - " + f)
|
||||||
|
sys.exit(1)
|
||||||
|
print("\nALL PASS (email-proposal Matrix bridge)")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -12,7 +12,7 @@ Read this before editing Gmail capture or draft creation.
|
|||||||
## What it does
|
## What it does
|
||||||
|
|
||||||
- `backend/email_integration/` captures Gmail via **domain-wide delegation** (`credentials.py`, `matcher.py`, `parser.py`, `db.py`, `sync.py`, `scheduler.py`, `routes.py`) and creates Tier-B in-thread drafts (`compose.py`). It has its own `migrations/`.
|
- `backend/email_integration/` captures Gmail via **domain-wide delegation** (`credentials.py`, `matcher.py`, `parser.py`, `db.py`, `sync.py`, `scheduler.py`, `routes.py`) and creates Tier-B in-thread drafts (`compose.py`). It has its own `migrations/`.
|
||||||
- Captured email becomes CRM activity through a **propose → approve** flow — nothing lands on a contact record until a human approves the proposal.
|
- Captured email becomes CRM activity through a **propose → approve** flow — nothing lands on a contact record until a human approves the proposal. The proposed grid notes show on the **Email Capture** page (admin-only): each card has a **View email** toggle that fetches `GET /api/email/detail?id=` and shows the source email inline (from/to/cc/date/subject + scrollable body) so you can judge the note against it. The same proposals can also be reviewed/approved/edited from a **dedicated Matrix room**, kept in sync with this panel (decide on either surface; the other reflects it) — that CRM→Matrix bridge lives in the **review bot**, see `docs/guides/matrix-intake.md`. The proposal model itself (`email_activity_proposals` + the `propose_email_activity_notes` drafter + the decide path) lives in `backend/server.py`, not this package.
|
||||||
|
|
||||||
## Hard rule
|
## Hard rule
|
||||||
|
|
||||||
|
|||||||
@@ -90,6 +90,42 @@ rows ≥ `min_score` (0.62), ranked, capped at 5:
|
|||||||
only the shortlist, never the whole LP list — intentionally NOT built in this pass, because the
|
only the shortlist, never the whole LP list — intentionally NOT built in this pass, because the
|
||||||
deterministic filter already surfaces every duplicate the human then resolves.
|
deterministic filter already surfaces every duplicate the human then resolves.
|
||||||
|
|
||||||
|
## Email-activity proposal review (the CRM→Matrix bridge, v0.1.0:89)
|
||||||
|
|
||||||
|
A second, separate flow runs alongside intake: reviewing the **proposed grid notes** the CRM
|
||||||
|
drafts from newly-matched email (`server.propose_email_activity_notes`, surfaced on the web Email
|
||||||
|
Capture panel). The bot lets the team approve/dismiss/edit those on mobile, kept **in sync** with
|
||||||
|
the web panel. The CRM (box, stdlib, no matrix-nio) can't post to Matrix, so the bot **pulls**.
|
||||||
|
|
||||||
|
- **Dedicated room** (`MATRIX_EMAIL_REVIEW_ROOM`, see *Config*) — separate from the intake room
|
||||||
|
so high-volume email proposals don't drown the conversational intake. Unset → the whole leg is
|
||||||
|
off (the bot just does intake). The bot must be a **member** of this room.
|
||||||
|
- **Poll loop** (`bot.poll_email_proposals`, every `EMAIL_POLL_SEC`=20s) calls `crm_client.
|
||||||
|
list_email_proposals` → `GET /api/intake/email-proposals`, which returns three work-lists:
|
||||||
|
- **to_post** — pending, not yet posted → the bot posts a review card (metadata + a short email
|
||||||
|
**snippet** + the drafted note; the full body is the web popup's job, kept compact for mobile),
|
||||||
|
then records the thread-root event id via `POST .../{id}/matrix {event_id}`.
|
||||||
|
- **open** — pending, posted, not closed → the bot rebuilds its `event_id → proposal` routing map
|
||||||
|
from these on **every poll**, so replies still route **after a bot restart** (unlike intake's
|
||||||
|
in-memory-only store — the state lives CRM-side in `email_proposal_matrix`).
|
||||||
|
- **to_close** — decided on the **web** while a thread was open → the bot posts a "decided on the
|
||||||
|
web — thread closed" line and `POST .../{id}/matrix {closed:true}`.
|
||||||
|
- **In-thread replies** (`bot.handle_email_reply`, `email_proposals.interpret`): `yes` →
|
||||||
|
`POST .../{id}/decide {decision:"approve", note}` (appends the note to the grid, source='matrix',
|
||||||
|
closes the thread atomically); `no` → dismiss; **anything else → NL revision of the note** via
|
||||||
|
local Qwen (`email_proposals.revise_note`, no Claude/scrub) — re-rendered for re-approval, so the
|
||||||
|
draft→approve gate holds. A no-op/empty revision re-prompts instead of saying "Updated".
|
||||||
|
- **Two surfaces, one source of truth.** Decide on the web → the bot announces + closes the thread;
|
||||||
|
decide on Matrix → the web panel polls `/api/activity/proposals` (~25s) and the card clears.
|
||||||
|
`email_proposal_matrix` (1:1 side row, migration `0003`) carries `event_id`/`posted_at`/`closed_at`;
|
||||||
|
a matrix decision sets `closed_at` in the same txn so it's never re-announced via `to_close`.
|
||||||
|
- **Pure logic is `email_proposals.py`** (card render, reply grammar, note revision) — unit-tested
|
||||||
|
offline in `test_email_proposals.py`; the async poll/post wiring is in `bot.py` (live-smoke only).
|
||||||
|
- **Known minors (low-likelihood, ~5-person team):** if the CRM is unreachable *between* posting a
|
||||||
|
card and recording its event id, the next poll re-posts a duplicate card (the orphan's replies
|
||||||
|
won't route — re-send/decide the recorded one). A mid-revise bot restart loses the in-memory
|
||||||
|
revised note (rebuilt from `open` = the original `proposed_note`; still a valid proposal).
|
||||||
|
|
||||||
## Rules / gotchas
|
## Rules / gotchas
|
||||||
|
|
||||||
- **Module-name collision:** the intake config module is `settings.py`, **not** `config.py`,
|
- **Module-name collision:** the intake config module is `settings.py`, **not** `config.py`,
|
||||||
@@ -161,7 +197,11 @@ rows ≥ `min_score` (0.62), ranked, capped at 5:
|
|||||||
- **Server-side endpoints ship in the s9pk, not the bot.** `GET /api/intake/match` and the
|
- **Server-side endpoints ship in the s9pk, not the bot.** `GET /api/intake/match` and the
|
||||||
`source` provenance on `log-communication` live in `backend/server.py`, so they reach the box
|
`source` provenance on `log-communication` live in `backend/server.py`, so they reach the box
|
||||||
only via an **s9pk build + install** — a bot restart won't deliver them. (Missed in v83: the
|
only via an **s9pk build + install** — a bot restart won't deliver them. (Missed in v83: the
|
||||||
box 404'd `/api/intake/match` until **v0.1.0:84**.)
|
box 404'd `/api/intake/match` until **v0.1.0:84**.) **Same split for the email-review bridge
|
||||||
|
(v0.1.0:89):** the `/api/intake/email-proposals*` endpoints + the `email_proposal_matrix`
|
||||||
|
migration (`0003`) + the `bot` role ship in the **s9pk**; the poll loop + review-room handling
|
||||||
|
ship on the **Spark** (git pull + restart). A bot restart against a pre-v89 box returns nothing
|
||||||
|
useful (404/empty), so install the s9pk first, then set the bot user's role + the review room.
|
||||||
- **`CRM_API_BASE` is the box over the LAN, not localhost** (bot on the Spark, CRM on the box).
|
- **`CRM_API_BASE` is the box over the LAN, not localhost** (bot on the Spark, CRM on the box).
|
||||||
`https://immense-voyage.local` (443) is the **StartOS dashboard**, not the CRM — the CRM has
|
`https://immense-voyage.local` (443) is the **StartOS dashboard**, not the CRM — the CRM has
|
||||||
its own interface address (the URL you open in a browser); container port 8080 isn't
|
its own interface address (the URL you open in a browser); container port 8080 isn't
|
||||||
@@ -174,6 +214,19 @@ All in `.env` (names in `.env.example`): `MATRIX_HOMESERVER`, `MATRIX_USER`,
|
|||||||
`CRM_BOT_USERNAME`, `CRM_BOT_PASSWORD`, `CRM_API_VERIFY_TLS`. Spark settings are inherited from
|
`CRM_BOT_USERNAME`, `CRM_BOT_PASSWORD`, `CRM_API_VERIFY_TLS`. Spark settings are inherited from
|
||||||
the ingest client (`SPARK_CONTROL_URL`, `CRM_CHAT_MODEL`).
|
the ingest client (`SPARK_CONTROL_URL`, `CRM_CHAT_MODEL`).
|
||||||
|
|
||||||
|
- **`MATRIX_EMAIL_REVIEW_ROOM`** (optional) — the dedicated room for the email-activity proposal
|
||||||
|
review leg (above). Unset/empty disables that leg entirely (the bot does intake only). The bot
|
||||||
|
must be invited to + joined in this room. Read once at startup, like the room/roster.
|
||||||
|
- **Bot CRM user needs role `bot`.** The email-proposal endpoints (`/api/intake/email-proposals*`)
|
||||||
|
are gated to `require_bot_or_admin` because they expose LP email content (the proposals are
|
||||||
|
admin-only on the web). The `bot` role is **authenticated-but-not-admin** — it passes these
|
||||||
|
endpoints + the auth-only ones the bot already uses (login, `/api/intake/match`,
|
||||||
|
`log-communication`), but **never** `require_admin` (no user-management/settings/security reach).
|
||||||
|
One-time flip of the existing service account (kept out of the invite UI's member/admin dropdown
|
||||||
|
— provision deliberately): an admin `PATCH /api/users/<id> {"role":"bot"}`, or on the box
|
||||||
|
`UPDATE users SET role='bot' WHERE username='<CRM_BOT_USERNAME>';`. Role controls *reach*; the
|
||||||
|
draft→approve gate (a human still approves every write) controls *autonomy* — two separate axes.
|
||||||
|
|
||||||
- **`INTAKE_TEAM_ROSTER`** (optional, comma-separated) — Ten31 team-member names that frame the
|
- **`INTAKE_TEAM_ROSTER`** (optional, comma-separated) — Ten31 team-member names that frame the
|
||||||
parse (see *Flow* step 1). Use the **first names as actually typed in the room** ("Grant,
|
parse (see *Flow* step 1). Use the **first names as actually typed in the room** ("Grant,
|
||||||
Jonathan, …"). Read once at startup by `settings.team_roster()`, so **a roster change needs a
|
Jonathan, …"). Read once at startup by `settings.team_roster()`, so **a roster change needs a
|
||||||
|
|||||||
@@ -10037,6 +10037,8 @@
|
|||||||
const [proposals, setProposals] = useState([]);
|
const [proposals, setProposals] = useState([]);
|
||||||
const [edits, setEdits] = useState({});
|
const [edits, setEdits] = useState({});
|
||||||
const [deciding, setDeciding] = useState(null);
|
const [deciding, setDeciding] = useState(null);
|
||||||
|
const [openEmail, setOpenEmail] = useState(null); // proposal id whose source email is expanded
|
||||||
|
const [emailCache, setEmailCache] = useState({}); // email_id -> {loading, data, error}
|
||||||
|
|
||||||
const load = useCallback(async () => {
|
const load = useCallback(async () => {
|
||||||
let s;
|
let s;
|
||||||
@@ -10086,6 +10088,21 @@
|
|||||||
return () => { cancelled = true; clearInterval(iv); };
|
return () => { cancelled = true; clearInterval(iv); };
|
||||||
}, [backfilling, load]);
|
}, [backfilling, load]);
|
||||||
|
|
||||||
|
// Steady-state poll of just the proposals so a decision made on Matrix (approve/dismiss
|
||||||
|
// in the review room) clears its card here without a manual reload — the mirror of the
|
||||||
|
// bot announcing a web-side decision in-thread. Admin-only (only admins see proposals).
|
||||||
|
const refreshProposals = useCallback(async () => {
|
||||||
|
try {
|
||||||
|
const pr = await api('/api/activity/proposals', {}, token);
|
||||||
|
setProposals(Array.isArray(pr?.proposals) ? pr.proposals : []);
|
||||||
|
} catch (_) { /* admin-only / transient — leave the current list */ }
|
||||||
|
}, [token]);
|
||||||
|
useEffect(() => {
|
||||||
|
if (!isAdmin) return undefined;
|
||||||
|
const iv = setInterval(() => { refreshProposals(); }, 25000);
|
||||||
|
return () => clearInterval(iv);
|
||||||
|
}, [isAdmin, refreshProposals]);
|
||||||
|
|
||||||
const runAction = async (key, endpoint, successMsg, confirmMsg, body) => {
|
const runAction = async (key, endpoint, successMsg, confirmMsg, body) => {
|
||||||
if (busy) return;
|
if (busy) return;
|
||||||
if (confirmMsg && !window.confirm(confirmMsg)) return;
|
if (confirmMsg && !window.confirm(confirmMsg)) return;
|
||||||
@@ -10120,6 +10137,51 @@
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// Click a proposal to see the email it was drafted from (from/to/cc/date/subject +
|
||||||
|
// scrollable body) so you can judge whether the note is right. Lazily fetched +
|
||||||
|
// cached per email; reuses the admin-only /api/email/detail used by Communications.
|
||||||
|
const toggleEmail = async (p) => {
|
||||||
|
if (openEmail === p.id) { setOpenEmail(null); return; }
|
||||||
|
setOpenEmail(p.id);
|
||||||
|
const eid = p.email_id;
|
||||||
|
if (!eid || emailCache[eid]) return;
|
||||||
|
setEmailCache((c) => ({ ...c, [eid]: { loading: true } }));
|
||||||
|
try {
|
||||||
|
const res = await api(`/api/email/detail?id=${encodeURIComponent(eid)}`, {}, token);
|
||||||
|
setEmailCache((c) => ({ ...c, [eid]: { loading: false, data: res } }));
|
||||||
|
} catch (err) {
|
||||||
|
setEmailCache((c) => ({ ...c, [eid]: { loading: false, error: getErrorMessage(err, 'Failed to load email') } }));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const renderProposalEmail = (p) => {
|
||||||
|
const det = emailCache[p.email_id];
|
||||||
|
if (!det) return null;
|
||||||
|
if (det.loading) return <div style={{ marginTop: '8px' }}><SkeletonBlock lines={4} /></div>;
|
||||||
|
if (det.error) return <div className="toast error" style={{ position: 'static', marginTop: '8px' }}>{det.error}</div>;
|
||||||
|
const d = det.data || {};
|
||||||
|
const rcpt = (kind) => (d.recipients || []).filter((r) => r.kind === kind)
|
||||||
|
.map((r) => r.display_name ? `${r.display_name} <${r.address}>` : r.address).join(', ');
|
||||||
|
const to = rcpt('to'), cc = rcpt('cc');
|
||||||
|
const from = d.from_name ? `${d.from_name} <${d.from_email || ''}>` : (d.from_email || '—');
|
||||||
|
const lbl = { fontSize: '11px', color: '#8ea2b7' };
|
||||||
|
return (
|
||||||
|
<div style={{ marginTop: '8px', border: '1px solid #263548', borderRadius: '6px', background: '#0d1622', padding: '10px' }}>
|
||||||
|
<div style={lbl}><b>From:</b> {from}</div>
|
||||||
|
{to && <div style={lbl}><b>To:</b> {to}</div>}
|
||||||
|
{cc && <div style={lbl}><b>Cc:</b> {cc}</div>}
|
||||||
|
<div style={lbl}><b>Date:</b> {d.sent_at ? new Date(d.sent_at).toLocaleString() : '—'}</div>
|
||||||
|
<div style={lbl}><b>Subject:</b> {d.subject || '(no subject)'}</div>
|
||||||
|
{(d.attachments || []).length > 0 && (
|
||||||
|
<div style={{ ...lbl, marginTop: '2px' }}><b>Attachments:</b> {d.attachments.map((a) => a.filename).join(', ')}</div>
|
||||||
|
)}
|
||||||
|
<pre style={{ margin: '8px 0 0', maxHeight: '280px', overflowY: 'auto', whiteSpace: 'pre-wrap', wordBreak: 'break-word', fontSize: '12px', lineHeight: 1.5, color: '#cdd9e5', fontFamily: 'inherit' }}>
|
||||||
|
{d.body_text || (d.has_html ? '(HTML-only email — open in Gmail to view formatting)' : '(no body captured)')}
|
||||||
|
</pre>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
if (loading) return <div style={{ padding: '20px' }}><SkeletonBlock lines={8} /></div>;
|
if (loading) return <div style={{ padding: '20px' }}><SkeletonBlock lines={8} /></div>;
|
||||||
if (error) return <div className="toast error" style={{ position: 'static' }}>{error}</div>;
|
if (error) return <div className="toast error" style={{ position: 'static' }}>{error}</div>;
|
||||||
if (!status) return <div className="empty-state">No data</div>;
|
if (!status) return <div className="empty-state">No data</div>;
|
||||||
@@ -10204,7 +10266,16 @@
|
|||||||
{deciding === p.id ? 'Adding…' : 'Approve & add to grid'}
|
{deciding === p.id ? 'Adding…' : 'Approve & add to grid'}
|
||||||
</button>
|
</button>
|
||||||
<button onClick={() => decide(p, 'dismiss')} disabled={deciding === p.id}>Dismiss</button>
|
<button onClick={() => decide(p, 'dismiss')} disabled={deciding === p.id}>Dismiss</button>
|
||||||
|
{p.email_id && (
|
||||||
|
<button
|
||||||
|
onClick={() => toggleEmail(p)}
|
||||||
|
style={{ background: 'transparent', border: '1px solid #263548', color: '#8ea2b7' }}
|
||||||
|
>
|
||||||
|
{openEmail === p.id ? 'Hide email' : 'View email'}
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
{openEmail === p.id && renderProposalEmail(p)}
|
||||||
</div>
|
</div>
|
||||||
))}
|
))}
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -53,8 +53,9 @@ export const PACKAGE_TITLE = 'Ten31 Database'
|
|||||||
// * 0.1.0:85 (cosmetic: drop the redundant "[note]" tag from the fundraising-grid note line — now "YYYY-MM-DD Contact: summary"; informative comm types [call, meeting, …] keep their "[type]" tag; shared by the Matrix intake bot + grid-UI logging; no schema change)
|
// * 0.1.0:85 (cosmetic: drop the redundant "[note]" tag from the fundraising-grid note line — now "YYYY-MM-DD Contact: summary"; informative comm types [call, meeting, …] keep their "[type]" tag; shared by the Matrix intake bot + grid-UI logging; no schema change)
|
||||||
// * 0.1.0:86 (Matrix intake fuzzy matching: GET /api/intake/match now returns ranked `candidates` [fuzzy near-matches — deterministic difflib name similarity + token overlap + email edit-distance ≤ 2, legal-suffix-aware] alongside the exact `match`, so the bot can surface near-duplicates ["Charlie"/"Charles", "Acme Capital"/"Acme Capital LLC", a one-char email typo] for human confirmation instead of silently creating a second investor; the bot-side disambiguation + conversational-edit UX ships on the Spark, not the s9pk; code-only, no schema change)
|
// * 0.1.0:86 (Matrix intake fuzzy matching: GET /api/intake/match now returns ranked `candidates` [fuzzy near-matches — deterministic difflib name similarity + token overlap + email edit-distance ≤ 2, legal-suffix-aware] alongside the exact `match`, so the bot can surface near-duplicates ["Charlie"/"Charles", "Acme Capital"/"Acme Capital LLC", a one-char email typo] for human confirmation instead of silently creating a second investor; the bot-side disambiguation + conversational-edit UX ships on the Spark, not the s9pk; code-only, no schema change)
|
||||||
// * 0.1.0:87 (Adopt the Pipeline — grid drives the deal board: new "Add to Pipeline" row action creates+links an opportunity via opportunities.fundraising_investor_id [migration 0005, additive], reusing the grid's synced contact [no POST /api/contacts side-door] and mapping the grid lead→owner; idempotent [one live opp/investor, re-link never reseeds board-owned stage/probability]; read-only Pipeline + Pipeline Stage grid columns derived live from the linked opp; "Remove from Pipeline" soft-deletes the opp [grid row untouched]; deleting a grid investor archives its orphaned opp; folds in the soft-delete fix for the pipeline report + dashboard aggregates [archived opps no longer counted])
|
// * 0.1.0:87 (Adopt the Pipeline — grid drives the deal board: new "Add to Pipeline" row action creates+links an opportunity via opportunities.fundraising_investor_id [migration 0005, additive], reusing the grid's synced contact [no POST /api/contacts side-door] and mapping the grid lead→owner; idempotent [one live opp/investor, re-link never reseeds board-owned stage/probability]; read-only Pipeline + Pipeline Stage grid columns derived live from the linked opp; "Remove from Pipeline" soft-deletes the opp [grid row untouched]; deleting a grid investor archives its orphaned opp; folds in the soft-delete fix for the pipeline report + dashboard aggregates [archived opps no longer counted])
|
||||||
// * Current: 0.1.0:88 (frontend-only: retire the Pipeline page's "+ New Opportunity" button + its create-by-contact modal — opportunities are now born only from a fundraising-grid investor row ["+ Pipeline"], so the board is a view + stage-management surface; replaced the button with a muted "Add deals from the Fundraising Grid" hint; removed the now-dead handler/state + the page's unused /api/contacts fetch)
|
// * 0.1.0:88 (frontend-only: retire the Pipeline page's "+ New Opportunity" button + its create-by-contact modal — opportunities are now born only from a fundraising-grid investor row ["+ Pipeline"], so the board is a view + stage-management surface; replaced the button with a muted "Add deals from the Fundraising Grid" hint; removed the now-dead handler/state + the page's unused /api/contacts fetch)
|
||||||
export const PACKAGE_VERSION = '0.1.0:88'
|
// * Current: 0.1.0:89 (email-proposal review over Matrix + a dedicated agent role: Email Capture's proposed grid notes gain a click-to-view inline popup of the source email [from/to/cc/date/subject/scrollable body, via the existing GET /api/email/detail]; and a CRM→Matrix review bridge — the intake bot [Spark] pulls pending proposals, posts a review card to a dedicated review room [MATRIX_EMAIL_REVIEW_ROOM], and relays in-thread yes/no/NL-edit back to the CRM, with web panel ↔ Matrix kept in sync [decide on either surface; the other reflects it]. New side table email_proposal_matrix [email-integration migration 0003, additive + idempotent] holds per-proposal Matrix thread state; new bot-or-admin endpoints GET /api/intake/email-proposals + .../{id}/matrix + .../{id}/decide, gated by a new 'bot' role [authenticated, never admin]. Bot poll loop + review-room handling ship on the Spark, not the s9pk)
|
||||||
|
export const PACKAGE_VERSION = '0.1.0:89'
|
||||||
|
|
||||||
export const DATA_MOUNT_PATH = '/data'
|
export const DATA_MOUNT_PATH = '/data'
|
||||||
export const WEB_PORT = 8080
|
export const WEB_PORT = 8080
|
||||||
|
|||||||
@@ -49,8 +49,9 @@ import { v_0_1_0_85 } from './v0.1.0.85'
|
|||||||
import { v_0_1_0_86 } from './v0.1.0.86'
|
import { v_0_1_0_86 } from './v0.1.0.86'
|
||||||
import { v_0_1_0_87 } from './v0.1.0.87'
|
import { v_0_1_0_87 } from './v0.1.0.87'
|
||||||
import { v_0_1_0_88 } from './v0.1.0.88'
|
import { v_0_1_0_88 } from './v0.1.0.88'
|
||||||
|
import { v_0_1_0_89 } from './v0.1.0.89'
|
||||||
|
|
||||||
export const versionGraph = VersionGraph.of({
|
export const versionGraph = VersionGraph.of({
|
||||||
current: v_0_1_0_88,
|
current: v_0_1_0_89,
|
||||||
other: [v_0_1_0_39, v_0_1_0_40, v_0_1_0_41, v_0_1_0_42, v_0_1_0_43, v_0_1_0_44, v_0_1_0_45, v_0_1_0_46, v_0_1_0_47, v_0_1_0_48, v_0_1_0_49, v_0_1_0_50, v_0_1_0_51, v_0_1_0_52, v_0_1_0_53, v_0_1_0_54, v_0_1_0_55, v_0_1_0_56, v_0_1_0_57, v_0_1_0_58, v_0_1_0_59, v_0_1_0_60, v_0_1_0_61, v_0_1_0_62, v_0_1_0_63, v_0_1_0_64, v_0_1_0_65, v_0_1_0_66, v_0_1_0_67, v_0_1_0_68, v_0_1_0_69, v_0_1_0_70, v_0_1_0_71, v_0_1_0_72, v_0_1_0_73, v_0_1_0_74, v_0_1_0_75, v_0_1_0_76, v_0_1_0_77, v_0_1_0_78, v_0_1_0_79, v_0_1_0_80, v_0_1_0_81, v_0_1_0_82, v_0_1_0_83, v_0_1_0_84, v_0_1_0_85, v_0_1_0_86, v_0_1_0_87],
|
other: [v_0_1_0_39, v_0_1_0_40, v_0_1_0_41, v_0_1_0_42, v_0_1_0_43, v_0_1_0_44, v_0_1_0_45, v_0_1_0_46, v_0_1_0_47, v_0_1_0_48, v_0_1_0_49, v_0_1_0_50, v_0_1_0_51, v_0_1_0_52, v_0_1_0_53, v_0_1_0_54, v_0_1_0_55, v_0_1_0_56, v_0_1_0_57, v_0_1_0_58, v_0_1_0_59, v_0_1_0_60, v_0_1_0_61, v_0_1_0_62, v_0_1_0_63, v_0_1_0_64, v_0_1_0_65, v_0_1_0_66, v_0_1_0_67, v_0_1_0_68, v_0_1_0_69, v_0_1_0_70, v_0_1_0_71, v_0_1_0_72, v_0_1_0_73, v_0_1_0_74, v_0_1_0_75, v_0_1_0_76, v_0_1_0_77, v_0_1_0_78, v_0_1_0_79, v_0_1_0_80, v_0_1_0_81, v_0_1_0_82, v_0_1_0_83, v_0_1_0_84, v_0_1_0_85, v_0_1_0_86, v_0_1_0_87, v_0_1_0_88],
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -0,0 +1,24 @@
|
|||||||
|
import { VersionInfo } from '@start9labs/start-sdk'
|
||||||
|
|
||||||
|
// Email-proposal review over Matrix + a dedicated agent role. The CRM-drafted "proposed grid
|
||||||
|
// notes" (Email Capture panel) gain (1) a click-to-view inline popup of the source email
|
||||||
|
// (from/to/cc/date/subject/scrollable body, via the existing /api/email/detail) so a reviewer
|
||||||
|
// can judge the note against the email, and (2) a CRM→Matrix review bridge: the intake bot
|
||||||
|
// (Spark) pulls pending proposals, posts a review card to a dedicated review room, and relays
|
||||||
|
// the human's in-thread yes/no/edit back to the CRM — with the web panel and Matrix kept in
|
||||||
|
// sync (decide on either; the other surface reflects it). New side table email_proposal_matrix
|
||||||
|
// (email-integration migration 0003, additive + idempotent — CREATE TABLE IF NOT EXISTS) holds
|
||||||
|
// the per-proposal Matrix thread state. New bot-or-admin endpoints under /api/intake/
|
||||||
|
// email-proposals (list/mark/decide), gated by a new 'bot' role (authenticated, never admin).
|
||||||
|
// The bot's poll loop + review-room handling ship on the Spark (git pull + restart), not here.
|
||||||
|
export const v_0_1_0_89 = VersionInfo.of({
|
||||||
|
version: '0.1.0:89',
|
||||||
|
releaseNotes: {
|
||||||
|
en_US: [
|
||||||
|
'Email Capture: click a proposed grid note to see the source email inline',
|
||||||
|
'(from/to/cc/date/subject + body) before approving, and review/approve/dismiss/edit',
|
||||||
|
'proposals from a dedicated Matrix room on mobile — decisions sync both ways.',
|
||||||
|
].join(' '),
|
||||||
|
},
|
||||||
|
migrations: { up: async () => {}, down: async () => {} },
|
||||||
|
})
|
||||||
Reference in New Issue
Block a user