Redact whole review threads on decision (replies too)

The bot was granted a redact/mod power level in the review room, so it can now
clear a resolved thread entirely, not just the card: redact_thread redacts the
card root then scans recent history for its m.thread replies (the human's yes/no
+ any bot messages) and redacts those too, so decided threads drop out of the
threads view, not only the main timeline. Drops the in-thread confirmation on a
successful decision (the thread clearing is the ack; a confirmation would keep
the thread alive). redact_resolved.py extended to also clear replies of already-
resolved threads for the one-time backfill. Bot-only; no s9pk change.

AGENTS.md

@@ -105,7 +105,7 @@ Subsystem rules live in `docs/guides/` and lazy-load in Claude Code via `.claude
 
 _Phase 0 + Phase 1 built; **box and repo at v0.1.0:90** (v89 + v90 installed & verified live 2026-06-18 — `installed-version` = 0.1.0:90, server up on :8080, no errors; the StartOS version-graph traversal logs an inert down-to-39-then-up-to-90 because the per-version `up`/`down` hooks are no-ops — the real SQLite migrations run in-app at startup). **The fundraising grid + email capture is the canonical system of record** (2026-06-16) — vestigial classic-CRM surfaces get pruned/repurposed. Deploy/feature history lives in git log + `start9/0.4/startos/versions/`; longer-term backlog + debt in `ROADMAP.md` / `EVALUATION.md`._
 
-- **Email-proposal review over Matrix + a `bot` role — DEPLOYED & LIVE 2026-06-18 (box v0.1.0:91), smoke-tested OK.** **Post-smoke refinements (v0.1.0:91 + bot):** (1) every review card/reply is framed with `-----------------------` dash rules so threads don't bleed together on mobile (`email_proposals.frame`); (2) on a conclusive decision (either surface) the bot **redacts the card** (`client.room_redact`) so the room clears down to only undecided items — redacting its *own* card needs no power; wiping the human's reply for a fully-empty thread needs a `redact`/mod power level (not set, optional extension); (3) the proposed note now NAMES who emailed whom — "{teammate} emailed {investor}" / "{sender} emailed the team" — instead of a bare "Sent/Received", and outbound detection now also matches our corporate domain (public providers excluded) so a teammate's non-enrolled `@ten31.xyz` mail no longer reads as "Received". Feature 1 (the inline source-email popup) is live on the box. The Matrix leg is live: bot CRM user flipped to `bot` (via the v90 Settings → Admin dropdown), `MATRIX_EMAIL_REVIEW_ROOM=!CImVJWFmzNxPcCcrZl:matrix.gilliam.ai` set on the Spark `.env`, bot joins the room on startup, polls every 20s. **End-to-end proven on deploy:** the whole pending backlog (N=16) posted as review cards to the room and was marked posted on the box (no errors); the bot now idles (to_post empty) holding 16 open threads. **Deploy gotchas learned:** (a) order matters — the bot caches its JWT (role embedded) at login and only re-logins on 401, not 403, so the role must be `bot` *before* the bot process starts; (b) "invited" ≠ "joined" — the bot must `client.join` the room (added in `2998706`) or `room_send` fails M_FORBIDDEN; (c) on first run the bot posts a card per *pending* proposal — fine here at N=16, but a large backlog would flood + get homeserver-429'd (a `since`-floor on `to_post` is the upgrade path if that ever bites). **Smoke-test still pending (manual):** reply `yes`/`no`/edit **in-thread** on a card → grid note + thread closes; approve one on the web → the bot announces + closes its thread (≤20s); confirm a Matrix decision clears the web card (≤25s). Two asks on the email-capture "proposed grid notes": (1) **inline source-email popup** — each proposed-note card on the Email Capture page gets a **View email** toggle that lazily fetches the existing **`GET /api/email/detail`** and renders from/to/cc/date/subject + scrollable body inline, so you can judge the note against the email (frontend-only, reuses the Communications detail pattern). (2) **CRM→Matrix review bridge** — the CRM (box, no matrix-nio) can't push, so the **intake bot (Spark) PULLS**: new **`GET /api/intake/email-proposals`** returns three work-lists (`to_post`/`open`/`to_close`); the bot posts a review card (metadata + snippet + draft note) to a **dedicated review room** (`MATRIX_EMAIL_REVIEW_ROOM`), records the thread root (`POST .../{id}/matrix`), and relays in-thread **yes / no / NL-edit** (`POST .../{id}/decide`, NL-revise via local Qwen). **Bidirectional sync:** decide on the web → bot announces + closes the thread; decide on Matrix → the web panel's ~25s poll clears the card. State is CRM-side in **`email_proposal_matrix`** (1:1 side row, email-integration migration **`0003`**, additive + idempotent `CREATE TABLE IF NOT EXISTS` — the email runner re-runs every boot via `executescript`, so no `ALTER`), so it survives a bot restart. New **`bot` role** (authenticated, never admin — `require_bot_or_admin`) gates the email-proposal endpoints; the bot's CRM user must be flipped `member→bot` (one-time, kept out of the invite UI). **Deploy split:** endpoints + migration + role + frontend → **s9pk (v89/v90, installed)**; poll loop + review-room handling → **Spark git pull + restart (pending)**. Tests: `backend/test_email_proposal_matrix.py` + `backend/matrix_intake/test_email_proposals.py`; **30/30 suite green**, render-smoke green, migration verified twice on a copy of `data/crm.db` + applied clean on the box. **Next (see the two manual steps above): flip the bot user → `bot`; then Spark deploy with the review room; then live-smoke the web↔Matrix round-trip** (the popup is already verifiable on the box). Guide: `docs/guides/matrix-intake.md` "Email-activity proposal review".
+- **Email-proposal review over Matrix + a `bot` role — DEPLOYED & LIVE 2026-06-18 (box v0.1.0:91), smoke-tested OK.** **Post-smoke refinements (v0.1.0:91 + bot):** (1) every review card/reply is framed with `-----------------------` dash rules so threads don't bleed together on mobile (`email_proposals.frame`); (2) on a conclusive decision (either surface) the bot **redacts the whole thread** (`redact_thread`: the card + a backward room scan for its `m.thread` replies) so a resolved thread clears from both the timeline and the threads view — the bot was granted a `redact`/mod power level in the room (needed to redact the human's reply); no in-thread confirmation is posted on success (the thread vanishing is the ack — a confirmation would keep the thread alive). One-time backfill of pre-existing threads via `backend/matrix_intake/redact_resolved.py` (dry-run default; `--apply`); (3) the proposed note now NAMES who emailed whom — "{teammate} emailed {investor}" / "{sender} emailed the team" — instead of a bare "Sent/Received", and outbound detection now also matches our corporate domain (public providers excluded) so a teammate's non-enrolled `@ten31.xyz` mail no longer reads as "Received". Feature 1 (the inline source-email popup) is live on the box. The Matrix leg is live: bot CRM user flipped to `bot` (via the v90 Settings → Admin dropdown), `MATRIX_EMAIL_REVIEW_ROOM=!CImVJWFmzNxPcCcrZl:matrix.gilliam.ai` set on the Spark `.env`, bot joins the room on startup, polls every 20s. **End-to-end proven on deploy:** the whole pending backlog (N=16) posted as review cards to the room and was marked posted on the box (no errors); the bot now idles (to_post empty) holding 16 open threads. **Deploy gotchas learned:** (a) order matters — the bot caches its JWT (role embedded) at login and only re-logins on 401, not 403, so the role must be `bot` *before* the bot process starts; (b) "invited" ≠ "joined" — the bot must `client.join` the room (added in `2998706`) or `room_send` fails M_FORBIDDEN; (c) on first run the bot posts a card per *pending* proposal — fine here at N=16, but a large backlog would flood + get homeserver-429'd (a `since`-floor on `to_post` is the upgrade path if that ever bites). **Smoke-test still pending (manual):** reply `yes`/`no`/edit **in-thread** on a card → grid note + thread closes; approve one on the web → the bot announces + closes its thread (≤20s); confirm a Matrix decision clears the web card (≤25s). Two asks on the email-capture "proposed grid notes": (1) **inline source-email popup** — each proposed-note card on the Email Capture page gets a **View email** toggle that lazily fetches the existing **`GET /api/email/detail`** and renders from/to/cc/date/subject + scrollable body inline, so you can judge the note against the email (frontend-only, reuses the Communications detail pattern). (2) **CRM→Matrix review bridge** — the CRM (box, no matrix-nio) can't push, so the **intake bot (Spark) PULLS**: new **`GET /api/intake/email-proposals`** returns three work-lists (`to_post`/`open`/`to_close`); the bot posts a review card (metadata + snippet + draft note) to a **dedicated review room** (`MATRIX_EMAIL_REVIEW_ROOM`), records the thread root (`POST .../{id}/matrix`), and relays in-thread **yes / no / NL-edit** (`POST .../{id}/decide`, NL-revise via local Qwen). **Bidirectional sync:** decide on the web → bot announces + closes the thread; decide on Matrix → the web panel's ~25s poll clears the card. State is CRM-side in **`email_proposal_matrix`** (1:1 side row, email-integration migration **`0003`**, additive + idempotent `CREATE TABLE IF NOT EXISTS` — the email runner re-runs every boot via `executescript`, so no `ALTER`), so it survives a bot restart. New **`bot` role** (authenticated, never admin — `require_bot_or_admin`) gates the email-proposal endpoints; the bot's CRM user must be flipped `member→bot` (one-time, kept out of the invite UI). **Deploy split:** endpoints + migration + role + frontend → **s9pk (v89/v90, installed)**; poll loop + review-room handling → **Spark git pull + restart (pending)**. Tests: `backend/test_email_proposal_matrix.py` + `backend/matrix_intake/test_email_proposals.py`; **30/30 suite green**, render-smoke green, migration verified twice on a copy of `data/crm.db` + applied clean on the box. **Next (see the two manual steps above): flip the bot user → `bot`; then Spark deploy with the review room; then live-smoke the web↔Matrix round-trip** (the popup is already verifiable on the box). Guide: `docs/guides/matrix-intake.md` "Email-activity proposal review".
 
 - **Adopt the Pipeline — grid drives the deal board — DEPLOYED & live-smoked 2026-06-18 (v0.1.0:88; the full +Pipeline → board → advance-stage → remove round-trip is verified on the box). v88 (frontend-only): retired the Pipeline page's "+ New Opportunity" button + its create-by-contact modal** — opportunities are now born **only** from a grid investor row (matches how the team works; the board is view + stage-management; button replaced with a muted "Add deals from the Fundraising Grid" hint). An **"Add to Pipeline"** row action on the fundraising grid opens a seed modal (primary contact / target fund / expected amount / stage / probability) and creates a durably-linked `opportunities` row via the new **`opportunities.fundraising_investor_id`** (migration 0005, additive + reversible). **Grid owns the link + seed; the board owns stage/probability/owner** — a grid save never reseeds a live opp (`POST /api/fundraising/pipeline/link` is idempotent, one live opp/investor). Contact is **reused from the grid's synced `fundraising_contacts.contact_id`** (the `POST /api/contacts` side-door is gone); grid `lead`→owner. Two **read-only** grid columns (Pipeline action + Pipeline Stage) are **injected on read** from the live opp and **stripped on write** (never persisted, never dirty the autosave). **Remove from pipeline** (`POST .../unlink`) **soft-deletes the opp; the grid row stays fully intact**; deleting an investor from the grid archives its orphaned opp (`reconcile_grid_pipeline_links`, after `sync_fundraising_relational`). **Folded in:** the standing P2 soft-delete leak in `handle_pipeline_report` + dashboard pipeline aggregates (archived opps no longer counted). Tests: `backend/test_grid_pipeline_link.py`; 28/28 suite green, render-smoke green; migration verified on a copy of `data/crm.db` and **applied clean on the box**. **Next: live-smoke on the box — add an investor to the pipeline, confirm it lands on the board, advance a stage, and remove (opp archived, grid row intact).** Detail + locked decisions in `ROADMAP.md` "Adopt the Pipeline".
 

backend/matrix_intake/bot.py

@@ -11,7 +11,7 @@ Lifts matrix-bridge's prime-then-listen + threaded-reply plumbing. Config: repo
 """
 import asyncio
 
-from nio import AsyncClient, MatrixRoom, RoomMessageText
+from nio import AsyncClient, MatrixRoom, MessageDirection, RoomMessageText
 
 import crm_client
 import email_proposals
@@ -27,6 +27,7 @@ UNCLEAR_HELP = (
 )
 
 EMAIL_POLL_SEC = 20  # how often the bot polls the CRM for new/decided email-activity proposals
+MAX_THREAD_SCAN_PAGES = 8  # how far back to scan for a resolved thread's replies before redacting
 
 
 async def main():
@@ -163,14 +164,42 @@ async def main():
             await say(room_id, "I didn't catch that.\n\n" + proposals.render_disambiguation(proposal), root)
 
     async def redact_card(event_id):
-        """Remove a decided card from the room so only undecided ones remain. Redacting our OWN
+        """Redact one event (best-effort). Redacting our OWN message needs no special power;
-        card needs no special power; in Element a redacted message drops out of the timeline. (To
+        redacting someone else's reply needs the bot to hold a redact/mod power level."""
-        also wipe the human's yes/no reply for a fully-empty thread, give the bot a redact/mod
-        power level — not required for this.)"""
         try:
             await client.room_redact(review_room, event_id, reason="proposal resolved")
         except Exception as exc:
-            print(f"matrix-intake: could not redact card {event_id}: {exc}", flush=True)
+            print(f"matrix-intake: could not redact {event_id}: {exc}", flush=True)
+
+    async def redact_thread(root):
+        """Clear a resolved thread: redact the card AND every reply under it, so the thread drops
+        out of the threads view (not just the main timeline). The card is ours (always redactable);
+        the human's yes/no reply needs the bot's redact/mod power — if it lacks power that redact
+        just no-ops and the reply lingers. Finds replies by scanning recent room history for
+        m.thread events pointing at this root (the triggering reply is already synced, so a
+        backward scan from the current token includes it)."""
+        await redact_card(root)
+        token = getattr(client, "next_batch", None)
+        if not token:
+            return
+        try:
+            scanned = 0
+            for _ in range(MAX_THREAD_SCAN_PAGES):
+                resp = await client.room_messages(review_room, start=token,
+                                                  direction=MessageDirection.back, limit=100)
+                chunk = getattr(resp, "chunk", None)
+                if not chunk:
+                    break
+                for ev in chunk:
+                    rel = ((getattr(ev, "source", None) or {}).get("content", {}) or {}).get("m.relates_to") or {}
+                    if rel.get("rel_type") == "m.thread" and rel.get("event_id") == root:
+                        await redact_card(ev.event_id)
+                token = getattr(resp, "end", None)
+                scanned += len(chunk)
+                if not token or scanned > 1000:
+                    break
+        except Exception as exc:
+            print(f"matrix-intake: thread reply cleanup failed for {root}: {exc}", flush=True)
 
     async def handle_email_reply(room_id, root, text):
         """An in-thread reply to a CRM-drafted email-proposal card: yes commits, no dismisses, and
@@ -190,8 +219,9 @@ async def main():
                 email_threads[root] = item  # restore for retry
                 await say(room_id, email_proposals.frame(f"⚠️ couldn't add it ({str(exc)[:200]}). Reply **yes** to retry, **no** to dismiss."), root)
                 return
-            await say(room_id, email_proposals.frame(f"✅ Added to the grid for **{item.get('investor_name') or 'the investor'}**."), root)
+            # Success → clear the whole thread (card + replies). No confirmation: the thread
-            await redact_card(root)
+            # vanishing is the acknowledgment, and a confirmation reply would keep it alive.
+            await redact_thread(root)
         elif decision == "reject":
             email_threads.pop(root, None)
             try:
@@ -200,8 +230,7 @@ async def main():
                 email_threads[root] = item
                 await say(room_id, email_proposals.frame(f"⚠️ couldn't dismiss it ({str(exc)[:200]}). Try again."), root)
                 return
-            await say(room_id, email_proposals.frame("🗑️ Dismissed — nothing added to the grid."), root)
+            await redact_thread(root)
-            await redact_card(root)
         else:
             try:
                 new_note = await asyncio.to_thread(email_proposals.revise_note, item.get("note") or "", text)
@@ -244,12 +273,12 @@ async def main():
                                              "note": it.get("proposed_note") or ""}
                     except Exception as exc:
                         print(f"matrix-intake: failed to post email proposal {it.get('id')}: {exc}", flush=True)
-                for it in lists["to_close"]:  # decided on the web → remove the card, then close
+                for it in lists["to_close"]:  # decided on the web → clear the thread, then close
                     ev = it.get("event_id")
                     if not ev:
                         continue
                     try:
-                        await redact_card(ev)
+                        await redact_thread(ev)
                         await asyncio.to_thread(crm_client.mark_email_proposal_closed, it["id"])
                         email_threads.pop(ev, None)
                     except Exception as exc:

backend/matrix_intake/redact_resolved.py

@@ -39,7 +39,8 @@ async def main(apply):
 
         sync = await client.sync(timeout=10000, full_state=False)
         token = sync.next_batch
-        cards = {}  # event_id -> snippet (dedup across pages)
+        cards = {}    # root event_id -> snippet (still-identifiable card bodies)
+        replies = {}  # reply event_id -> (thread_root, snippet)
         for _ in range(MAX_PAGES):
             resp = await client.room_messages(review_room, start=token,
                                               direction=MessageDirection.back, limit=100)
@@ -47,24 +48,28 @@ async def main(apply):
             if not chunk:
                 break
             for ev in chunk:
-                if getattr(ev, "sender", None) != mx["user_id"]:
+                body = (getattr(ev, "body", "") or "").replace("\n", " ")
-                    continue
+                rel = ((getattr(ev, "source", None) or {}).get("content", {}) or {}).get("m.relates_to") or {}
-                body = getattr(ev, "body", "") or ""
+                if rel.get("rel_type") == "m.thread" and rel.get("event_id"):
-                if CARD_MARKER in body:
+                    replies[ev.event_id] = (rel["event_id"], body[:50])  # a threaded reply (card already redacted)
-                    cards[ev.event_id] = body.replace("\n", " ")[:70]
+                elif getattr(ev, "sender", None) == mx["user_id"] and CARD_MARKER in body:
+                    cards[ev.event_id] = body[:70]                       # an un-redacted card root
             token = getattr(resp, "end", None)
             if not token:
                 break
 
-        to_redact = [(eid, snip) for eid, snip in cards.items() if eid not in open_ids]
+        # Redact card roots that aren't still pending, AND any reply whose thread isn't still pending.
-        print(f"bot cards found: {len(cards)}; resolved (to redact): {len(to_redact)}")
+        targets = [(eid, "card  :: " + snip) for eid, snip in cards.items() if eid not in open_ids]
-        for eid, snip in to_redact:
+        targets += [(eid, "reply :: " + snip) for eid, (root, snip) in replies.items() if root not in open_ids]
-            print(("APPLY redact " if apply else "WOULD redact ") + eid + "  ::  " + snip)
+        print(f"resolved cards: {sum(1 for e,_ in cards.items() if e not in open_ids)}; "
+              f"thread replies to clear: {sum(1 for _,(r,_) in replies.items() if r not in open_ids)}")
+        for eid, label in targets:
+            print(("APPLY redact " if apply else "WOULD redact ") + eid + "  ::  " + label)
             if apply:
-                r = await client.room_redact(review_room, eid, reason="retroactive cleanup of resolved cards")
+                r = await client.room_redact(review_room, eid, reason="retroactive cleanup of resolved review threads")
                 if not hasattr(r, "event_id"):
                     print(f"   ! redact failed: {r}")
-        print(("done — redacted " if apply else "dry run — would redact ") + f"{len(to_redact)} card(s).")
+        print(("done — redacted " if apply else "dry run — would redact ") + f"{len(targets)} event(s).")
     finally:
         await client.close()