From b470ea2659734e1ae48553cd7a3feb3dcecd6e25 Mon Sep 17 00:00:00 2001 From: Keysat Date: Wed, 17 Jun 2026 20:10:16 -0500 Subject: [PATCH] Containerize the Matrix intake bot as a managed service (restart: unless-stopped) Turn the bot from a bare nohup process (silently dies on a Spark reboot) into a docker-compose service. Dockerfile bundles backend/matrix_intake + the stdlib backend/ingest Spark client it reuses; .env is mounted read-only at runtime, never baked. The existing repo-root .dockerignore (shared with the s9pk build) already keeps data/ and .env out of context. Also adds a handoff doc for wiring a spark-control dashboard card in a later session. --- backend/matrix_intake/Dockerfile | 22 +++++++ docker-compose.yml | 22 +++++++ .../add-intake-bot-to-spark-control.md | 65 +++++++++++++++++++ 3 files changed, 109 insertions(+) create mode 100644 backend/matrix_intake/Dockerfile create mode 100644 docker-compose.yml create mode 100644 docs/handoffs/add-intake-bot-to-spark-control.md diff --git a/backend/matrix_intake/Dockerfile b/backend/matrix_intake/Dockerfile new file mode 100644 index 0000000..1993f3e --- /dev/null +++ b/backend/matrix_intake/Dockerfile @@ -0,0 +1,22 @@ +# Container image for the Matrix intake bot — turns it from a bare nohup process into a managed +# service (docker compose `restart: unless-stopped` survives a Spark reboot). +# +# Build context is the REPO ROOT (see ../../docker-compose.yml), not this directory: the bot is +# NOT self-contained — spark.py reaches into backend/ingest/{llm,config,http_util}.py (stdlib +# only) via sys.path, so the image must carry both trees with the repo layout preserved. That +# keeps settings.load_env's REPO_ROOT (three dirs up from settings.py) = /app and spark.py's +# ingest path = /app/backend/ingest both correct at runtime. +FROM python:3.12-slim + +WORKDIR /app + +# The only third-party dep is matrix-nio; the reused ingest Spark client is pure stdlib. +COPY backend/matrix_intake/requirements.txt ./requirements.txt +RUN pip install --no-cache-dir -r requirements.txt + +COPY backend/matrix_intake/ ./backend/matrix_intake/ +COPY backend/ingest/ ./backend/ingest/ + +# .env (Matrix + CRM + Spark creds) is mounted read-only at /app/.env at runtime — never baked. +# `-u` keeps stdout/stderr unbuffered so `docker logs` shows the bot's lifecycle lines live. +CMD ["python", "-u", "backend/matrix_intake/bot.py"] diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..3483fae --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,22 @@ +# Runs the Matrix intake bot as a managed container on the Spark (spark-32d0, user `modelo`). +# +# `restart: unless-stopped` is the actual durability fix — the bot now survives a Spark reboot +# (it was a bare nohup process before, which silently died on reboot). `network_mode: host` so +# it can reach Matrix (clearnet TLS), the CRM API (box LAN), and Spark Control (local Qwen). +# The container name is fixed so a future spark-control dashboard card can find it by +# `docker inspect matrix-intake` (see docs/handoffs/add-intake-bot-to-spark-control.md). +# +# Deploy / update on the Spark: docker compose up -d --build +# Logs: docker logs -f matrix-intake +# Stop: docker compose down (or: docker stop matrix-intake) +services: + intake: + build: + context: . + dockerfile: backend/matrix_intake/Dockerfile + image: matrix-intake-bot + container_name: matrix-intake + network_mode: host + restart: unless-stopped + volumes: + - ./.env:/app/.env:ro diff --git a/docs/handoffs/add-intake-bot-to-spark-control.md b/docs/handoffs/add-intake-bot-to-spark-control.md new file mode 100644 index 0000000..c2b536d --- /dev/null +++ b/docs/handoffs/add-intake-bot-to-spark-control.md @@ -0,0 +1,65 @@ +# Handoff: add the Matrix intake bot as a spark-control dashboard card + +**Do this work in the `spark-control` repo (`~/Projects/spark-control`), in a separate session.** +This repo (ten31-database) only owns the bot + its container; the dashboard card is driven +entirely by spark-control code. Prereq (DONE 2026-06-17): the bot already runs as a docker +container named **`matrix-intake`** on the Spark (`spark-32d0`, user `modelo`), via +`docker-compose.yml` at this repo's root. spark-control reaches it over the **same SSH channel it +already uses for `matrix-bridge`** (`modelo@spark-32d0`) — no new key/host needed. + +The card is a near-exact clone of the existing `matrix-bridge` card. Mirror that, with **three +deltas** (below). File paths/line numbers are from the 2026-06-17 review; reconfirm against the +current code. + +## Deltas from matrix-bridge (do NOT copy these blindly) +1. **Branch is `main`, not `master`.** The Update button runs `git reset --hard origin/` + — it MUST be `main` for ten31-database, or Update silently resets to the wrong/empty ref. +2. **Project dir is `/home/modelo/ten31-database`** (the CRM monorepo clone), not `~/matrix-intake`. +3. **Coupling caveat:** because the bot lives in the CRM monorepo, the Update one-liner does + `git reset --hard origin/main` on the **whole CRM clone**. Safe today (`.env` is gitignored, + the clone has no needed local edits), but this is exactly the blast-radius smell that motivates + eventually extracting the bot to its own repo (logged in ten31-database `ROADMAP.md`). If that + extraction happens first, point dir/branch/remote at the new repo instead. + +## Edits in spark-control (mirror the matrix-bridge wiring) +1. **`image/app/config.py`** (matrix-bridge entry ~lines 99–111): add `matrix_intake_host` + (default `spark2_host`), `matrix_intake_user`, `matrix_intake_container` (default + `"matrix-intake"`), `matrix_intake_dir` (default `"/home/modelo/ten31-database"`), + `matrix_intake_branch` (default **`"main"`**), each with a `MATRIX_INTAKE_*` env fallback. +2. **`image/app/services.py`** (matrix-bridge ServiceDef ~lines 95–102): add a + `"matrix-intake": ServiceDef(name="matrix-intake", kind="bot", host=…, user=…, + container=…, port=0)` entry. `port=0` → judged by docker state alone (no HTTP probe), same as + matrix-bridge. +3. **`image/app/matrix_intake.py`** (new): copy `matrix_bridge.py`, rename + `matrix_bridge`→`matrix_intake` throughout. The Update command (`build_update_command`) must + produce: `cd /home/modelo/ten31-database && git fetch origin && git reset --hard origin/main && + docker compose up -d --build`. +4. **`image/app/server.py`**: (a) add `"matrix-intake"` to the `service_action` whitelist + (~line 621); (b) `from .matrix_intake import MatrixIntakeManager` + instantiate + `matrix_intake = MatrixIntakeManager(settings)` (~line 47); (c) add the 4 endpoints mirroring + matrix-bridge: `POST /api/matrix-intake/update`, `GET …/update/{job_id}`, + `GET …/update/{job_id}/stream`, `GET /api/matrix-intake/logs`. +5. **`image/app/static/app.js`** — THE RISKY EDIT. The Update/View-logs handlers are hardcoded to + matrix-bridge (`data-mb-update`, `onMatrixBridgeUpdate`, `/api/matrix-bridge/...`). Generalize + them to dispatch by the card's bot name (e.g. read `name` off the card, call + `/api//update` and `/api//logs`). Start/Stop/Restart are already generic. **Regression-check + that the existing matrix-bridge card still updates + tails logs after this change.** +6. **`package/startos/fileModels/sparkConfig.yaml.ts`** (~lines 27–32): add + `matrix_intake_user: z.string().catch('')`. +7. **`package/startos/main.ts`** (~line 68): inject `MATRIX_INTAKE_USER: cfg.matrix_intake_user`. +8. **(optional) `package/startos/actions/configureSparks.ts`**: add the intake-bot SSH-user field + to the form. + +## Deploy + ops +- spark-control is itself an s9pk: **bump its version, rebuild, reinstall** per spark-control's own + packaging docs (don't forget — same "0.4.x ignores same-version" rule). +- One-time: run **Configure Sparks** → set the intake bot's SSH user to `modelo` (same as + matrix-bridge → key already authorized). The card appears once the `matrix-intake` container + exists and the user is set; it hides itself if the container is absent or the user is blank. +- Status pill = `docker inspect matrix-intake .State.Status` (running→Healthy). No Matrix-liveness + check — a running-but-silent bot still shows Healthy (same limitation as matrix-bridge). + +## Done when +The dashboard shows a `matrix-intake` card alongside `matrix-bridge` with a Healthy pill and +working Update / Start / Restart / Stop / View-logs buttons — and the matrix-bridge card is +unregressed.