diff --git a/AGENTS.md b/AGENTS.md index 4d5a2be..052c56e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,8 +55,8 @@ Subsystem guidance lives in `docs/guides/` and loads when matching files are tou - **In progress — Signal Engine "flakiness":** diagnosed, not a server bug — transient 1–4s unresponsiveness while the single GPU is continuously busy. Remedy is client-side; a drafted message (in-flight cap 2, hard ceiling 3 global across audio endpoints, retry-with-backoff on timeout/503) is with the owner to forward to that dev. - **Decided, not implemented:** remote access stays WireGuard/Tailscale split-tunnel — no public interface, so no API auth built; an empirical concurrency sweep is offered but needs the owner's explicit OK in a quiet window. **Revisit (full-eval 2026-06-12):** the "LAN-only, so no auth" call is now load-bearing against RCE — unquoted user input reaches the SSH shell on several endpoints, so the network boundary is the *only* thing preventing cluster takeover. Quoting the injection sinks (work queue) is needed regardless of the auth decision; a defense-in-depth auth/CSRF gate is the follow-on. - **Known limits:** `/health` blips while the GPU is busy (mitigated client-side); dual-channel can miss a quiet local word under loud remote bleed; the connectivity log misses sub-5s outages between 5s polls; diarizer caps at 4 speakers. -- **Portability:** working tree scrubbed 2026-06-12 — all owner-specific IPs/hostnames/usernames/names replaced with placeholders in tracked files; `claude-code-starter-prompt.md` deleted (old build-time prompt). Real cluster values live only in StartOS install config, shell env vars, and the gitignored `settings.local.json`. **Caveat (full-eval 2026-06-12): git *history* was not rewritten** — the old IPs/hosts/user ``/key name are still recoverable pre-`50c67cd`. The scrub is working-tree-only; treat the repo as private until history is rewritten (see work queue below). -- **Repo wart:** commit `367d986` is labeled `v0.13.0:4` but actually contains everything through v0.18.0:0 — per-version commits for v0.14–v0.18 are missing. Keep commit messages accurate going forward. +- **Portability:** working tree + **full git history** scrubbed 2026-06-12 — all owner-specific IPs/hostnames/usernames/names replaced with placeholders; `claude-code-starter-prompt.md` deleted (old build-time prompt). Real cluster values live only in StartOS install config, shell env vars, and the gitignored `settings.local.json`. History was rewritten with `git filter-repo` (every commit SHA changed; force-pushed to `gitea`); a token sweep across all refs now returns 0 for every owner-specific value, including three LAN IPs + one Start9 address the original working-tree scrub had missed. Backup bundle of the pre-rewrite history is at `../spark-control-prehistory-rewrite.bundle`. **Still owner-side:** the `` SSH key was leaked by *name* only (not material) — rotate it if it's still authorized on the Sparks. +- **Repo wart:** commit `8d839e3` (was `367d986` before the 2026-06-12 history rewrite) is labeled `v0.13.0:4` but actually contains everything through v0.18.0:0 — per-version commits for v0.14–v0.18 are missing. Keep commit messages accurate going forward. - **Hosting:** repo pushes to the owner's self-hosted Gitea — remote `gitea`, branch `master`, over SSH (host alias + key live in the local `~/.ssh/config`; no owner-specific details belong in the repo). Push there after committing. - **Next (pre-eval backlog):** (1) owner forwards the concurrency note to the Signal Engine dev; (2) run the concurrency sweep if the dev wants the measured knee; (3) add the `--memory` cap to parakeet-asr via the Reapply-patches action; (4) pick the next item from ROADMAP.md. @@ -66,11 +66,11 @@ Source: `EVALUATION.md` at repo root (full evidence, file:line pointers, scoreca **Work queue — P0/P1, fix before sharing the package wider:** 1. ~~**[P0] Shell-quote/validate every user value crossing into SSH**~~ — **DONE (code, 2026-06-12; not yet shipped).** New `image/app/shellsafe.py` (`validate_repo`/`validate_image`/`validate_container` whitelists + `quote_arg`/`quote_args`). Boundary validation added to `POST /api/models` (repo) and `POST /api/nim/install` (image+container); `shlex.quote` applied at every SSH sink — `models.build_launch_command` (repo+args, covers `vllm_args`+knobs), `download._do` (repo), `nim._do` (image/container/volume/port/env), `services.docker_state`+`run_action` (container). Verified: injection survives only as a single quoted token, vLLM preflight `shlex.split` round-trip intact, both redaction suites still pass. Side-benefit: NGC key now `shlex.quote`'d in `nim._do` (was single-quoted) — closes the quote-breakout half of the P2 NGC-key item; the process-list-exposure half remains. **Ship step pending:** version bump + release notes + rebuilt s9pk. -2. **[P0] Decide the git-history question** — owner IPs/hosts/user ``/key name persist pre-`50c67cd` despite the working-tree scrub. Either rewrite history (`git-filter-repo`) + rotate the `` key, or keep the repo private-forever. Blocks any public/shared publish. **(Open — git-ops decision, not code.)** +2. ~~**[P0] Decide the git-history question**~~ — **DONE (2026-06-12).** Chose to rewrite. `git filter-repo --replace-text --replace-message` mapped every owner-specific value to its canonical placeholder across all commits + tags + messages (incl. three LAN IPs and one Start9 address the working-tree scrub missed — the Start9 address was still live in HEAD at `docs/AUDIO_API.md`). Verified 0 hits across all refs; force-pushed to `gitea`. Pre-rewrite backup bundle kept at `../spark-control-prehistory-rewrite.bundle`. **Owner follow-up:** rotate the `` SSH key if still authorized on the Sparks (only its name leaked, not the key). 3. ~~**[P1] Defense-in-depth gate on mutating endpoints**~~ — **DONE (code, 2026-06-12; not yet shipped).** `csrf_guard` HTTP middleware in `server.py` rejects state-changing requests whose `Origin`/`Referer` hostname ≠ the served host. Scoped to control endpoints; the programmatic API surface is exempt (`/v1/*`, `/scrub`, `/rehydrate`, `/api/search`, `/api/audio/`, `/api/health-event`) so downstream consumers are unaffected. No app-layer token auth (deliberate — would break consumers + the non-technical owner). Verified via TestClient: cross-origin control POST→403, same-origin/no-Origin→pass, exempt prefixes always pass, GET never blocked. **Verify on-box:** confirm the StartOS reverse proxy passes `Host`/`Origin` so the dashboard isn't false-positive-blocked. 4. ~~**[P1] Validate the Qdrant `collection`**~~ — **DONE (code, 2026-06-12; not yet shipped).** `_safe_collection` whitelist (`[A-Za-z0-9._-]`, rejects `..`) + URL-encoded path segment in `embeddings_proxy.py`. The raw `filter` is left as a passthrough (Qdrant parses it; pydantic enforces `dict`) — locking it to an allowlist would break hybrid-search consumers; the path segment was the real injection vector. -**Shipping (all of #1/#3/#4 batched):** version bumped `0.18.0:1`→`0.19.0:0` with release notes (`versions/v0_1_0.ts`). Rebuild `make x86`; `make install` (live-service restart) needs explicit go-ahead. Not committed yet. +**Shipping (all of #1/#3/#4 batched):** SHIPPED 2026-06-12 — version `0.18.0:1`→`0.19.0:0`, release notes in `versions/v0_1_0.ts`, s9pk rebuilt (`make x86`) and sideloaded to the live Start9 server (`make install`). Committed as `1c4e861` (its SHA after the same-day history rewrite) and force-pushed to `gitea`. **On-box check still pending:** click a control action (swap / service stop) to confirm the new CSRF guard doesn't false-positive-block the dashboard behind the StartOS proxy. **Known debt — P2, track but not blocking:** - Test coverage is redaction-only; swap state machine, proxies, SSH wrapper, and the package have zero automated tests. Live-cluster paths (swap exec, audio, embeddings/search) couldn't be exercised at all — biggest blind spot. diff --git a/EVALUATION.md b/EVALUATION.md index 6812e5e..a18b271 100644 --- a/EVALUATION.md +++ b/EVALUATION.md @@ -18,7 +18,7 @@ This is a capable, well-documented single-operator control plane: a ~960-line Fa ## Priority queue - [P0] Command injection via unquoted user input (`repo`, `vllm_args`, NIM `image`/`container`/`port`, custom-service `container`) interpolated into SSH shell commands → arbitrary RCE as the SSH user on the Sparks — `models.py:80`, `swap.py:101`, `download.py:129`, `nim.py:145-166`, `services.py:144`; demonstrated via `build_launch_command` — evaluator + security-auditor -- [P0] Owner infra topology (IPs `/.87`, QSFP `/11`, hosts ``/``, user ``, key ``) persists in git history pre-`50c67cd` despite the working-tree scrub → target list for the unauthenticated endpoints — security-auditor +- [P0] Owner infra topology (IPs ``/``, QSFP ``/``, hosts ``/``, user ``, key ``) persisted in git history despite the working-tree scrub → target list for the unauthenticated endpoints — security-auditor [RESOLVED 2026-06-12: history rewritten with git filter-repo; 0 hits across all refs] - [P1] No auth + no CSRF protection on state-changing endpoints (plaintext `http`, `interfaces.ts:8`) → any LAN peer, or a malicious page in the operator's browser, can drive swap/install/stop/delete and chain into the P0 injections — security-auditor (CSRF P1) + evaluator (auth P2, escalated) - [P1] SSRF / Qdrant path injection: caller `collection` interpolated into the Qdrant URL with no validation and raw `filter` forwarded verbatim — `embeddings_proxy.py:237,175,204` — security-auditor - [P2] Test coverage is redaction-only; the swap state machine, proxies, SSH wrapper, and the StartOS package have zero automated tests — evaluator