Compare commits

..

10 Commits

Author SHA1 Message Date
Keysat b10399819b Add inbox-check line; align .gitignore with canonical .claude policy
Cross-repo git-hygiene audit remediation: surface ~/Projects/standards/INBOX.md items at session start, and switch .gitignore to the deny-by-default .claude/* block (shared wiring allow-listed) plus the canonical secrets/env lines — per standards/portability.md.
2026-06-14 12:17:16 -05:00
Keysat e4c6c30ee3 docs: refresh Current state after P1/P2 security pass; move P3+ to ROADMAP 2026-06-13 18:28:27 -05:00
Keysat 3e33728013 Mark three P2 hardening items done in Current state 2026-06-13 18:22:20 -05:00
Keysat 693d72431b Return clean JSON for body-parser/unhandled errors
Malformed JSON request bodies fell through to Express's default error
handler, which renders an HTML page including the local-filesystem stack
trace (info leak). Add a final error-handling middleware: JSON 400 for
entity.parse.failed, 413 for entity.too.large, generic 500 otherwise —
closing the stack-trace leak on every propagated error, not just JSON.
2026-06-13 18:22:00 -05:00
Keysat da1bba2e6b Compare operator key in constant time
resolveIdentity and verifyOperatorKey compared the shared
relay_cloud_operator_key with ===/!==, which short-circuits on the first
differing byte — a timing oracle on a high-value key. Use a
timingSafeEqual-based constantTimeEqual, matching admin-auth.js.
2026-06-13 18:22:00 -05:00
Keysat cbd9748a79 Guard meeting :id against path traversal
saveMeeting/loadMeeting/deleteMeeting built path.join(meetingsDir, id +
'.json') straight from req.params.id, so an admin-authed :id like
'../../etc/passwd' could read/write/delete outside internal-meetings/.
Centralize a meetingPath() helper that strips anything outside
[A-Za-z0-9_-] (mirrors output-store.js) and throws on an empty result;
load/delete catch it as 404/no-op. Add a regression test.
2026-06-13 18:22:00 -05:00
Keysat 54ddaffced Mark P1 work queue done in Current state 2026-06-13 16:23:55 -05:00
Keysat 3a601e166a Bump multer 1.4.5-lts.1 -> ^2.0.1 (DoS CVEs)
multer 1.x is affected by CVE-2025-47944/47935/48997/7338 (malformed
multipart crashes the process / leaks memory). 2.x raises catchable
errors instead. Usage (diskStorage + .single("file")) is unchanged.
Commit the server lockfile so the Dockerfile's npm-ci path pins the fix.
2026-06-13 16:23:26 -05:00
Keysat d2caa98248 Fix credit-counter reset on early subscription renewal
extendUserTier called setUserTier, which unconditionally zeroed
monthly_consumed and re-anchored the cycle. A user who renewed mid-cycle
(or a webhook double-firing across a restart) got their full monthly
allotment back for free. The monthly cycle already rolls on its own
anniversary via ensureRenewalRollover, so renewal must not reset it. Add
resetCycle to setUserTier (default true, preserving operator-grant
behavior); extendUserTier passes false for an in-force subscription and
true only for a brand-new or lapsed one. Add regression tests.
2026-06-13 16:23:26 -05:00
Keysat 8ad7c54da4 Block SSRF on media_url downloads (transcribe-url/summarize-url)
downloadDirect fetched any caller-supplied media_url with redirect-follow
and no host/scheme validation; the route is reachable via a self-chosen
X-Recap-Install-Id, so a caller could probe the operator's LAN or cloud
metadata (169.254.169.254). Add safe-url.js: assertPublicHttpUrl rejects
non-http(s) schemes and hosts resolving to private/loopback/link-local/
reserved ranges, and safeFetch follows redirects manually, re-validating
each hop. Route downloadDirect through it (covers transcribe-url,
summarize-url, and admin-test-run).
2026-06-13 16:23:26 -05:00
14 changed files with 1863 additions and 54 deletions
+8 -2
View File
@@ -26,8 +26,14 @@ ytdlp-cache/
# Local dev secrets # Local dev secrets
.env .env
.env.*
!.env.example
# Claude Code state (worktrees, plans, etc.) — but commit the lazy-load # Claude Code — deny by default (worktrees, plans, local settings stay out),
# rule symlinks under .claude/rules/ (they point into docs/guides/). # allow-list shared wiring (see standards/portability.md).
.claude/* .claude/*
!.claude/rules/ !.claude/rules/
!.claude/agents/
!.claude/commands/
!.claude/skills/
!.claude/settings.json
+16 -33
View File
@@ -2,6 +2,9 @@
Operator-side, credit-metered service that sits in front of Gemini and the operator's local AI hardware ("Spark Control": Parakeet ASR, Sortformer diarization, TitaNet voice embeddings, a vLLM/Gemma analyze endpoint). The Recaps app (`../recap`) is the client; this repo owns transcription/diarization/analysis routing, the cloud Pro/Max tier + expiry, self-serve billing settlement, and the **internal-meetings** feature (upload audio → transcribe → diarize → cluster → analyze → polish → operator dashboard). **Private. Ships to the operator's own Start9 box via `make install` only — NEVER to the public registry.** Operator-side, credit-metered service that sits in front of Gemini and the operator's local AI hardware ("Spark Control": Parakeet ASR, Sortformer diarization, TitaNet voice embeddings, a vLLM/Gemma analyze endpoint). The Recaps app (`../recap`) is the client; this repo owns transcription/diarization/analysis routing, the cloud Pro/Max tier + expiry, self-serve billing settlement, and the **internal-meetings** feature (upload audio → transcribe → diarize → cluster → analyze → polish → operator dashboard). **Private. Ships to the operator's own Start9 box via `make install` only — NEVER to the public registry.**
> **Inbox check:** At session start, if `~/Projects/standards/INBOX.md` exists, scan it for
> items tagged `(recap-relay)` and surface them before proposing next steps; triage with `/triage`.
## Stack ## Stack
- **Server**: Node.js (`type: module`, ES modules). Same dev box as the app (`v25.6.1`); container runtime is whatever the `Dockerfile` pins. - **Server**: Node.js (`type: module`, ES modules). Same dev box as the app (`v25.6.1`); container runtime is whatever the `Dockerfile` pins.
@@ -43,7 +46,8 @@ server/
chunked-analyze.js windowed analyze (planWindowsByDuration, runPipelinedAnalysis, …) chunked-analyze.js windowed analyze (planWindowsByDuration, runPipelinedAnalysis, …)
config.js getConfigSnapshot() + relay_* config defaults config.js getConfigSnapshot() + relay_* config defaults
hardware-config.js resolveHardwareConfig() → Spark Control endpoint discovery hardware-config.js resolveHardwareConfig() → Spark Control endpoint discovery
test/ node --test files (speaker-clustering, meeting-speaker-edits, credits) safe-url.js SSRF guard: assertPublicHttpUrl + safeFetch for caller-supplied URLs
test/ node --test *.test.js (speaker tools, billing/credits, SSRF, path-traversal, …)
public/dashboard.html operator dashboard (meetings detail view + speaker tools) public/dashboard.html operator dashboard (meetings detail view + speaker tools)
startos/versions/<vN>.ts one file per version + index.ts graph startos/versions/<vN>.ts one file per version + index.ts graph
docs/issues-backlog.md detailed issue log docs/issues-backlog.md detailed issue log
@@ -120,6 +124,7 @@ this. When unsure whether a change is contract-affecting, assume it is and check
- **Before editing the internal-meetings / diarization / speaker subsystem, read `docs/guides/internal-meetings.md`** — the diarize→cluster→polish pipeline, the four-places speaker-label sync rule, the clustering-threshold knobs, and the post-hoc speaker-edit (merge / recluster / repolish) semantics live there. Scoped to `server/{speaker-clustering,post-cluster-polish,meeting-extras,meeting-speaker-edits,chunked-analyze}.js`, `server/routes/internal-meetings.js`, `server/backends/hardware.js`. - **Before editing the internal-meetings / diarization / speaker subsystem, read `docs/guides/internal-meetings.md`** — the diarize→cluster→polish pipeline, the four-places speaker-label sync rule, the clustering-threshold knobs, and the post-hoc speaker-edit (merge / recluster / repolish) semantics live there. Scoped to `server/{speaker-clustering,post-cluster-polish,meeting-extras,meeting-speaker-edits,chunked-analyze}.js`, `server/routes/internal-meetings.js`, `server/backends/hardware.js`.
- **Doc layout**: `AGENTS.md` is canonical; `CLAUDE.md` is a symlink to it (don't overwrite it). Subsystem guides are real files in `docs/guides/<topic>.md` (with `paths:` frontmatter); `.claude/rules/<topic>.md` are relative symlinks into them (`.gitignore` carves out `!.claude/rules/` so the symlinks commit). New guide = add `docs/guides/<topic>.md`, symlink it from `.claude/rules/`, add an index line above. - **Doc layout**: `AGENTS.md` is canonical; `CLAUDE.md` is a symlink to it (don't overwrite it). Subsystem guides are real files in `docs/guides/<topic>.md` (with `paths:` frontmatter); `.claude/rules/<topic>.md` are relative symlinks into them (`.gitignore` carves out `!.claude/rules/` so the symlinks commit). New guide = add `docs/guides/<topic>.md`, symlink it from `.claude/rules/`, add an index line above.
- **Fetching a caller-supplied URL? Go through `server/safe-url.js`** (`safeFetch` / `assertPublicHttpUrl`) — the SSRF guard that rejects non-http(s) schemes and hosts resolving to private/loopback/link-local/reserved ranges, and re-validates every redirect hop. `downloadDirect` (the transcribe-url/summarize-url/admin-test-run download path) already routes through it; never raw-`fetch` an untrusted URL. Calls to the operator's OWN hardware/LAN use `lan-fetch.js` instead — those URLs are config-set and intentionally private.
- **`make install` correctness**: see [Always]. Honest reports; failing test/build is a failure. Comments explain WHY. Write tests alongside (`server/test/*.test.js`, `node --test`). - **`make install` correctness**: see [Always]. Honest reports; failing test/build is a failure. Comments explain WHY. Write tests alongside (`server/test/*.test.js`, `node --test`).
## Always ## Always
@@ -136,36 +141,14 @@ this. When unsure whether a change is contract-affecting, assume it is and check
- **Never edit a `startos/versions/<v>.ts` that's already been built/installed** — add a new version file. - **Never edit a `startos/versions/<v>.ts` that's already been built/installed** — add a new version file.
- **Don't push to GitHub by default** — remote is self-hosted Gitea. - **Don't push to GitHub by default** — remote is self-hosted Gitea.
## Current state — full eval done (2026-06-13); findings triaged below ## Current state — post-eval security pass landed (2026-06-13)
- **Box, local tree, and git aligned at relay `0.2.124`** (app at `0.2.155`). `startos/versions/index.ts` `current: v_0_2_124`. Git history is local-only (no remote). Working tree is clean apart from an untracked `server/package-lock.json` left by the eval's `npm install` — a generated artifact, intentionally NOT committed. - **Box, local tree, git aligned at relay `0.2.124`** (app `0.2.155`); `current: v_0_2_124`. Git is local-only (no remote). Working tree clean. **Suite green at 60 tests** (`cd server && npm test`); server boots clean.
- **Full independent evaluation run 2026-06-13** (evaluator + security-auditor + exerciser + doc-auditor + start9-spec-checker). Report committed at `EVALUATION.md` (`b08e836`); it's overwritten in place each run so re-running gives a reviewable diff. 47/47 tests still pass; server boots clean. Findings triaged into the three buckets below. - **Full independent eval done** (evaluator + security-auditor + exerciser + doc-auditor + start9-spec-checker) `EVALUATION.md` (overwritten in place each run, so re-running diffs cleanly).
- **Post-hoc speaker tools remain live**: `meeting-speaker-edits.js` (merge / recluster / repolish + backfill) + the `PATCH/POST /admin/internal-meetings/:id/{merge-speakers,recluster,repolish}` routes; dashboard exposes the controls. - **All P0/P1 fixed** this session (commits `8ad7c54`/`d2caa98`/`3a601e1`): SSRF guard on caller-supplied media URLs (new `server/safe-url.js`), the early-renewal credit-reset money-leak (`extendUserTier`/`setUserTier` `resetCycle`), and the `multer``^2.0.1` DoS bump. None touch the `../recap` client contract.
- **Three P2 fixed** (commits `cbd9748`/`da1bba2`/`693d724`): meeting-`:id` path-traversal guard (`meetingPath()`), constant-time operator-key compare, and a JSON error handler that closes the malformed-body stack-trace leak.
### Work queue — P0/P1 (fix first) - **Next (open P2), in priority order:**
1. Persist webhook dedup so a restart can't double-credit/double-extend — `routes/credits.js:63`, `zaprite-webhook.js:27`.
1. **SSRF on `/relay/transcribe-url` + `/relay/summarize-url`**`downloadDirect` (`server/routes/transcribe-url.js:99-159`) fetches any caller `media_url` with redirect-follow and no scheme/host allowlist; reachable via a self-chosen `X-Recap-Install-Id` (effectively unauthenticated). Reject non-http(s) + private/loopback/link-local hosts; re-validate per redirect hop. *(evaluator + security-auditor + exerciser — headline risk)* 2. **Needs operator decision:** is BTCPay a hard requirement or truly optional? It's `optional:false`/`kind:'running'` despite "optional" comments, so StartOS won't start the relay without BTCPay co-installed — `startos/manifest/index.ts:38-49` + `dependencies.ts`. Then make manifest/deps/comment agree.
2. **Billing money-leak: early/mid-cycle renewal resets `monthly_consumed = 0`**`extendUserTier``setUserTier` (`server/credits.js:770→641`) hands back a full monthly allotment for free; a webhook double-fire across restart compounds it. Preserve `monthly_consumed`/`last_renewal_at`, roll only on a true period boundary; add a money-path test (`tier-expiry.test.js:63` asserts only the date today). *(evaluator)* 3. Money-path unit tests (`commitCredit`/`refundCredit`/`applyTierPromotion`/grant handlers); scope `cors()` off `/admin/*` (`index.js`); split the 2225-line `routes/internal-meetings.js`; fix the two AGENTS.md auth-doc drifts (Stack-line `/relay/*` auth; missing `/admin/btcpay/callback` exempt path).
3. **`multer@1.4.5-lts.2` DoS CVEs** (CVE-2025-47944/47935/48997/7338) — malformed multipart can crash the process. Upgrade to `^2.0.1`, re-test the `file` upload field (`server/package.json:15`). *(security-auditor + exerciser)* - **Risks/notes:** SSRF guard leaves a DNS-rebinding TOCTOU open (acceptable for a private box; revisit if exposed). P3+ deferred tail + pre-existing speaker-tool/empty-section backlog → `ROADMAP.md` / `docs/issues-backlog.md`.
### Known debt — P2 (accepted for now; fix opportunistically)
- Path traversal on internal-meetings `:id` (admin-gated): validate `^[A-Za-z0-9_-]+$` before `path.join``routes/internal-meetings.js:84,91,242` (`output-store.js:52` shows the pattern). *(security-auditor + exerciser)*
- Non-constant-time operator-key compare (`!==`) on `relay_cloud_operator_key``server/identity.js:43,84`; use `timingSafeEqual` like the admin path. *(evaluator + security-auditor)*
- In-memory webhook dedup Set lost on restart → double-credit/double-extend — `routes/credits.js:63`, `zaprite-webhook.js:27`; persist processed invoice/order ids. *(security-auditor)*
- Malformed JSON body → full Node stack trace (FS paths) — add an Express `entity.parse.failed` → JSON-400 handler. *(exerciser)*
- BTCPay declared `optional:false`/`kind:'running'` despite "optional" comments → StartOS won't start the relay without BTCPay co-installed — `startos/manifest/index.ts:38-49`, `startos/dependencies.ts`. Decide, then make manifest + dependencies + comment agree. *(start9-spec-checker)*
- No money-path unit tests (`commitCredit`/`refundCredit`/`applyTierPromotion`/`planBackend`/grant handlers) — why the P1 billing bug ships green. *(evaluator)*
- `routes/internal-meetings.js` is 2225 lines; extract the MD/HTML formatters + storage/backfill layer. *(evaluator)*
- Fully-open `cors()` incl. `/admin/*` — scope origins — `server/index.js:54`. *(evaluator)*
- Doc drift: AGENTS.md "Stack" line mis-states `/relay/*` auth (most routes are per-call header auth; only `routes/user-tier.js` needs the operator key); the admin-exempt list omits `/admin/btcpay/callback` (`admin-auth.js:70`). *(doc-auditor)*
### Deferred — P3+ (later decision or bulk cleanup)
- Security hardening: no `/relay/*` rate limiting; container likely runs as root (entrypoint `chown`s uid 1001 but no `USER` directive); dashboard `innerHTML` stored-XSS surface; `lan-fetch` TLS verify off (admin-set URL only); debug/error fields leaked to clients. *(security-auditor + evaluator)*
- Packaging/ops: prune the 126 `startos/versions/*.ts` files; pin `yt-dlp` in the Dockerfile; Dockerfile per-subdir `COPY` footgun; manifest polish (SPDX license, `docsUrls`, real repo URLs, icon format); no `README.md` (blocks public-registry submission only — moot for this private box). *(start9-spec-checker + evaluator)*
- `/relay/health` reports stale `0.2.11``server/package.json` never bumped past 0.2.11; bump it to track the StartOS version. *(exerciser + doc-auditor)*
- Doc fixes (bulk): the `test/` layout lists 3 of 6 files; `server/index.js:3-6` "two endpoints" header comment is stale; `POST /admin/logout` undocumented. *(doc-auditor)*
- Untested blind spot: the live upload → merge → recluster → repolish pipeline (admin-gated + needs Spark Control) has only unit coverage; the dependency audit ran offline — re-run `npm audit`/`osv-scanner` with network to confirm the multer finding and catch transitive CVEs. *(all agents)*
**Pre-existing backlog** (separate from the eval): speaker-tool follow-ups and the empty-analysis-section issue — see `ROADMAP.md` / `docs/issues-backlog.md`.
+10
View File
@@ -14,6 +14,16 @@ Longer-term backlog for the relay. Near-term in-flight work + known box/local st
- **Empty analysis section at a window boundary** (observed v0.2.77 smoke test). Likely the LLM returning an empty `{title:"",summary:""}` section the stitcher accepts, or a window-merge boundary hole. Low priority. Full triage path in `docs/issues-backlog.md`. - **Empty analysis section at a window boundary** (observed v0.2.77 smoke test). Likely the LLM returning an empty `{title:"",summary:""}` section the stitcher accepts, or a window-merge boundary hole. Low priority. Full triage path in `docs/issues-backlog.md`.
## Post-eval P3+ backlog (full eval 2026-06-13 — deferred, low risk for the private box)
From `EVALUATION.md`. P1 + three P2 items already fixed (see git log `8ad7c54``693d724`); these are the deferred tail.
- **Security hardening:** no `/relay/*` rate limiting; container likely runs as root (entrypoint `chown`s uid 1001 but no `USER` directive); dashboard `innerHTML` stored-XSS surface; `lan-fetch` TLS verify off (admin-set URL only); debug/error fields leaked to clients.
- **Packaging/ops:** prune the 126 `startos/versions/*.ts` files; pin `yt-dlp` in the Dockerfile; the Dockerfile per-subdir `COPY` footgun; manifest polish (SPDX license, `docsUrls`, real repo URLs, icon format); no `README.md` (blocks public-registry submission only — moot for this private box).
- **`/relay/health` reports stale `0.2.11`** — `server/package.json` version never bumped past 0.2.11; bump to track the StartOS version.
- **Doc fixes (bulk):** the `test/` layout line; `server/index.js:3-6` "two endpoints" header comment is stale; `POST /admin/logout` undocumented.
- **Untested blind spot:** the live upload → merge → recluster → repolish pipeline (admin-gated + needs Spark Control) has only unit coverage; re-run `npm audit`/`osv-scanner` with network to catch transitive CVEs the offline audit missed.
## Adjacent (lives in `../recap`) ## Adjacent (lives in `../recap`)
The app surfaces relay features but owns its own roadmap. Relay-side items the app is waiting on, or that change app behavior, belong in `../recap/ROADMAP.md` under its "Adjacent" section — keep them cross-referenced, not duplicated. The app surfaces relay features but owns its own roadmap. Relay-side items the app is waiting on, or that change app behavior, belong in `../recap/ROADMAP.md` under its "Adjacent" section — keep them cross-referenced, not duplicated.
+27 -11
View File
@@ -741,21 +741,26 @@ export async function addPurchasedCredits({
// stored on the user's credit row (keyed `user:<id>`). Set by the // stored on the user's credit row (keyed `user:<id>`). Set by the
// operator (today) and the self-serve purchase flow (later slice). // operator (today) and the self-serve purchase flow (later slice).
// Operator-set a cloud user's tier. Resets the monthly counters and // Operator-set a cloud user's tier. With `resetCycle` (the default) it
// anchors the renewal to now (so the monthly cycle starts on the grant // starts a fresh monthly cycle anchored to now — so an operator comp
// date), mirroring applyTierPromotion. `expiresAt` is stored for // grant, or a first/lapsed self-serve purchase, begins its allowance on
// reporting / future self-serve billing but NOT auto-enforced in this // the grant date (mirroring applyTierPromotion). A renewal of an
// slice — to revoke, set tier back to "core". // in-force subscription passes `resetCycle: false` so it extends the
export async function setUserTier({ userId, tier, expiresAt = null }) { // expiry WITHOUT zeroing monthly_consumed — see extendUserTier.
// `expiresAt` is stored for reporting / self-serve billing but NOT
// auto-enforced here — to revoke, set tier back to "core".
export async function setUserTier({ userId, tier, expiresAt = null, resetCycle = true }) {
if (!userId) throw new Error("setUserTier: userId required"); if (!userId) throw new Error("setUserTier: userId required");
const t = tier === "pro" || tier === "max" ? tier : "core"; const t = tier === "pro" || tier === "max" ? tier : "core";
const row = await getOrCreateRow({ creditKey: `user:${userId}` }); const row = await getOrCreateRow({ creditKey: `user:${userId}` });
const now = new Date(); const now = new Date();
row.tier_snapshot = t; row.tier_snapshot = t;
row.monthly_consumed = 0; if (resetCycle) {
row.monthly_gemini_consumed = 0; row.monthly_consumed = 0;
row.last_renewal_at = now.toISOString(); row.monthly_gemini_consumed = 0;
row.anniversary_day = now.getUTCDate(); row.last_renewal_at = now.toISOString();
row.anniversary_day = now.getUTCDate();
}
row.subscription_expires_at = expiresAt || null; row.subscription_expires_at = expiresAt || null;
row.last_active_at = now.toISOString(); row.last_active_at = now.toISOString();
await persist(); await persist();
@@ -767,6 +772,13 @@ export async function setUserTier({ userId, tier, expiresAt = null }) {
// (still-active) expiry — so paying early ADDS time rather than resetting // (still-active) expiry — so paying early ADDS time rather than resetting
// it. `periodDays` defaults to 30. Both payment rails (BTCPay + Zaprite) // it. `periodDays` defaults to 30. Both payment rails (BTCPay + Zaprite)
// land here on a settled payment. Returns the updated row. // land here on a settled payment. Returns the updated row.
//
// Renewing an IN-FORCE subscription must NOT reset the monthly credit
// counter — otherwise a user who paid early (or a webhook that double-
// fired across a restart) would get their whole monthly allotment back
// for free. The monthly cycle rolls on its own anniversary via
// ensureRenewalRollover, independent of renewals. Only a brand-new or
// lapsed subscription starts a fresh cycle.
export async function extendUserTier({ userId, tier, periodDays = 30 }) { export async function extendUserTier({ userId, tier, periodDays = 30 }) {
if (!userId) throw new Error("extendUserTier: userId required"); if (!userId) throw new Error("extendUserTier: userId required");
const t = tier === "pro" || tier === "max" ? tier : "core"; const t = tier === "pro" || tier === "max" ? tier : "core";
@@ -775,11 +787,15 @@ export async function extendUserTier({ userId, tier, periodDays = 30 }) {
const curExp = row.subscription_expires_at const curExp = row.subscription_expires_at
? new Date(row.subscription_expires_at).getTime() ? new Date(row.subscription_expires_at).getTime()
: 0; : 0;
const hasActiveSub =
Number.isFinite(curExp) &&
curExp > now &&
(row.tier_snapshot === "pro" || row.tier_snapshot === "max");
const base = Math.max(now, Number.isFinite(curExp) ? curExp : 0); const base = Math.max(now, Number.isFinite(curExp) ? curExp : 0);
const expiresAt = new Date( const expiresAt = new Date(
base + periodDays * 24 * 60 * 60 * 1000, base + periodDays * 24 * 60 * 60 * 1000,
).toISOString(); ).toISOString();
return setUserTier({ userId, tier: t, expiresAt }); return setUserTier({ userId, tier: t, expiresAt, resetCycle: !hasActiveSub });
} }
// Read a cloud user's credit row (creates a blank Core row if none yet). // Read a cloud user's credit row (creates a blank Core row if none yet).
+17 -2
View File
@@ -23,10 +23,25 @@
// A route bills against `creditKey`; for tier it uses the stored row tier // A route bills against `creditKey`; for tier it uses the stored row tier
// (cloud) or `license.tier` (license) — see identityTier(). // (cloud) or `license.tier` (license) — see identityTier().
import { timingSafeEqual } from "crypto";
import { getConfigSnapshot } from "./config.js"; import { getConfigSnapshot } from "./config.js";
import { resolveLicense } from "./keysat-client.js"; import { resolveLicense } from "./keysat-client.js";
import { getCreditKey } from "./credits.js"; import { getCreditKey } from "./credits.js";
// Constant-time string compare for the shared operator key, so a token
// guess can't be tuned byte-by-byte off response timing. Mirrors the
// helper in admin-auth.js. A length mismatch returns false early —
// length isn't the secret, and timingSafeEqual requires equal length.
function constantTimeEqual(a, b) {
if (typeof a !== "string" || typeof b !== "string") return false;
if (a.length !== b.length) return false;
try {
return timingSafeEqual(Buffer.from(a), Buffer.from(b));
} catch {
return false;
}
}
// user-ids come from Recaps (base64url / hex account ids). Constrain the // user-ids come from Recaps (base64url / hex account ids). Constrain the
// charset so a header can't smuggle a path-ish or oversized key into the // charset so a header can't smuggle a path-ish or oversized key into the
// ledger. // ledger.
@@ -40,7 +55,7 @@ export async function resolveIdentity(req) {
const cfg = await getConfigSnapshot(); const cfg = await getConfigSnapshot();
const expected = (cfg.relay_cloud_operator_key || "").trim(); const expected = (cfg.relay_cloud_operator_key || "").trim();
const presented = (req.header("X-Recap-Operator-Key") || "").trim(); const presented = (req.header("X-Recap-Operator-Key") || "").trim();
if (!expected || !presented || presented !== expected) { if (!expected || !presented || !constantTimeEqual(presented, expected)) {
const e = new Error( const e = new Error(
"X-Recap-User-Id requires a valid X-Recap-Operator-Key" "X-Recap-User-Id requires a valid X-Recap-Operator-Key"
); );
@@ -81,7 +96,7 @@ export async function verifyOperatorKey(req) {
const cfg = await getConfigSnapshot(); const cfg = await getConfigSnapshot();
const expected = (cfg.relay_cloud_operator_key || "").trim(); const expected = (cfg.relay_cloud_operator_key || "").trim();
const presented = (req.header("X-Recap-Operator-Key") || "").trim(); const presented = (req.header("X-Recap-Operator-Key") || "").trim();
return !!expected && !!presented && presented === expected; return !!expected && !!presented && constantTimeEqual(presented, expected);
} }
// The tier to bill/quota at for a resolved identity. // The tier to bill/quota at for a resolved identity.
+16
View File
@@ -105,6 +105,22 @@ app.get("/", (_req, res) => {
res.redirect("/dashboard.html"); res.redirect("/dashboard.html");
}); });
// Error handler (must be last). Without it, body-parser parse failures
// and any other propagated error fall through to Express's default
// handler, which renders an HTML page including the local-filesystem
// stack trace — an info leak. Return clean JSON instead.
app.use((err, _req, res, next) => {
if (res.headersSent) return next(err); // mid-stream (SSE) — can't rewrite
if (err?.type === "entity.parse.failed") {
return res.status(400).json({ error: "invalid JSON body" });
}
if (err?.type === "entity.too.large") {
return res.status(413).json({ error: "request body too large" });
}
console.error("[relay] unhandled error:", err?.message || err);
return res.status(500).json({ error: "internal error" });
});
const HOSTNAME = process.env.HOSTNAME || "0.0.0.0"; const HOSTNAME = process.env.HOSTNAME || "0.0.0.0";
app.listen(PORT, HOSTNAME, () => { app.listen(PORT, HOSTNAME, () => {
console.log(`[relay] listening on http://${HOSTNAME}:${PORT}`); console.log(`[relay] listening on http://${HOSTNAME}:${PORT}`);
+1442
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -12,7 +12,7 @@
"cors": "^2.8.5", "cors": "^2.8.5",
"cookie-parser": "^1.4.6", "cookie-parser": "^1.4.6",
"express": "^4.21.0", "express": "^4.21.0",
"multer": "^1.4.5-lts.1", "multer": "^2.0.1",
"undici": "^6.21.0" "undici": "^6.21.0"
} }
} }
+15 -3
View File
@@ -77,19 +77,31 @@ async function ensureMeetingsDir(dataDir) {
await fs.mkdir(meetingsDir(dataDir), { recursive: true }).catch(() => {}); await fs.mkdir(meetingsDir(dataDir), { recursive: true }).catch(() => {});
} }
// Build the on-disk path for a meeting record, sanitizing the id so a
// caller-supplied :id can't traverse out of internal-meetings/. Real
// ids are UUIDs; anything outside [A-Za-z0-9_-] is stripped (mirrors
// output-store.js's pathFor). Throws when the id sanitizes to empty —
// load/delete catch it (→ 404 / no-op); save only ever gets a freshly
// minted id.
export function meetingPath(dataDir, id) {
const safe = String(id || "").replace(/[^A-Za-z0-9_-]/g, "");
if (!safe) throw new Error("invalid meeting id");
return path.join(meetingsDir(dataDir), `${safe}.json`);
}
// ─── Storage layer ────────────────────────────────────────────────── // ─── Storage layer ──────────────────────────────────────────────────
async function saveMeeting(dataDir, id, record) { async function saveMeeting(dataDir, id, record) {
await ensureMeetingsDir(dataDir); await ensureMeetingsDir(dataDir);
const filePath = path.join(meetingsDir(dataDir), `${id}.json`); const filePath = meetingPath(dataDir, id);
await fs.writeFile(filePath, JSON.stringify(record, null, 2), { await fs.writeFile(filePath, JSON.stringify(record, null, 2), {
mode: 0o600, mode: 0o600,
}); });
} }
async function loadMeeting(dataDir, id) { async function loadMeeting(dataDir, id) {
const filePath = path.join(meetingsDir(dataDir), `${id}.json`);
try { try {
const filePath = meetingPath(dataDir, id);
const raw = await fs.readFile(filePath, "utf8"); const raw = await fs.readFile(filePath, "utf8");
const rec = JSON.parse(raw); const rec = JSON.parse(raw);
// Retroactive chunk-contiguity backfill must run BEFORE the // Retroactive chunk-contiguity backfill must run BEFORE the
@@ -239,8 +251,8 @@ async function listMeetings(dataDir) {
} }
async function deleteMeeting(dataDir, id) { async function deleteMeeting(dataDir, id) {
const filePath = path.join(meetingsDir(dataDir), `${id}.json`);
try { try {
const filePath = meetingPath(dataDir, id);
await fs.unlink(filePath); await fs.unlink(filePath);
return true; return true;
} catch { } catch {
+7 -2
View File
@@ -50,6 +50,7 @@ import { calcGeminiCost } from "../pricing.js";
import { getAudioDurationSeconds } from "../audio-meta.js"; import { getAudioDurationSeconds } from "../audio-meta.js";
import { resolveHardwareConfig } from "../hardware-config.js"; import { resolveHardwareConfig } from "../hardware-config.js";
import { reportHealthEvent } from "../spark-control-events.js"; import { reportHealthEvent } from "../spark-control-events.js";
import { safeFetch } from "../safe-url.js";
import { import {
createJob, createJob,
markRunning, markRunning,
@@ -97,8 +98,12 @@ function guessMimeFromExt(filePath) {
// would exceed MAX_DOWNLOAD_BYTES. Returns { filePath, bytes, // would exceed MAX_DOWNLOAD_BYTES. Returns { filePath, bytes,
// mimeType }. // mimeType }.
export async function downloadDirect(url, tmpDir) { export async function downloadDirect(url, tmpDir) {
const res = await fetch(url, { // safeFetch is the SSRF choke point: it rejects non-http(s) schemes
redirect: "follow", // and hosts resolving to private/reserved ranges, and re-validates
// every redirect hop. downloadDirect is the single download path for
// transcribe-url / summarize-url / admin-test-run, so guarding it
// here covers all three.
const res = await safeFetch(url, {
signal: AbortSignal.timeout(DOWNLOAD_TIMEOUT_MS), signal: AbortSignal.timeout(DOWNLOAD_TIMEOUT_MS),
}); });
if (!res.ok) { if (!res.ok) {
+158
View File
@@ -0,0 +1,158 @@
// SSRF guard for user-supplied media URLs.
//
// /relay/transcribe-url and /relay/summarize-url download whatever
// `media_url` the caller passes, and the route is reachable by anyone
// presenting a self-chosen X-Recap-Install-Id, so an unguarded fetch
// lets a caller probe the operator's LAN (Spark Control, BTCPay, other
// StartOS services) or cloud metadata at 169.254.169.254. This module
// rejects non-http(s) schemes and any hostname that resolves to a
// private / loopback / link-local / reserved address, and follows
// redirects MANUALLY so every hop is re-validated — a public URL can
// 302 to an internal one after the first check passes.
//
// LAN calls to the operator's OWN hardware go through lan-fetch.js
// instead: those URLs are config-set, not caller-set, and intentionally
// reach private hosts.
import dns from "node:dns/promises";
import net from "node:net";
export class BlockedUrlError extends Error {
constructor(message) {
super(message);
this.name = "BlockedUrlError";
this.code = "BLOCKED_URL";
}
}
// Parse an IPv4 dotted-quad into its 32-bit integer, or null if it
// isn't a well-formed IPv4 literal.
function ipv4ToInt(ip) {
const parts = ip.split(".");
if (parts.length !== 4) return null;
let n = 0;
for (const p of parts) {
if (!/^\d{1,3}$/.test(p)) return null;
const v = Number(p);
if (v > 255) return null;
n = n * 256 + v;
}
return n >>> 0;
}
function inV4Range(n, base, bits) {
const mask = bits === 0 ? 0 : (~((1 << (32 - bits)) - 1)) >>> 0;
return (n & mask) === (base & mask);
}
// IPv4 ranges that must never be fetched from a user-supplied URL.
const BLOCKED_V4 = [
["0.0.0.0", 8], // "this host"
["10.0.0.0", 8], // private
["100.64.0.0", 10], // CGNAT
["127.0.0.0", 8], // loopback
["169.254.0.0", 16], // link-local (incl. 169.254.169.254 cloud metadata)
["172.16.0.0", 12], // private
["192.0.0.0", 24], // IETF protocol assignments
["192.0.2.0", 24], // TEST-NET-1
["192.168.0.0", 16], // private
["198.18.0.0", 15], // benchmarking
["198.51.100.0", 24], // TEST-NET-2
["203.0.113.0", 24], // TEST-NET-3
["224.0.0.0", 4], // multicast
["240.0.0.0", 4], // reserved (incl. 255.255.255.255 broadcast)
];
function isBlockedV4(ip) {
const n = ipv4ToInt(ip);
if (n === null) return false;
for (const [base, bits] of BLOCKED_V4) {
if (inV4Range(n, ipv4ToInt(base), bits)) return true;
}
return false;
}
// Classify the reserved/private IPv6 ranges we block. Handles
// IPv4-mapped (::ffff:a.b.c.d) by delegating to the v4 check.
function isBlockedV6(ip) {
let addr = ip.toLowerCase();
const pct = addr.indexOf("%"); // strip zone id (fe80::1%eth0)
if (pct !== -1) addr = addr.slice(0, pct);
// IPv4-mapped / -embedded (::ffff:192.168.0.1, ::192.168.0.1).
const tail = addr.slice(addr.lastIndexOf(":") + 1);
if (tail.includes(".") && isBlockedV4(tail)) return true;
if (addr === "::1") return true; // loopback
if (addr === "::") return true; // unspecified
// fe80::/10 link-local spans fe80febf.
if (/^fe[89ab]/.test(addr)) return true;
if (/^f[cd]/.test(addr)) return true; // fc00::/7 unique-local
if (addr.startsWith("ff")) return true; // multicast
return false;
}
// True if `ip` (an IP literal) is in a range we refuse to fetch from.
// Returns false for non-IP strings — the caller resolves DNS first.
export function isBlockedAddress(ip) {
const kind = net.isIP(ip);
if (kind === 4) return isBlockedV4(ip);
if (kind === 6) return isBlockedV6(ip);
return false;
}
// Validate that `urlStr` is an http(s) URL whose host does NOT resolve
// to a private/reserved address. Throws BlockedUrlError otherwise;
// returns the parsed URL on success.
export async function assertPublicHttpUrl(urlStr) {
let u;
try {
u = new URL(urlStr);
} catch {
throw new BlockedUrlError("media_url is not a valid URL");
}
if (u.protocol !== "http:" && u.protocol !== "https:") {
throw new BlockedUrlError(`media_url scheme "${u.protocol}" is not allowed`);
}
const host = u.hostname.replace(/^\[|\]$/g, ""); // strip IPv6 brackets
let addresses;
if (net.isIP(host)) {
addresses = [host];
} else {
let looked;
try {
looked = await dns.lookup(host, { all: true });
} catch {
throw new BlockedUrlError(`media_url host "${host}" did not resolve`);
}
addresses = looked.map((a) => a.address);
}
if (!addresses.length) {
throw new BlockedUrlError(`media_url host "${host}" did not resolve`);
}
for (const addr of addresses) {
if (isBlockedAddress(addr)) {
throw new BlockedUrlError(
`media_url host "${host}" resolves to a blocked address`,
);
}
}
return u;
}
// fetch() wrapper that re-validates the URL on every redirect hop. Node
// fetch's redirect:"follow" would jump to an internal host AFTER the
// initial check passed, so we follow manually with redirect:"manual"
// and re-run assertPublicHttpUrl on each Location.
export async function safeFetch(urlStr, { signal, headers, maxRedirects = 5 } = {}) {
let current = urlStr;
for (let hop = 0; hop <= maxRedirects; hop++) {
await assertPublicHttpUrl(current);
const res = await fetch(current, { redirect: "manual", signal, headers });
const location = res.headers.get("location");
if (res.status >= 300 && res.status < 400 && location) {
current = new URL(location, current).toString(); // resolve relative redirects
continue;
}
return res;
}
throw new BlockedUrlError("media_url exceeded the redirect limit");
}
+34
View File
@@ -0,0 +1,34 @@
// Path-traversal guard for meeting record ids (internal-meetings.js
// meetingPath). A caller-supplied :id must never escape the
// internal-meetings/ directory.
import { test, describe } from "node:test";
import assert from "node:assert/strict";
import path from "node:path";
import { meetingPath } from "../routes/internal-meetings.js";
const DATA = "/data";
const DIR = path.join(DATA, "internal-meetings");
describe("meetingPath", () => {
test("a normal UUID id maps into internal-meetings/", () => {
const id = "2f1c9b3a-0e4d-4a77-9d2a-abc123def456";
assert.equal(meetingPath(DATA, id), path.join(DIR, `${id}.json`));
});
test("traversal-shaped ids are sanitized and stay inside the dir", () => {
for (const id of ["../../etc/passwd", "../../../root/.ssh/id", "..%2f..%2fx", "a/b/c", "....//x"]) {
const p = meetingPath(DATA, id);
const rel = path.relative(DIR, p);
assert.ok(!rel.startsWith(".."), `${id} escaped to ${p}`);
assert.ok(!p.includes(".."), `${id} left ".." in ${p}`);
assert.ok(p.endsWith(".json"));
}
});
test("an id that sanitizes to empty throws (load/delete catch → 404 / no-op)", () => {
for (const id of ["", null, undefined, "/", "../", "...", "!!!"]) {
assert.throws(() => meetingPath(DATA, id), /invalid meeting id/);
}
});
});
+90
View File
@@ -0,0 +1,90 @@
// SSRF guard for user-supplied media URLs (safe-url.js). Uses literal
// IPs so the address checks need no DNS / network.
import { test, describe } from "node:test";
import assert from "node:assert/strict";
import {
isBlockedAddress,
assertPublicHttpUrl,
BlockedUrlError,
} from "../safe-url.js";
describe("isBlockedAddress", () => {
test("blocks private / loopback / link-local / reserved IPv4", () => {
for (const ip of [
"127.0.0.1",
"10.0.0.5",
"172.16.0.1",
"172.31.255.255",
"192.168.1.1",
"169.254.169.254", // cloud metadata
"100.64.0.1",
"0.0.0.0",
"198.18.0.1",
"224.0.0.1",
"255.255.255.255",
]) {
assert.equal(isBlockedAddress(ip), true, `${ip} should be blocked`);
}
});
test("allows public IPv4 (incl. the /12 boundaries around 172.16/12)", () => {
for (const ip of ["8.8.8.8", "1.1.1.1", "172.15.0.1", "172.32.0.1", "93.184.216.34"]) {
assert.equal(isBlockedAddress(ip), false, `${ip} should be allowed`);
}
});
test("blocks loopback / ULA / link-local / IPv4-mapped IPv6", () => {
for (const ip of [
"::1",
"::",
"fe80::1",
"febf::1",
"fc00::1",
"fd12:3456::1",
"ff02::1",
"::ffff:127.0.0.1",
"::ffff:192.168.0.1",
]) {
assert.equal(isBlockedAddress(ip), true, `${ip} should be blocked`);
}
});
test("allows public IPv6", () => {
assert.equal(isBlockedAddress("2606:4700:4700::1111"), false);
});
});
describe("assertPublicHttpUrl", () => {
test("rejects non-http(s) schemes", async () => {
for (const u of [
"file:///etc/passwd",
"gopher://x/_",
"ftp://h/f",
"data:text/plain,hi",
]) {
await assert.rejects(() => assertPublicHttpUrl(u), BlockedUrlError);
}
});
test("rejects literal private / metadata IP hosts (no DNS needed)", async () => {
for (const u of [
"http://127.0.0.1/x",
"http://169.254.169.254/latest/meta-data/",
"http://[::1]/x",
"http://192.168.0.10:9000/a",
"https://10.1.2.3/audio.mp3",
]) {
await assert.rejects(() => assertPublicHttpUrl(u), BlockedUrlError);
}
});
test("rejects malformed URLs", async () => {
await assert.rejects(() => assertPublicHttpUrl("not a url"), BlockedUrlError);
});
test("allows a public literal IP host", async () => {
const u = await assertPublicHttpUrl("https://8.8.8.8/audio.mp3");
assert.equal(u.hostname, "8.8.8.8");
});
});
+22
View File
@@ -76,4 +76,26 @@ describe("extendUserTier (prepaid periods)", () => {
const days = (new Date(renewed.subscription_expires_at).getTime() - Date.now()) / DAY; const days = (new Date(renewed.subscription_expires_at).getTime() - Date.now()) / DAY;
assert.ok(days > 29.9 && days < 30.1, `fresh ~30 days from now, got ${days}`); assert.ok(days > 29.9 && days < 30.1, `fresh ~30 days from now, got ${days}`);
}); });
test("early renewal PRESERVES the monthly credit counter (no free reset)", async () => {
await extendUserTier({ userId: "u4", tier: "pro", periodDays: 30 });
const row = await getUserCreditRow("u4");
row.monthly_consumed = 7; // simulate credits already spent this cycle
row.monthly_gemini_consumed = 3;
// Pay early / renew while the subscription is still in force.
await extendUserTier({ userId: "u4", tier: "pro", periodDays: 30 });
const after = await getUserCreditRow("u4");
assert.equal(after.monthly_consumed, 7, "consumed credits must survive an early renewal");
assert.equal(after.monthly_gemini_consumed, 3);
});
test("resubscribing AFTER a lapse starts a fresh cycle (counter reset)", async () => {
await extendUserTier({ userId: "u5", tier: "pro", periodDays: 30 });
const row = await getUserCreditRow("u5");
row.monthly_consumed = 9;
row.subscription_expires_at = new Date(Date.now() - 5 * DAY).toISOString(); // lapsed
await extendUserTier({ userId: "u5", tier: "pro", periodDays: 30 });
const after = await getUserCreditRow("u5");
assert.equal(after.monthly_consumed, 0, "a lapsed resubscribe starts a clean cycle");
});
}); });