Compare commits

..

4 Commits

Author SHA1 Message Date
Keysat 5d0d1b2dd2 docs: note webhook-dedup module + processed-webhooks.json in AGENTS.md 2026-06-15 20:04:47 -05:00
Keysat 238689ddcc Persist payment-webhook dedup; declare BTCPay required; scope CORS
Replace the in-memory dedup Sets in the BTCPay and Zaprite webhook
handlers (and the BTCPay rescan path) with a persistent JSON-backed
store (server/webhook-dedup.js). The in-memory sets were cleared on
restart, so a duplicate webhook delivery straddling a relay restart
could double-credit (BTCPay) or double-extend a subscription (Zaprite).
The store atomically writes /data/processed-webhooks.json, namespaces
keys per rail (storeId|invoiceId vs zaprite:orderId), and prunes
entries older than 180 days (safely beyond any retry window).

Also:
- BTCPay is a required running dependency (operator decision). Config
  was already optional:false/kind:'running'; corrected the contradictory
  "optional" comment in the manifest to match.
- Scope cors() to /relay/* only — off /admin/* and the same-origin
  dashboard, which don't need permissive CORS.
- Add money-path unit tests (commitCredit/refundCredit/applyTierPromotion)
  and webhook-dedup tests (incl. the survives-a-restart guarantee).
- Fix two AGENTS.md auth-doc drifts; refresh Current state.

Version 0.2.125 -> 0.2.126.
2026-06-15 18:15:00 -05:00
Keysat 798a698132 Add Users dashboard tab with per-user balances and credit grants
New cookie-gated "Users" tab on the operator dashboard: a sortable view
of every credit-ledger row (typed cloud/license/install) with computed
remaining/total balances, key filter, and a per-row "grant free credits"
action.

Endpoints (routes/admin.js):
- GET /admin/credits — snapshotAll() enriched with a type derived from
  the credit-key prefix and a computed balance (computeRemaining against
  live tier quotas), since the ledger stores consumed counters only.
- POST /admin/credits/grant {credit_key, amount} — adds free top-up via
  addPurchasedCredits. Grants land in the never-expires purchased bucket
  (spent after the tier allowance). Guards: positive integer, <=1,000,000,
  and the row must already exist (a typo can't spawn a ghost row).

Admin-only; no /relay/* client contract change. Tests added in
server/test/admin-credits.test.js (mount the real router over HTTP).
Version bumped 0.2.124 -> 0.2.125.
2026-06-15 16:25:14 -05:00
Keysat 00da92a872 docs: note Gitea remote in Current state 2026-06-15 12:26:56 -05:00
14 changed files with 720 additions and 62 deletions
+17 -15
View File
@@ -8,10 +8,10 @@ Operator-side, credit-metered service that sits in front of Gemini and the opera
## Stack ## Stack
- **Server**: Node.js (`type: module`, ES modules). Same dev box as the app (`v25.6.1`); container runtime is whatever the `Dockerfile` pins. - **Server**: Node.js (`type: module`, ES modules). Same dev box as the app (`v25.6.1`); container runtime is whatever the `Dockerfile` pins.
- **HTTP**: `express` + `multer` (audio upload). Admin routes under `/admin/*` behind an admin-session-cookie gate; relay-to-relay routes under `/relay/*` behind the operator key. - **HTTP**: `express` + `multer` (audio upload). Admin routes under `/admin/*` behind an admin-session-cookie gate. `/relay/*` uses per-call header auth — install-id/license, or operator-key + user-id for the cloud control plane (a few routes like `health`/`policy`/`capabilities` are public). See the Auth model under Endpoints. `cors()` is scoped to `/relay/*` only.
- **Dashboard**: `public/dashboard.html` — single-file vanilla JS, render-string-into-innerHTML, same shape as the app's `index.html`. - **Dashboard**: `public/dashboard.html` — single-file vanilla JS, render-string-into-innerHTML, same shape as the app's `index.html`.
- **Packaging**: `@start9labs/start-sdk` under `startos/` — version graph at `startos/versions/index.ts`. - **Packaging**: `@start9labs/start-sdk` under `startos/` — version graph at `startos/versions/index.ts`.
- **Storage**: filesystem under the StartOS data dir (`/data`). Internal meetings persist as `/data/internal-meetings/<id>.json`. No SQLite here. - **Storage**: filesystem under the StartOS data dir (`/data`). No SQLite — flat JSON files: credit ledger `/data/credits.json`, payment-webhook dedup `/data/processed-webhooks.json`, internal meetings `/data/internal-meetings/<id>.json`.
- **Upstreams**: Gemini (`@google/genai`); operator hardware via "Spark Control" HTTP (Parakeet transcribe, `/api/audio/diarize-chunk` for Sortformer+TitaNet, a vLLM/Gemma OpenAI-shape analyze endpoint). - **Upstreams**: Gemini (`@google/genai`); operator hardware via "Spark Control" HTTP (Parakeet transcribe, `/api/audio/diarize-chunk` for Sortformer+TitaNet, a vLLM/Gemma OpenAI-shape analyze endpoint).
## Commands ## Commands
@@ -47,8 +47,10 @@ server/
config.js getConfigSnapshot() + relay_* config defaults config.js getConfigSnapshot() + relay_* config defaults
hardware-config.js resolveHardwareConfig() → Spark Control endpoint discovery hardware-config.js resolveHardwareConfig() → Spark Control endpoint discovery
safe-url.js SSRF guard: assertPublicHttpUrl + safeFetch for caller-supplied URLs safe-url.js SSRF guard: assertPublicHttpUrl + safeFetch for caller-supplied URLs
webhook-dedup.js persistent payment-webhook dedup (BTCPay + Zaprite share it);
initWebhookDedup/isWebhookProcessed/markWebhookProcessed
test/ node --test *.test.js (speaker tools, billing/credits, SSRF, path-traversal, …) test/ node --test *.test.js (speaker tools, billing/credits, SSRF, path-traversal, …)
public/dashboard.html operator dashboard (meetings detail view + speaker tools) public/dashboard.html operator dashboard (Overview / Jobs / Users / Internal Meetings / Settings)
startos/versions/<vN>.ts one file per version + index.ts graph startos/versions/<vN>.ts one file per version + index.ts graph
docs/issues-backlog.md detailed issue log docs/issues-backlog.md detailed issue log
docs/guides/internal-meetings.md diarization / speaker subsystem guide (path-scoped; lazy-loads via .claude/rules/) docs/guides/internal-meetings.md diarization / speaker subsystem guide (path-scoped; lazy-loads via .claude/rules/)
@@ -62,7 +64,7 @@ All routes mount in `server/index.js`. Public paths sit under `/relay/*`; operat
- **`X-Recap-Operator-Key`** + **`X-Recap-User-Id`** → "cloud" path. The Recaps cloud server (`recaps.cc`) authenticates once with a shared operator key (`relay_cloud_operator_key`) and names the acting user. Credit pool keyed `user:<id>`, tier comes from the relay's stored row, NOT a per-user license. See `server/identity.js`. - **`X-Recap-Operator-Key`** + **`X-Recap-User-Id`** → "cloud" path. The Recaps cloud server (`recaps.cc`) authenticates once with a shared operator key (`relay_cloud_operator_key`) and names the acting user. Credit pool keyed `user:<id>`, tier comes from the relay's stored row, NOT a per-user license. See `server/identity.js`.
- **`X-Recap-Install-Id`** (+ optional `Authorization: <license>`) → "license" path. Self-hosted installs and the operator's single-mode app. Credits/tier come from the resolved Keysat license + install id. - **`X-Recap-Install-Id`** (+ optional `Authorization: <license>`) → "license" path. Self-hosted installs and the operator's single-mode app. Credits/tier come from the resolved Keysat license + install id.
- **Admin session cookie** → `/admin/*`. Cookie issued by `POST /admin/login`; `/admin/login` and `/admin/status` are exempt inside `setupAdminAuthMiddleware`. - **Admin session cookie** → `/admin/*`. Cookie issued by `POST /admin/login`; `/admin/login`, `/admin/status`, and `/admin/btcpay/callback` are exempt inside `setupAdminAuthMiddleware`.
- **Webhook signature** → `POST /relay/btcpay/webhook` validates `BTCPay-Sig` against `relay_btcpay_webhook_secret`. Zaprite's webhook re-fetches the order through the Zaprite API to verify, so no shared-secret signing. - **Webhook signature** → `POST /relay/btcpay/webhook` validates `BTCPay-Sig` against `relay_btcpay_webhook_secret`. Zaprite's webhook re-fetches the order through the Zaprite API to verify, so no shared-secret signing.
- **`X-Recap-Job-Id`** is a billing key, not auth: the first call with a given id charges one credit; later calls with the same id are free (so transcribe + analyze for one summary = one credit total). - **`X-Recap-Job-Id`** is a billing key, not auth: the first call with a given id charges one credit; later calls with the same id are free (so transcribe + analyze for one summary = one credit total).
@@ -94,7 +96,7 @@ All require a valid `X-Recap-Operator-Key`. Defined in `routes/user-tier.js`.
### `/admin/*` (operator dashboard; cookie-gated) ### `/admin/*` (operator dashboard; cookie-gated)
`routes/admin.js`: `GET /admin/{usage,config,license-cache,hardware-queue,jobs,jobs-history,job-output/:id,job/:id/details,output-store-stats,output-store-ids,dashboard,dashboard.csv,settings}`, `POST /admin/{quotas,wipe-all,settings/promote-prompt}`, `PUT /admin/settings`, `DELETE /admin/job-outputs`. `routes/admin-test-run.js`: `POST /admin/{test-run,test-run-suite}`. BTCPay setup wizard under `/admin/btcpay/*` (`routes/btcpay-setup.js`). `routes/admin.js`: `GET /admin/{usage,credits,config,license-cache,hardware-queue,jobs,jobs-history,job-output/:id,job/:id/details,output-store-stats,output-store-ids,dashboard,dashboard.csv,settings}`, `POST /admin/{quotas,credits/grant,wipe-all,settings/promote-prompt}`, `PUT /admin/settings`, `DELETE /admin/job-outputs`. (`GET /admin/credits` = ledger rows enriched with type + computed balance for the dashboard Users tab; `POST /admin/credits/grant` `{ credit_key, amount }` adds free top-up credits to an existing row.) `routes/admin-test-run.js`: `POST /admin/{test-run,test-run-suite}`. BTCPay setup wizard under `/admin/btcpay/*` (`routes/btcpay-setup.js`).
### `/admin/internal-meetings/*` (cookie-gated; `routes/internal-meetings.js`) ### `/admin/internal-meetings/*` (cookie-gated; `routes/internal-meetings.js`)
@@ -141,14 +143,14 @@ this. When unsure whether a change is contract-affecting, assume it is and check
- **Never edit a `startos/versions/<v>.ts` that's already been built/installed** — add a new version file. - **Never edit a `startos/versions/<v>.ts` that's already been built/installed** — add a new version file.
- **Don't push to GitHub by default** — remote is self-hosted Gitea. - **Don't push to GitHub by default** — remote is self-hosted Gitea.
## Current state — post-eval security pass landed (2026-06-13) ## Current state — Users tab + webhook-dedup/P2 batch landed (2026-06-15)
- **Box, local tree, git aligned at relay `0.2.124`** (app `0.2.155`); `current: v_0_2_124`. Git is local-only (no remote). Working tree clean. **Suite green at 60 tests** (`cd server && npm test`); server boots clean. - **Box, local tree, git aligned at relay `0.2.126`** (app `0.2.155`); `current: v_0_2_126`. Gitea remote `origin` (`ssh://git@immense-voyage.local:59916/grant/recap-relay.git`); `master` tracks `origin/master`. Working tree clean. **Suite green at 79 tests** (`cd server && npm test`); server boots clean.
- **Full independent eval done** (evaluator + security-auditor + exerciser + doc-auditor + start9-spec-checker) → `EVALUATION.md` (overwritten in place each run, so re-running diffs cleanly). - **Users dashboard tab** (`0.2.125`): new cookie-gated tab — every credit-ledger row (typed cloud/license/install) with computed remaining/total balances, key filter, and a per-row "grant free credits" action. `GET /admin/credits` (enriched read) + `POST /admin/credits/grant {credit_key, amount}` (free top-up via `addPurchasedCredits`, guards: positive int ≤1M, must be an existing row). Admin-only; no `../recap` contract change.
- **All P0/P1 fixed** this session (commits `8ad7c54`/`d2caa98`/`3a601e1`): SSRF guard on caller-supplied media URLs (new `server/safe-url.js`), the early-renewal credit-reset money-leak (`extendUserTier`/`setUserTier` `resetCycle`), and the `multer``^2.0.1` DoS bump. None touch the `../recap` client contract. - **Webhook dedup now persistent** (`0.2.126`): new `server/webhook-dedup.js` (JSON store at `/data/processed-webhooks.json`, atomic writes, 180-day prune) replaces the in-memory Sets in `routes/credits.js` + `zaprite-webhook.js` (and the rescan path) — a duplicate delivery straddling a restart can no longer double-credit/double-extend. Keys namespaced `<storeId>|<invoiceId>` vs `zaprite:<orderId>`.
- **Three P2 fixed** (commits `cbd9748`/`da1bba2`/`693d724`): meeting-`:id` path-traversal guard (`meetingPath()`), constant-time operator-key compare, and a JSON error handler that closes the malformed-body stack-trace leak. - **BTCPay is REQUIRED** (operator decision, 2026-06-15): config was already `optional:false`/`kind:'running'`; corrected the contradictory "optional" comment in `startos/manifest/index.ts`. It's the only paid rail, so the relay shouldn't run without it.
- **Next (open P2), in priority order:** - **CORS scoped to `/relay/*`** (`index.js`) — off `/admin/*` + dashboard (same-origin). Plus money-path unit tests (`commitCredit`/`refundCredit`/`applyTierPromotion`) and the two AGENTS.md auth-doc drift fixes.
1. Persist webhook dedup so a restart can't double-credit/double-extend — `routes/credits.js:63`, `zaprite-webhook.js:27`. - **Next (open P2 / deferred):**
2. **Needs operator decision:** is BTCPay a hard requirement or truly optional? It's `optional:false`/`kind:'running'` despite "optional" comments, so StartOS won't start the relay without BTCPay co-installed — `startos/manifest/index.ts:38-49` + `dependencies.ts`. Then make manifest/deps/comment agree. 1. Split the 2225-line `routes/internal-meetings.js`**deferred as likely overkill** for a private service; do only if it becomes painful to work in.
3. Money-path unit tests (`commitCredit`/`refundCredit`/`applyTierPromotion`/grant handlers); scope `cors()` off `/admin/*` (`index.js`); split the 2225-line `routes/internal-meetings.js`; fix the two AGENTS.md auth-doc drifts (Stack-line `/relay/*` auth; missing `/admin/btcpay/callback` exempt path). 2. P3+ deferred tail (no `/relay/*` rate limiting, container-as-root, dashboard `innerHTML` XSS surface, prune 126 version files, `/relay/health` stale `0.2.11`, etc.) + speaker-tool/empty-section backlog → `ROADMAP.md` / `docs/issues-backlog.md`.
- **Risks/notes:** SSRF guard leaves a DNS-rebinding TOCTOU open (acceptable for a private box; revisit if exposed). P3+ deferred tail + pre-existing speaker-tool/empty-section backlog → `ROADMAP.md` / `docs/issues-backlog.md`. - **Risks/notes:** webhook dedup keeps the pre-existing check-then-mark race for *truly simultaneous* duplicate deliveries (vanishingly rare on a private box; would need locking). SSRF guard leaves a DNS-rebinding TOCTOU open (acceptable for a private box). Full prior eval → `EVALUATION.md`.
+179 -1
View File
@@ -1044,10 +1044,15 @@
activeTab: (() => { activeTab: (() => {
try { try {
const saved = localStorage.getItem("recap-relay-active-tab"); const saved = localStorage.getItem("recap-relay-active-tab");
if (saved === "jobs" || saved === "settings" || saved === "overview") return saved; if (saved === "jobs" || saved === "settings" || saved === "overview" || saved === "users") return saved;
} catch {} } catch {}
return "overview"; return "overview";
})(), })(),
// Users-tab state — enriched credit-ledger rows from /admin/credits.
creditsData: null,
creditsLoading: false,
creditsSort: { col: "last_active_at", dir: "desc" },
creditsQuery: "",
// Jobs-tab state. // Jobs-tab state.
jobsData: null, jobsData: null,
jobsLoading: false, jobsLoading: false,
@@ -1176,6 +1181,9 @@
if (state.authed && state.activeTab === "meetings" && !state.meetingsList) { if (state.authed && state.activeTab === "meetings" && !state.meetingsList) {
loadMeetingsList(); loadMeetingsList();
} }
if (state.authed && state.activeTab === "users" && !state.creditsData) {
loadCredits();
}
// Start the Overview auto-refresh poll on boot regardless of // Start the Overview auto-refresh poll on boot regardless of
// current tab — cheap (one fetch every 10s) and ensures the // current tab — cheap (one fetch every 10s) and ensures the
// tab is current the moment the operator switches to it. // tab is current the moment the operator switches to it.
@@ -1460,6 +1468,8 @@
renderSettingsTab(); renderSettingsTab();
} else if (state.activeTab === "meetings") { } else if (state.activeTab === "meetings") {
renderMeetingsTab(); renderMeetingsTab();
} else if (state.activeTab === "users") {
renderUsersTab();
} else { } else {
renderDashboard(); renderDashboard();
} }
@@ -1471,6 +1481,7 @@
return '<div class="tabs">' + return '<div class="tabs">' +
t("overview", "Overview") + t("overview", "Overview") +
t("jobs", "Jobs") + t("jobs", "Jobs") +
t("users", "Users") +
t("meetings", "Internal Meetings") + t("meetings", "Internal Meetings") +
t("settings", "Settings") + t("settings", "Settings") +
'</div>'; '</div>';
@@ -1561,6 +1572,9 @@
if (tab === "meetings") { if (tab === "meetings") {
loadMeetingsList(); loadMeetingsList();
} }
if (tab === "users") {
loadCredits();
}
// Overview tab: re-fetch the dashboard data on EVERY entry (so // Overview tab: re-fetch the dashboard data on EVERY entry (so
// the summaries / errors / perf tables show current state, not // the summaries / errors / perf tables show current state, not
// whatever was last cached) AND start the 10-second auto-refresh // whatever was last cached) AND start the 10-second auto-refresh
@@ -6662,6 +6676,170 @@
'</div>'; '</div>';
} }
// ── Users tab ───────────────────────────────────────────────────
// A flat view of the credit ledger: every user/install + their
// current balance, with a per-row "grant free credits" action.
// Balances are COMPUTED server-side (/admin/credits) since the
// ledger stores consumed counters, not a remaining number.
async function loadCredits() {
state.creditsLoading = true;
if (state.activeTab === "users") render();
try {
const r = await fetch("/admin/credits", { cache: "no-store" });
if (!r.ok) throw new Error("HTTP " + r.status);
const data = await r.json();
state.creditsData = Array.isArray(data.rows) ? data.rows : [];
} catch (err) {
state.creditsData = [];
console.warn("loadCredits failed:", err);
}
state.creditsLoading = false;
if (state.activeTab === "users") render();
}
// Credits consumed in the current cycle: Core spends a lifetime
// budget, paid tiers spend a monthly one.
function usedCount(r) {
return (r.tier_snapshot === "pro" || r.tier_snapshot === "max")
? (r.monthly_consumed || 0)
: (r.lifetime_consumed || 0);
}
// null = unlimited tier (no cap). Render it as a word, not a number.
function fmtCredits(v) {
return v == null ? '<span class="dim">Unlimited</span>' : fmtInt(v);
}
const USERS_COLS = [
{ key: "credit_key", label: "Key", val: (r) => r.credit_key },
{ key: "type", label: "Type", val: (r) => r.type },
{ key: "tier_snapshot", label: "Tier", val: (r) => r.tier_snapshot || "core" },
{ key: "remaining", label: "Remaining", num: true, val: (r) => r.remaining == null ? Infinity : r.remaining },
{ key: "purchased", label: "Purchased", num: true, val: (r) => r.purchased || 0 },
{ key: "total", label: "Total", num: true, val: (r) => r.total == null ? Infinity : r.total },
{ key: "used", label: "Used (cycle)", num: true, val: (r) => usedCount(r) },
{ key: "last_active_at", label: "Last active", val: (r) => new Date(r.last_active_at || 0).getTime() },
];
function sortUsers(col) {
const s = state.creditsSort;
if (s.col === col) { s.dir = s.dir === "asc" ? "desc" : "asc"; }
else { s.col = col; s.dir = (col === "credit_key" || col === "type") ? "asc" : "desc"; }
refreshUsersTable();
}
function filterUsers(v) {
state.creditsQuery = v;
refreshUsersTable();
}
// Re-render ONLY the table (sort/filter) so the search box outside
// the wrap keeps focus + caret between keystrokes.
function refreshUsersTable() {
const wrap = document.getElementById("users-table-wrap");
if (wrap) wrap.innerHTML = usersTableHtml();
}
function usersTableHtml() {
const rows = (state.creditsData || []).slice();
const q = (state.creditsQuery || "").trim().toLowerCase();
const filtered = q
? rows.filter((r) => (r.credit_key || "").toLowerCase().includes(q))
: rows;
if (filtered.length === 0) {
return '<div class="empty">' + (q ? "No users match “" + esc(q) + "”." : "No users yet.") + '</div>';
}
const { col, dir } = state.creditsSort;
const colDef = USERS_COLS.find((c) => c.key === col) || USERS_COLS[0];
const mul = dir === "asc" ? 1 : -1;
filtered.sort((a, b) => {
const av = colDef.val(a), bv = colDef.val(b);
if (typeof av === "number" && typeof bv === "number") return (av - bv) * mul;
return String(av).localeCompare(String(bv)) * mul;
});
const thead = USERS_COLS.map((c) => {
const ind = c.key === col ? (dir === "asc" ? " ▲" : " ▼") : "";
return '<th class="' + (c.num ? "num " : "") + '" style="cursor:pointer;" onclick="sortUsers(\'' + c.key + '\')">'
+ esc(c.label) + '<span style="opacity:0.7;font-size:9px;">' + ind + '</span></th>';
}).join("") + '<th>Grant</th>';
const tbody = filtered.map((r) => {
const typeBadge = '<span style="display:inline-block;padding:1px 7px;border-radius:999px;font-size:10px;'
+ 'background:rgba(165,180,252,0.12);border:1px solid var(--line-2);color:var(--fg-dim);">'
+ esc(r.type) + '</span>';
const grantCell = '<div style="display:flex;gap:5px;align-items:center;">'
+ '<input type="number" min="1" step="1" placeholder="N" class="grant-amt" '
+ 'style="width:62px;padding:4px 6px;font-size:12px;background:var(--bg);border:1px solid var(--line-2);border-radius:5px;color:var(--fg);" />'
+ '<button class="grant-btn" data-key="' + esc(r.credit_key) + '" onclick="grantCredits(this)" '
+ 'style="padding:4px 10px;font-size:11px;background:transparent;border:1px solid var(--line-2);border-radius:5px;color:var(--accent);cursor:pointer;">Grant</button>'
+ '</div>';
return '<tr>'
+ '<td><code title="' + esc(r.credit_key) + '">' + esc(shortId(r.credit_key)) + '</code></td>'
+ '<td>' + typeBadge + '</td>'
+ '<td>' + tierPill(r.tier_snapshot) + '</td>'
+ '<td class="num">' + fmtCredits(r.remaining) + '</td>'
+ '<td class="num">' + fmtInt(r.purchased || 0) + '</td>'
+ '<td class="num">' + fmtCredits(r.total) + '</td>'
+ '<td class="num" title="' + (r.capped === "monthly" ? "monthly allowance" : "lifetime allowance") + '">' + fmtInt(usedCount(r)) + '</td>'
+ '<td><span class="dim">' + esc(fmtTs(r.last_active_at)) + '</span></td>'
+ '<td>' + grantCell + '</td>'
+ '</tr>';
}).join("");
return '<table><thead><tr>' + thead + '</tr></thead><tbody>' + tbody + '</tbody></table>';
}
async function grantCredits(btn) {
const key = btn.dataset.key;
const input = btn.parentElement.querySelector(".grant-amt");
const amount = parseInt(input && input.value, 10);
if (!Number.isInteger(amount) || amount <= 0) {
alert("Enter a positive whole number of credits.");
return;
}
if (!confirm("Grant " + amount + " free credit(s) to " + key + "?\n\nThese never expire and are spent AFTER the user's tier allowance.")) {
return;
}
btn.disabled = true;
try {
const r = await fetch("/admin/credits/grant", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ credit_key: key, amount }),
});
const data = await r.json().catch(() => ({}));
if (!r.ok) throw new Error(data.error || ("HTTP " + r.status));
// Refetch enriched rows so Remaining/Total/Purchased reflect the grant.
await loadCredits();
} catch (err) {
alert("Grant failed: " + (err?.message || err));
btn.disabled = false;
}
}
function renderUsersTab() {
const data = state.creditsData;
let body;
if (data == null) {
body = '<div class="loading">' + (state.creditsLoading ? "Loading users…" : "No data.") + '</div>';
} else {
const counts = data.reduce((acc, r) => { acc[r.type] = (acc[r.type] || 0) + 1; return acc; }, {});
const summary = '<div style="font-size:11px;color:var(--fg-dim);margin-bottom:12px;">'
+ fmtInt(data.length) + ' user(s) — '
+ (counts.cloud || 0) + ' cloud · ' + (counts.license || 0) + ' license · ' + (counts.install || 0) + ' install. '
+ 'Grants land in the never-expires top-up bucket (spent after the tier allowance).'
+ '</div>';
const search = '<input type="text" placeholder="Filter by key…" value="' + esc(state.creditsQuery || "") + '" '
+ 'oninput="filterUsers(this.value)" '
+ 'style="margin-bottom:12px;width:260px;padding:6px 10px;font-size:12px;background:var(--bg);border:1px solid var(--line-2);border-radius:5px;color:var(--fg);" />';
body = summary + search + '<div id="users-table-wrap">' + usersTableHtml() + '</div>';
}
root.innerHTML =
'<h1>Recap Relay — Operator Dashboard</h1>' +
tabsHtml() +
'<div style="max-width:1100px; padding:12px 0;">' +
body +
'</div>';
}
function table(headers, rows) { function table(headers, rows) {
if (rows.length === 0) return '<div class="empty">No data in this range.</div>'; if (rows.length === 0) return '<div class="empty">No data in this range.</div>';
const thead = headers.map(h => '<th>' + esc(h) + '</th>').join(""); const thead = headers.map(h => '<th>' + esc(h) + '</th>').join("");
+7 -1
View File
@@ -17,6 +17,7 @@ import { initCredits } from "./credits.js";
import { initAuditLog } from "./audit-log.js"; import { initAuditLog } from "./audit-log.js";
import { initJobCredits } from "./job-credits.js"; import { initJobCredits } from "./job-credits.js";
import { initOutputStore } from "./output-store.js"; import { initOutputStore } from "./output-store.js";
import { initWebhookDedup } from "./webhook-dedup.js";
import { import {
setupAdminAuthMiddleware, setupAdminAuthMiddleware,
setupAdminAuthRoutes, setupAdminAuthRoutes,
@@ -49,9 +50,14 @@ await initCredits({ dataDir: DATA_DIR });
await initJobCredits({ dataDir: DATA_DIR }); await initJobCredits({ dataDir: DATA_DIR });
await initAuditLog({ dataDir: DATA_DIR }); await initAuditLog({ dataDir: DATA_DIR });
await initOutputStore({ dataDir: DATA_DIR }); await initOutputStore({ dataDir: DATA_DIR });
await initWebhookDedup({ dataDir: DATA_DIR });
const app = express(); const app = express();
app.use(cors()); // CORS only on /relay/* — those are the cross-origin clients (the Recaps
// app + cloud server). /admin/* and the dashboard are served same-origin
// to the operator's browser, so they don't need (and shouldn't get) the
// permissive Access-Control-Allow-Origin a global cors() would apply.
app.use("/relay", cors());
app.use(cookieParser()); app.use(cookieParser());
// Admin auth must run BEFORE the admin routes register so the cookie // Admin auth must run BEFORE the admin routes register so the cookie
+72 -2
View File
@@ -8,8 +8,8 @@
// action but reachable from the dashboard) // action but reachable from the dashboard)
import express from "express"; import express from "express";
import { getConfigSnapshot, getTierPrices } from "../config.js"; import { getConfigSnapshot, getTierPrices, getTierQuotas } from "../config.js";
import { snapshotAll } from "../credits.js"; import { snapshotAll, computeRemaining, addPurchasedCredits } from "../credits.js";
import { snapshotCache } from "../keysat-client.js"; import { snapshotCache } from "../keysat-client.js";
// snapshotJobs is exported by BOTH ../jobs.js (the in-memory job // snapshotJobs is exported by BOTH ../jobs.js (the in-memory job
// tracker) and ../job-credits.js (the credit-ledger). They return // tracker) and ../job-credits.js (the credit-ledger). They return
@@ -47,6 +47,16 @@ import { getHardwareQueueStatus } from "../hardware-queue.js";
import fs from "fs/promises"; import fs from "fs/promises";
import path from "path"; import path from "path";
// Human-facing row category derived from the ledger credit-key prefix.
// `user:` → cloud user (recaps.cc), `lic:` → shared license pool,
// `inst:` and legacy bare-installId rows → a single install.
function creditKeyType(key) {
if (typeof key !== "string") return "install";
if (key.startsWith("user:")) return "cloud";
if (key.startsWith("lic:")) return "license";
return "install";
}
export function adminRouter({ dataDir }) { export function adminRouter({ dataDir }) {
const router = express.Router(); const router = express.Router();
@@ -58,6 +68,66 @@ export function adminRouter({ dataDir }) {
}); });
}); });
// ── Users / credit-balance view ──────────────────────────────────
// Like /usage but enriched for the dashboard's Users tab: each row
// carries a human-facing `type` (derived from the credit-key prefix)
// and a COMPUTED balance (remaining tier credits + purchased top-up),
// since the ledger stores consumed counters, not a remaining number.
router.get("/credits", async (_req, res) => {
const quotas = await getTierQuotas();
const rows = snapshotAll().map((r) => {
const balance = computeRemaining(r, quotas);
return {
...r,
type: creditKeyType(r.credit_key),
remaining: balance.remaining, // tier portion; null = unlimited
purchased: balance.purchased,
total: balance.total, // remaining + purchased; null = unlimited
capped: balance.capped, // "monthly" | "lifetime"
gemini_remaining: balance.gemini_remaining,
};
});
res.json({ count: rows.length, rows });
});
// Grant free credits to one user. Lands in the never-expires
// `purchased_balance` bucket (spent AFTER the tier allotment), so this
// is a pure top-up — it doesn't touch the user's monthly/lifetime
// allowance. Only grants to an EXISTING ledger row: a typo'd key must
// not spawn a ghost row, so we check snapshotAll() first.
router.post("/credits/grant", express.json(), async (req, res) => {
const creditKey =
typeof req.body?.credit_key === "string" ? req.body.credit_key.trim() : "";
const amount = Number(req.body?.amount);
if (!creditKey) {
return res.status(400).json({ error: "credit_key required" });
}
if (!Number.isInteger(amount) || amount <= 0) {
return res.status(400).json({ error: "amount must be a positive integer" });
}
// Fat-finger guard — a manual top-up in the millions is almost
// certainly a typo, not intent. Operator can repeat the grant if
// they genuinely mean to add more.
if (amount > 1_000_000) {
return res.status(400).json({ error: "amount too large (max 1,000,000 per grant)" });
}
const exists = snapshotAll().some((r) => r.credit_key === creditKey);
if (!exists) {
return res.status(404).json({ error: "unknown credit_key" });
}
const newBalance = await addPurchasedCredits({ creditKey, amount });
console.log(
`[admin/credits] manual grant: +${amount} free credit(s) to ${creditKey} ` +
`(new purchased_balance: ${newBalance})`
);
res.json({
ok: true,
credit_key: creditKey,
granted: amount,
purchased_balance: newBalance,
});
});
router.get("/config", async (_req, res) => { router.get("/config", async (_req, res) => {
const cfg = await getConfigSnapshot(); const cfg = await getConfigSnapshot();
const hw = await (await import("../hardware-config.js")).resolveHardwareConfig(cfg); const hw = await (await import("../hardware-config.js")).resolveHardwareConfig(cfg);
+17 -19
View File
@@ -49,18 +49,16 @@ import {
BtcPayError, BtcPayError,
} from "../btcpay-client.js"; } from "../btcpay-client.js";
import { recordCall } from "../audit-log.js"; import { recordCall } from "../audit-log.js";
import { isWebhookProcessed, markWebhookProcessed } from "../webhook-dedup.js";
import { envelope, errorEnvelope } from "./envelope.js"; import { envelope, errorEnvelope } from "./envelope.js";
// In-memory dedup of processed BTCPay invoice ids. BTCPay retries // Dedup of processed BTCPay invoice ids. BTCPay retries webhook
// webhook deliveries on non-2xx responses or network errors, so the // deliveries on non-2xx responses or network errors, so the same
// same InvoiceSettled event can land more than once. We don't want // InvoiceSettled event can land more than once — we don't want to grant
// to grant credits twice for one paid invoice. // credits twice for one paid invoice. Backed by the persistent
// // webhook-dedup store (../webhook-dedup.js) so a duplicate straddling a
// Cleared on relay restart, which means an unlucky webhook duplicate // relay restart can't double-credit. Keys are namespaced
// straddling a restart would double-credit. Acceptable tradeoff for // `<storeId>|<invoiceId>` to share the store with the Zaprite rail.
// v1 (operator can manually adjust the ledger if it happens), but
// worth swapping for a persistent set once the relay sees real load.
const processedInvoices = new Set();
export function creditsRouter() { export function creditsRouter() {
const router = express.Router(); const router = express.Router();
@@ -452,7 +450,7 @@ export function creditsRouter() {
} }
const dedupKey = `${cfg.relay_btcpay_store_id}|${invoiceId}`; const dedupKey = `${cfg.relay_btcpay_store_id}|${invoiceId}`;
if (processedInvoices.has(dedupKey)) { if (isWebhookProcessed(dedupKey)) {
return res return res
.status(200) .status(200)
.json({ ok: true, ignored: "already_processed", invoiceId }); .json({ ok: true, ignored: "already_processed", invoiceId });
@@ -487,7 +485,7 @@ export function creditsRouter() {
const subTier = meta.tier; const subTier = meta.tier;
const periodDays = Number(meta.period_days) || 30; const periodDays = Number(meta.period_days) || 30;
if (!subUserId || (subTier !== "pro" && subTier !== "max")) { if (!subUserId || (subTier !== "pro" && subTier !== "max")) {
processedInvoices.add(dedupKey); await markWebhookProcessed(dedupKey);
return res return res
.status(200) .status(200)
.json({ ok: true, ignored: "bad_tier_metadata", invoiceId }); .json({ ok: true, ignored: "bad_tier_metadata", invoiceId });
@@ -498,7 +496,7 @@ export function creditsRouter() {
tier: subTier, tier: subTier,
periodDays, periodDays,
}); });
processedInvoices.add(dedupKey); await markWebhookProcessed(dedupKey);
console.log( console.log(
`[btcpay/webhook] ${subTier} +${periodDays}d for user ${subUserId.slice(0, 8)}… (invoice ${invoiceId}) → expires ${row.subscription_expires_at}`, `[btcpay/webhook] ${subTier} +${periodDays}d for user ${subUserId.slice(0, 8)}… (invoice ${invoiceId}) → expires ${row.subscription_expires_at}`,
); );
@@ -529,7 +527,7 @@ export function creditsRouter() {
// Not ours — could be an invoice from another product on the // Not ours — could be an invoice from another product on the
// same store. Mark processed so we don't keep retrying, and // same store. Mark processed so we don't keep retrying, and
// 200 so BTCPay stops calling. // 200 so BTCPay stops calling.
processedInvoices.add(dedupKey); await markWebhookProcessed(dedupKey);
return res return res
.status(200) .status(200)
.json({ ok: true, ignored: "no_recap_metadata" }); .json({ ok: true, ignored: "no_recap_metadata" });
@@ -541,7 +539,7 @@ export function creditsRouter() {
creditKey, creditKey,
amount: credits, amount: credits,
}); });
processedInvoices.add(dedupKey); await markWebhookProcessed(dedupKey);
await recordCall({ await recordCall({
install_id: installId, install_id: installId,
license_fingerprint: buyerFp, license_fingerprint: buyerFp,
@@ -614,8 +612,8 @@ function rewriteCheckoutUrl(url, browserBase) {
// Recovery: when the BTCPay webhook URL was broken and paid // Recovery: when the BTCPay webhook URL was broken and paid
// invoices never got credited, this scans BTCPay's recent settled // invoices never got credited, this scans BTCPay's recent settled
// invoices and grants credits for ones the relay hasn't processed. // invoices and grants credits for ones the relay hasn't processed.
// Idempotent via the same processedInvoices dedup the webhook uses, // Idempotent via the same persistent webhook-dedup store the webhook
// so re-running is safe. // uses, so re-running is safe.
// //
// Exported so the admin route in btcpay-setup.js can call it. Not // Exported so the admin route in btcpay-setup.js can call it. Not
// exposed via /relay/* because it's operator-initiated, not buyer. // exposed via /relay/* because it's operator-initiated, not buyer.
@@ -660,7 +658,7 @@ export async function rescanSettledInvoices() {
const details = []; const details = [];
for (const invoice of invoices || []) { for (const invoice of invoices || []) {
const dedupKey = `${cfg.relay_btcpay_store_id}|${invoice.id}`; const dedupKey = `${cfg.relay_btcpay_store_id}|${invoice.id}`;
if (processedInvoices.has(dedupKey)) { if (isWebhookProcessed(dedupKey)) {
alreadyProcessed++; alreadyProcessed++;
continue; continue;
} }
@@ -684,7 +682,7 @@ export async function rescanSettledInvoices() {
creditKey, creditKey,
amount: credits, amount: credits,
}); });
processedInvoices.add(dedupKey); await markWebhookProcessed(dedupKey);
await recordCall({ await recordCall({
install_id: installId, install_id: installId,
license_fingerprint: buyerFp, license_fingerprint: buyerFp,
+11 -13
View File
@@ -15,16 +15,14 @@ import express from "express";
import { extendUserTier } from "../credits.js"; import { extendUserTier } from "../credits.js";
import { getZapriteConfig } from "../config.js"; import { getZapriteConfig } from "../config.js";
import { getOrder, isOrderPaid, orderIdFromWebhook } from "../zaprite-client.js"; import { getOrder, isOrderPaid, orderIdFromWebhook } from "../zaprite-client.js";
import { isWebhookProcessed, markWebhookProcessed } from "../webhook-dedup.js";
// In-memory dedup of fully-processed orders (mirrors the BTCPay handler's // Dedup of fully-processed orders (mirrors the BTCPay handler). Zaprite
// processedInvoices). Zaprite retries on non-200, and may fire multiple // retries on non-200, and may fire multiple events per order, so we
// events per order, so we guard the grant. Cleared on restart — a // guard the grant. Backed by the persistent webhook-dedup store
// re-delivered webhook after restart re-fetches + re-grants, but // (../webhook-dedup.js) so a re-delivered webhook straddling a relay
// extendUserTier is keyed per order via this set within a process; a // restart can't double-extend a subscription. Keys are namespaced
// duplicate grant across a restart is the same harmless "extend by one // `zaprite:<orderId>` to share the store with the BTCPay rail.
// period" the operator-comp path already tolerates. (Acceptable: card
// double-fires across a restart are vanishingly rare.)
const processedZaprite = new Set();
export function zapriteWebhookRouter() { export function zapriteWebhookRouter() {
const router = express.Router(); const router = express.Router();
@@ -49,7 +47,7 @@ export function zapriteWebhookRouter() {
return res.status(200).json({ ok: true, ignored: "no_order_id" }); return res.status(200).json({ ok: true, ignored: "no_order_id" });
} }
const dedupKey = `zaprite:${orderId}`; const dedupKey = `zaprite:${orderId}`;
if (processedZaprite.has(dedupKey)) { if (isWebhookProcessed(dedupKey)) {
return res return res
.status(200) .status(200)
.json({ ok: true, ignored: "already_processed", orderId }); .json({ ok: true, ignored: "already_processed", orderId });
@@ -91,7 +89,7 @@ export function zapriteWebhookRouter() {
const meta = order.metadata || {}; const meta = order.metadata || {};
if (meta.product !== "recap_tier_subscription") { if (meta.product !== "recap_tier_subscription") {
processedZaprite.add(dedupKey); await markWebhookProcessed(dedupKey);
return res return res
.status(200) .status(200)
.json({ ok: true, ignored: "not_a_tier_order", orderId }); .json({ ok: true, ignored: "not_a_tier_order", orderId });
@@ -100,7 +98,7 @@ export function zapriteWebhookRouter() {
const subTier = meta.tier; const subTier = meta.tier;
const periodDays = Number(meta.period_days) || 30; const periodDays = Number(meta.period_days) || 30;
if (!subUserId || (subTier !== "pro" && subTier !== "max")) { if (!subUserId || (subTier !== "pro" && subTier !== "max")) {
processedZaprite.add(dedupKey); await markWebhookProcessed(dedupKey);
return res return res
.status(200) .status(200)
.json({ ok: true, ignored: "bad_tier_metadata", orderId }); .json({ ok: true, ignored: "bad_tier_metadata", orderId });
@@ -112,7 +110,7 @@ export function zapriteWebhookRouter() {
tier: subTier, tier: subTier,
periodDays, periodDays,
}); });
processedZaprite.add(dedupKey); await markWebhookProcessed(dedupKey);
console.log( console.log(
`[zaprite/webhook] ${subTier} +${periodDays}d for user ${subUserId.slice(0, 8)}… (order ${orderId}) → expires ${row.subscription_expires_at}`, `[zaprite/webhook] ${subTier} +${periodDays}d for user ${subUserId.slice(0, 8)}… (order ${orderId}) → expires ${row.subscription_expires_at}`,
); );
+106
View File
@@ -0,0 +1,106 @@
// The /admin/credits read view + /admin/credits/grant action behind the
// Users dashboard tab. Mounts the REAL admin router (sans the cookie
// auth middleware, which lives in index.js) over an ephemeral HTTP
// server so the handler's validation runs for real, not just the
// underlying ledger primitives.
import { test, describe, before, after } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import express from "express";
import { initCredits, setUserTier, addPurchasedCredits } from "../credits.js";
import { adminRouter } from "../routes/admin.js";
let baseUrl;
let server;
const getCredits = async () => {
const r = await fetch(`${baseUrl}/admin/credits`, { cache: "no-store" });
return { status: r.status, body: await r.json() };
};
const grant = async (payload) => {
const r = await fetch(`${baseUrl}/admin/credits/grant`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(payload),
});
return { status: r.status, body: await r.json().catch(() => ({})) };
};
const rowFor = (body, key) => body.rows.find((r) => r.credit_key === key);
describe("/admin/credits + /admin/credits/grant", () => {
before(async () => {
await initCredits({ dataDir: mkdtempSync(path.join(tmpdir(), "relay-admincred-")) });
// Seed one row of each ledger-key shape so the type derivation and
// the per-tier balance math are both exercised. Default quotas:
// Core lifetime=10, Pro monthly=50.
await setUserTier({ userId: "alice", tier: "pro" }); // user:alice
await addPurchasedCredits({ creditKey: "inst:bob", amount: 5 }); // Core install
await addPurchasedCredits({ creditKey: "lic:deadbeefdeadbeef", amount: 2 }); // license pool
const app = express();
app.use("/admin", adminRouter({ dataDir: "/tmp" }));
await new Promise((resolve) => { server = app.listen(0, resolve); });
baseUrl = `http://127.0.0.1:${server.address().port}`;
});
after(() => { server?.close(); });
test("GET /credits derives a type from the credit-key prefix", async () => {
const { status, body } = await getCredits();
assert.equal(status, 200);
assert.equal(rowFor(body, "user:alice").type, "cloud");
assert.equal(rowFor(body, "inst:bob").type, "install");
assert.equal(rowFor(body, "lic:deadbeefdeadbeef").type, "license");
});
test("GET /credits computes remaining/total (not raw counters)", async () => {
const { body } = await getCredits();
const alice = rowFor(body, "user:alice"); // Pro, monthly 50, nothing spent
assert.equal(alice.remaining, 50);
assert.equal(alice.purchased, 0);
assert.equal(alice.total, 50);
const bob = rowFor(body, "inst:bob"); // Core lifetime 10 + 5 purchased
assert.equal(bob.remaining, 10);
assert.equal(bob.purchased, 5);
assert.equal(bob.total, 15);
});
test("POST /grant adds to the purchased bucket and returns the new balance", async () => {
const { status, body } = await grant({ credit_key: "inst:bob", amount: 7 });
assert.equal(status, 200);
assert.equal(body.ok, true);
assert.equal(body.granted, 7);
assert.equal(body.purchased_balance, 12); // 5 seeded + 7 granted
// ...and the read view reflects it (purchased 12, total 10 + 12).
const { body: after } = await getCredits();
const bob = rowFor(after, "inst:bob");
assert.equal(bob.purchased, 12);
assert.equal(bob.total, 22);
});
test("POST /grant rejects non-positive / non-integer amounts", async () => {
for (const amount of [0, -5, 1.5, "x"]) {
const { status } = await grant({ credit_key: "inst:bob", amount });
assert.equal(status, 400, `amount=${amount} should be 400`);
}
});
test("POST /grant rejects an absurdly large amount (fat-finger guard)", async () => {
const { status } = await grant({ credit_key: "inst:bob", amount: 2_000_000 });
assert.equal(status, 400);
});
test("POST /grant 400s on missing credit_key, 404s on an unknown one", async () => {
assert.equal((await grant({ amount: 5 })).status, 400);
const unknown = await grant({ credit_key: "inst:nobody", amount: 5 });
assert.equal(unknown.status, 404);
// The unknown key must NOT have been created as a side effect.
const { body } = await getCredits();
assert.equal(rowFor(body, "inst:nobody"), undefined);
});
});
+111
View File
@@ -0,0 +1,111 @@
// Core billing primitives: commitCredit (debit), refundCredit (inverse),
// and applyTierPromotion (the Core→paid upgrade bookkeeping). Default
// quotas (no config file → getTierQuotas falls back): Core lifetime=10,
// geminiCapLifetime=5; Pro monthly=50, geminiCapMonthly=25.
import { test, describe, before } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import {
initCredits,
getOrCreateRow,
getUserCreditRow,
setUserTier,
commitCredit,
refundCredit,
applyTierPromotion,
} from "../credits.js";
before(async () => {
await initCredits({ dataDir: mkdtempSync(path.join(tmpdir(), "relay-money-")) });
});
describe("commitCredit", () => {
test("Core debits the lifetime bucket; Gemini tracked separately", async () => {
const creditKey = "inst:core-commit";
await commitCredit({ creditKey, tier: "core", backend: "gemini" });
let row = await getOrCreateRow({ creditKey });
assert.equal(row.lifetime_consumed, 1);
assert.equal(row.lifetime_gemini_consumed, 1);
// A hardware call bumps lifetime but NOT the Gemini sub-counter.
await commitCredit({ creditKey, tier: "core", backend: "hardware" });
row = await getOrCreateRow({ creditKey });
assert.equal(row.lifetime_consumed, 2);
assert.equal(row.lifetime_gemini_consumed, 1);
});
test("paid tier debits the monthly bucket", async () => {
await setUserTier({ userId: "p-commit", tier: "pro" });
await commitCredit({ creditKey: "user:p-commit", tier: "pro", backend: "gemini" });
const row = await getUserCreditRow("p-commit");
assert.equal(row.monthly_consumed, 1);
assert.equal(row.monthly_gemini_consumed, 1);
});
test("spend order: once the tier bucket is exhausted, debit purchased", async () => {
const creditKey = "inst:core-exhausted";
const row = await getOrCreateRow({ creditKey });
row.lifetime_consumed = 10; // at the Core lifetime cap
row.purchased_balance = 3;
await commitCredit({ creditKey, tier: "core", backend: "hardware" });
const after = await getOrCreateRow({ creditKey });
assert.equal(after.lifetime_consumed, 10, "tier counter must not exceed the cap");
assert.equal(after.purchased_balance, 2, "overflow comes out of purchased");
});
});
describe("refundCredit", () => {
test("mirrors a Core commit (tier bucket first, Gemini sub-counter too)", async () => {
const creditKey = "inst:core-refund";
await commitCredit({ creditKey, tier: "core", backend: "gemini" });
await refundCredit({ creditKey, tier: "core", backend: "gemini" });
const row = await getOrCreateRow({ creditKey });
assert.equal(row.lifetime_consumed, 0);
assert.equal(row.lifetime_gemini_consumed, 0);
});
test("refund with an empty tier bucket credits the purchased bucket", async () => {
const creditKey = "inst:refund-to-purchased";
const row = await getOrCreateRow({ creditKey });
row.lifetime_consumed = 0;
row.purchased_balance = 0;
await refundCredit({ creditKey, tier: "core", backend: "hardware" });
const after = await getOrCreateRow({ creditKey });
assert.equal(after.lifetime_consumed, 0, "must floor at 0, not go negative");
assert.equal(after.purchased_balance, 1, "refund lands in purchased when tier is already 0");
});
});
describe("applyTierPromotion", () => {
test("Core→paid transfers leftover Core credits to purchased and resets the cycle", async () => {
const row = await getOrCreateRow({ creditKey: "inst:promo" });
row.lifetime_consumed = 4; // 6 of 10 Core credits unused
row.monthly_consumed = 2; // stale paid-counter noise that must be zeroed
const promoted = await applyTierPromotion(row, "pro");
assert.equal(promoted, true);
assert.equal(row.tier_snapshot, "pro");
assert.equal(row.purchased_balance, 6, "leftover Core credits carry forward as durable top-up");
assert.equal(row.purchased_total_ever, 6);
assert.equal(row.monthly_consumed, 0, "promotion starts a fresh monthly cycle");
});
test("idempotent: a second promotion on an already-paid row is a no-op", async () => {
const row = await getOrCreateRow({ creditKey: "inst:promo-again" });
row.lifetime_consumed = 4;
await applyTierPromotion(row, "pro"); // first: fires
const purchasedAfterFirst = row.purchased_balance;
const promoted = await applyTierPromotion(row, "max"); // second: must bail
assert.equal(promoted, false);
assert.equal(row.purchased_balance, purchasedAfterFirst, "no second leftover transfer");
assert.equal(row.tier_snapshot, "pro", "bails before flipping tier again");
});
test("promoting to Core is a no-op", async () => {
const row = await getOrCreateRow({ creditKey: "inst:promo-core" });
assert.equal(await applyTierPromotion(row, "core"), false);
});
});
+70
View File
@@ -0,0 +1,70 @@
// Persistent webhook dedup — the store that stops a settled-payment
// webhook duplicate from double-crediting (BTCPay) or double-extending
// (Zaprite) when the duplicate straddles a relay restart.
import { test, describe } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import {
initWebhookDedup,
isWebhookProcessed,
markWebhookProcessed,
} from "../webhook-dedup.js";
// The module is a singleton; each test inits a fresh dir to reset state.
const freshDir = () => mkdtempSync(path.join(tmpdir(), "relay-dedup-"));
describe("webhook dedup (persistent)", () => {
test("unknown key is not processed; marking makes it processed", async () => {
await initWebhookDedup({ dataDir: freshDir() });
assert.equal(isWebhookProcessed("store1|inv1"), false);
await markWebhookProcessed("store1|inv1");
assert.equal(isWebhookProcessed("store1|inv1"), true);
});
// Headline guarantee for this change.
test("processed keys survive a restart (reload from disk)", async () => {
const dir = freshDir();
await initWebhookDedup({ dataDir: dir });
await markWebhookProcessed("store1|inv-restart");
// Simulate a relay restart: re-init from the SAME data dir.
await initWebhookDedup({ dataDir: dir });
assert.equal(
isWebhookProcessed("store1|inv-restart"),
true,
"a key marked before restart must still be deduped after restart"
);
});
test("BTCPay and Zaprite keys share the store without colliding", async () => {
await initWebhookDedup({ dataDir: freshDir() });
await markWebhookProcessed("store1|inv9");
await markWebhookProcessed("zaprite:order9");
assert.equal(isWebhookProcessed("store1|inv9"), true);
assert.equal(isWebhookProcessed("zaprite:order9"), true);
assert.equal(isWebhookProcessed("zaprite:inv9"), false);
});
test("marking is idempotent", async () => {
await initWebhookDedup({ dataDir: freshDir() });
await markWebhookProcessed("k");
await markWebhookProcessed("k"); // must not throw or duplicate
assert.equal(isWebhookProcessed("k"), true);
});
test("stale entries past the retention window are pruned on load", async () => {
const dir = freshDir();
const ancient = new Date(Date.now() - 365 * 24 * 60 * 60 * 1000).toISOString();
const recent = new Date().toISOString();
writeFileSync(
path.join(dir, "processed-webhooks.json"),
JSON.stringify({ keys: { "store1|old": ancient, "store1|new": recent } })
);
await initWebhookDedup({ dataDir: dir });
assert.equal(isWebhookProcessed("store1|old"), false, "1y-old key should be pruned");
assert.equal(isWebhookProcessed("store1|new"), true, "recent key should survive");
});
});
+93
View File
@@ -0,0 +1,93 @@
// Persistent dedup of settled-payment webhook deliveries — BTCPay
// invoices AND Zaprite orders share this one store. It replaces the
// per-process in-memory Sets the two webhook handlers used to keep:
// those were cleared on restart, so a duplicate delivery that straddled
// a relay restart would double-credit (BTCPay) or double-extend a
// subscription (Zaprite). Persisting the processed-keys to disk closes
// that window.
//
// JSON-file backed (single file at <dataDir>/processed-webhooks.json),
// same rationale as the credit ledger in credits.js: at most one
// mutation per settled payment, so a plain JSON file with serial writes
// is plenty.
//
// Keys are namespaced by their caller so the two rails can't collide in
// the shared store: BTCPay uses `<storeId>|<invoiceId>`, Zaprite uses
// `zaprite:<orderId>`. We store key → ISO timestamp (not a bare set) so
// entries far older than any processor's retry window can be pruned on
// load, keeping the file bounded.
import fs from "fs/promises";
import path from "path";
// Payment processors retry webhook delivery for hours-to-days, never
// months. 180 days is safely beyond any retry window, so pruning past
// it cannot re-open a double-grant gap.
const RETENTION_MS = 180 * 24 * 60 * 60 * 1000;
let storePath = null;
let processed = {}; // key → ISO timestamp
let writing = null; // serializes concurrent writes
export async function initWebhookDedup({ dataDir } = {}) {
const dir = dataDir || "/data";
storePath = path.join(dir, "processed-webhooks.json");
await fs.mkdir(dir, { recursive: true }).catch(() => {});
try {
const raw = await fs.readFile(storePath, "utf8");
const parsed = JSON.parse(raw);
processed =
parsed && typeof parsed === "object" && parsed.keys ? parsed.keys : {};
} catch (err) {
if (err.code !== "ENOENT") {
console.warn(
`[webhook-dedup] failed to read ${storePath}: ${err.message} — starting empty`
);
}
processed = {};
}
// Prune anything older than the retention window so the file stays
// bounded over the relay's lifetime.
const cutoff = Date.now() - RETENTION_MS;
let pruned = 0;
for (const [k, ts] of Object.entries(processed)) {
const t = Date.parse(ts);
if (!Number.isFinite(t) || t < cutoff) {
delete processed[k];
pruned += 1;
}
}
if (pruned > 0) await persist();
console.log(
`[webhook-dedup] loaded ${Object.keys(processed).length} processed key(s)` +
`${pruned ? `, pruned ${pruned} stale` : ""} from ${storePath}`
);
}
async function persist() {
// Coalesce concurrent writes — mirrors credits.js persist().
if (writing) await writing;
writing = (async () => {
const tmp = storePath + ".tmp";
await fs.writeFile(tmp, JSON.stringify({ keys: processed }), { mode: 0o600 });
await fs.rename(tmp, storePath);
})();
try {
await writing;
} finally {
writing = null;
}
}
export function isWebhookProcessed(key) {
return Object.prototype.hasOwnProperty.call(processed, key);
}
// Mark a webhook key processed and persist immediately. Callers invoke
// this AFTER the grant/extend succeeds, so a failed grant leaves the key
// unmarked and a processor retry can complete it. Idempotent.
export async function markWebhookProcessed(key) {
if (isWebhookProcessed(key)) return;
processed[key] = new Date().toISOString();
await persist();
}
+9 -9
View File
@@ -31,15 +31,15 @@ export const manifest = setupManifest({
start: null, start: null,
stop: null, stop: null,
}, },
// Relay has no REQUIRED dependencies — Gemini is internet-fronted // BTCPay Server is a REQUIRED running dependency (optional: false
// and the optional Parakeet/Gemma backends are at user-configured // here, kind: 'running' in dependencies.ts). It's the only payment
// URLs (typically a separate machine on the operator's LAN). // rail through which the operator actually gets paid, so the relay
// // shouldn't run without it. The dependency also wires up the internal
// BTCPay Server is declared optional so the dashboard's "Connect // docker hostname (btcpayserver.startos:23000) the relay-to-BTCPay API
// BTCPay" flow can auto-discover its URL via // calls rely on, and lets the dashboard's "Connect BTCPay" flow
// sdk.serviceInterface.getAll() when both are installed on the // auto-discover the URL via sdk.serviceInterface.getAll(). (Gemini is
// same Start9 box. When BTCPay is not installed, the relay still // internet-fronted and the Parakeet/Gemma backends are at
// runs fine — only the credit-purchase flow is disabled. // user-configured LAN URLs, so neither is a StartOS dependency.)
dependencies: { dependencies: {
btcpayserver: { btcpayserver: {
description: { description: {
+4 -2
View File
@@ -125,8 +125,10 @@ import { v_0_2_121 } from './v0.2.121'
import { v_0_2_122 } from './v0.2.122' import { v_0_2_122 } from './v0.2.122'
import { v_0_2_123 } from './v0.2.123' import { v_0_2_123 } from './v0.2.123'
import { v_0_2_124 } from './v0.2.124' import { v_0_2_124 } from './v0.2.124'
import { v_0_2_125 } from './v0.2.125'
import { v_0_2_126 } from './v0.2.126'
export const versionGraph = VersionGraph.of({ export const versionGraph = VersionGraph.of({
current: v_0_2_124, current: v_0_2_126,
other: [v_0_2_123, v_0_2_122, v_0_2_121, v_0_2_120, v_0_2_119, v_0_2_118, v_0_2_117, v_0_2_116, v_0_2_115, v_0_2_114, v_0_2_113, v_0_2_112, v_0_2_111, v_0_2_110, v_0_2_109, v_0_2_108, v_0_2_107, v_0_2_106, v_0_2_105, v_0_2_104, v_0_2_103, v_0_2_102, v_0_2_101, v_0_2_100, v_0_2_99, v_0_2_98, v_0_2_97, v_0_2_96, v_0_2_95, v_0_2_94, v_0_2_93, v_0_2_92, v_0_2_91, v_0_2_90, v_0_2_89, v_0_2_88, v_0_2_87, v_0_2_86, v_0_2_85, v_0_2_84, v_0_2_83, v_0_2_82, v_0_2_81, v_0_2_80, v_0_2_79, v_0_2_78, v_0_2_77, v_0_2_76, v_0_2_75, v_0_2_74, v_0_2_73, v_0_2_72, v_0_2_71, v_0_2_70, v_0_2_69, v_0_2_68, v_0_2_67, v_0_2_66, v_0_2_65, v_0_2_64, v_0_2_63, v_0_2_62, v_0_2_61, v_0_2_60, v_0_2_59, v_0_2_58, v_0_2_57, v_0_2_56, v_0_2_55, v_0_2_54, v_0_2_53, v_0_2_52, v_0_2_51, v_0_2_50, v_0_2_49, v_0_2_48, v_0_2_47, v_0_2_46, v_0_2_45, v_0_2_44, v_0_2_43, v_0_2_42, v_0_2_41, v_0_2_40, v_0_2_39, v_0_2_38, v_0_2_37, v_0_2_36, v_0_2_35, v_0_2_34, v_0_2_33, v_0_2_32, v_0_2_31, v_0_2_30, v_0_2_29, v_0_2_28, v_0_2_27, v_0_2_26, v_0_2_25, v_0_2_24, v_0_2_23, v_0_2_22, v_0_2_21, v_0_2_20, v_0_2_19, v_0_2_18, v_0_2_17, v_0_2_16, v_0_2_15, v_0_2_14, v_0_2_13, v_0_2_12, v_0_2_11, v_0_2_10, v_0_2_9, v_0_2_8, v_0_2_7, v_0_2_6, v_0_2_5, v_0_2_4, v_0_2_3, v_0_2_2, v_0_2_1, v_0_2_0, v_0_1_0], other: [v_0_2_125, v_0_2_124, v_0_2_123, v_0_2_122, v_0_2_121, v_0_2_120, v_0_2_119, v_0_2_118, v_0_2_117, v_0_2_116, v_0_2_115, v_0_2_114, v_0_2_113, v_0_2_112, v_0_2_111, v_0_2_110, v_0_2_109, v_0_2_108, v_0_2_107, v_0_2_106, v_0_2_105, v_0_2_104, v_0_2_103, v_0_2_102, v_0_2_101, v_0_2_100, v_0_2_99, v_0_2_98, v_0_2_97, v_0_2_96, v_0_2_95, v_0_2_94, v_0_2_93, v_0_2_92, v_0_2_91, v_0_2_90, v_0_2_89, v_0_2_88, v_0_2_87, v_0_2_86, v_0_2_85, v_0_2_84, v_0_2_83, v_0_2_82, v_0_2_81, v_0_2_80, v_0_2_79, v_0_2_78, v_0_2_77, v_0_2_76, v_0_2_75, v_0_2_74, v_0_2_73, v_0_2_72, v_0_2_71, v_0_2_70, v_0_2_69, v_0_2_68, v_0_2_67, v_0_2_66, v_0_2_65, v_0_2_64, v_0_2_63, v_0_2_62, v_0_2_61, v_0_2_60, v_0_2_59, v_0_2_58, v_0_2_57, v_0_2_56, v_0_2_55, v_0_2_54, v_0_2_53, v_0_2_52, v_0_2_51, v_0_2_50, v_0_2_49, v_0_2_48, v_0_2_47, v_0_2_46, v_0_2_45, v_0_2_44, v_0_2_43, v_0_2_42, v_0_2_41, v_0_2_40, v_0_2_39, v_0_2_38, v_0_2_37, v_0_2_36, v_0_2_35, v_0_2_34, v_0_2_33, v_0_2_32, v_0_2_31, v_0_2_30, v_0_2_29, v_0_2_28, v_0_2_27, v_0_2_26, v_0_2_25, v_0_2_24, v_0_2_23, v_0_2_22, v_0_2_21, v_0_2_20, v_0_2_19, v_0_2_18, v_0_2_17, v_0_2_16, v_0_2_15, v_0_2_14, v_0_2_13, v_0_2_12, v_0_2_11, v_0_2_10, v_0_2_9, v_0_2_8, v_0_2_7, v_0_2_6, v_0_2_5, v_0_2_4, v_0_2_3, v_0_2_2, v_0_2_1, v_0_2_0, v_0_1_0],
}) })
+12
View File
@@ -0,0 +1,12 @@
import { VersionInfo } from '@start9labs/start-sdk'
export const v_0_2_125 = VersionInfo.of({
version: '0.2.125:0',
releaseNotes: {
en_US: 'Add Users tab to the operator dashboard: per-user credit balances with a grant-free-credits action',
},
migrations: {
up: async ({ effects }) => {},
down: async ({ effects }) => {},
},
})
+12
View File
@@ -0,0 +1,12 @@
import { VersionInfo } from '@start9labs/start-sdk'
export const v_0_2_126 = VersionInfo.of({
version: '0.2.126:0',
releaseNotes: {
en_US: 'Persist payment-webhook dedup across restarts (no double-credit/double-extend); declare BTCPay a required dependency; scope CORS to /relay/*; add money-path + webhook-dedup tests; doc fixes',
},
migrations: {
up: async ({ effects }) => {},
down: async ({ effects }) => {},
},
})