Compare commits
4 Commits
b10399819b
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
 Keysat
|
5d0d1b2dd2 | ||
|
Keysat
|
238689ddcc | ||
|
Keysat
|
798a698132 | ||
|
Keysat
|
00da92a872 |
@@ -8,10 +8,10 @@ Operator-side, credit-metered service that sits in front of Gemini and the opera
|
||||
## Stack
|
||||
|
||||
- **Server**: Node.js (`type: module`, ES modules). Same dev box as the app (`v25.6.1`); container runtime is whatever the `Dockerfile` pins.
|
||||
- **HTTP**: `express` + `multer` (audio upload). Admin routes under `/admin/*` behind an admin-session-cookie gate; relay-to-relay routes under `/relay/*` behind the operator key.
|
||||
- **HTTP**: `express` + `multer` (audio upload). Admin routes under `/admin/*` behind an admin-session-cookie gate. `/relay/*` uses per-call header auth — install-id/license, or operator-key + user-id for the cloud control plane (a few routes like `health`/`policy`/`capabilities` are public). See the Auth model under Endpoints. `cors()` is scoped to `/relay/*` only.
|
||||
- **Dashboard**: `public/dashboard.html` — single-file vanilla JS, render-string-into-innerHTML, same shape as the app's `index.html`.
|
||||
- **Packaging**: `@start9labs/start-sdk` under `startos/` — version graph at `startos/versions/index.ts`.
|
||||
- **Storage**: filesystem under the StartOS data dir (`/data`). Internal meetings persist as `/data/internal-meetings/<id>.json`. No SQLite here.
|
||||
- **Storage**: filesystem under the StartOS data dir (`/data`). No SQLite — flat JSON files: credit ledger `/data/credits.json`, payment-webhook dedup `/data/processed-webhooks.json`, internal meetings `/data/internal-meetings/<id>.json`.
|
||||
- **Upstreams**: Gemini (`@google/genai`); operator hardware via "Spark Control" HTTP (Parakeet transcribe, `/api/audio/diarize-chunk` for Sortformer+TitaNet, a vLLM/Gemma OpenAI-shape analyze endpoint).
|
||||
|
||||
## Commands
|
||||
@@ -47,8 +47,10 @@ server/
|
||||
config.js getConfigSnapshot() + relay_* config defaults
|
||||
hardware-config.js resolveHardwareConfig() → Spark Control endpoint discovery
|
||||
safe-url.js SSRF guard: assertPublicHttpUrl + safeFetch for caller-supplied URLs
|
||||
webhook-dedup.js persistent payment-webhook dedup (BTCPay + Zaprite share it);
|
||||
initWebhookDedup/isWebhookProcessed/markWebhookProcessed
|
||||
test/ node --test *.test.js (speaker tools, billing/credits, SSRF, path-traversal, …)
|
||||
public/dashboard.html operator dashboard (meetings detail view + speaker tools)
|
||||
public/dashboard.html operator dashboard (Overview / Jobs / Users / Internal Meetings / Settings)
|
||||
startos/versions/<vN>.ts one file per version + index.ts graph
|
||||
docs/issues-backlog.md detailed issue log
|
||||
docs/guides/internal-meetings.md diarization / speaker subsystem guide (path-scoped; lazy-loads via .claude/rules/)
|
||||
@@ -62,7 +64,7 @@ All routes mount in `server/index.js`. Public paths sit under `/relay/*`; operat
|
||||
|
||||
- **`X-Recap-Operator-Key`** + **`X-Recap-User-Id`** → "cloud" path. The Recaps cloud server (`recaps.cc`) authenticates once with a shared operator key (`relay_cloud_operator_key`) and names the acting user. Credit pool keyed `user:<id>`, tier comes from the relay's stored row, NOT a per-user license. See `server/identity.js`.
|
||||
- **`X-Recap-Install-Id`** (+ optional `Authorization: <license>`) → "license" path. Self-hosted installs and the operator's single-mode app. Credits/tier come from the resolved Keysat license + install id.
|
||||
- **Admin session cookie** → `/admin/*`. Cookie issued by `POST /admin/login`; `/admin/login` and `/admin/status` are exempt inside `setupAdminAuthMiddleware`.
|
||||
- **Admin session cookie** → `/admin/*`. Cookie issued by `POST /admin/login`; `/admin/login`, `/admin/status`, and `/admin/btcpay/callback` are exempt inside `setupAdminAuthMiddleware`.
|
||||
- **Webhook signature** → `POST /relay/btcpay/webhook` validates `BTCPay-Sig` against `relay_btcpay_webhook_secret`. Zaprite's webhook re-fetches the order through the Zaprite API to verify, so no shared-secret signing.
|
||||
- **`X-Recap-Job-Id`** is a billing key, not auth: the first call with a given id charges one credit; later calls with the same id are free (so transcribe + analyze for one summary = one credit total).
|
||||
|
||||
@@ -94,7 +96,7 @@ All require a valid `X-Recap-Operator-Key`. Defined in `routes/user-tier.js`.
|
||||
|
||||
### `/admin/*` (operator dashboard; cookie-gated)
|
||||
|
||||
`routes/admin.js`: `GET /admin/{usage,config,license-cache,hardware-queue,jobs,jobs-history,job-output/:id,job/:id/details,output-store-stats,output-store-ids,dashboard,dashboard.csv,settings}`, `POST /admin/{quotas,wipe-all,settings/promote-prompt}`, `PUT /admin/settings`, `DELETE /admin/job-outputs`. `routes/admin-test-run.js`: `POST /admin/{test-run,test-run-suite}`. BTCPay setup wizard under `/admin/btcpay/*` (`routes/btcpay-setup.js`).
|
||||
`routes/admin.js`: `GET /admin/{usage,credits,config,license-cache,hardware-queue,jobs,jobs-history,job-output/:id,job/:id/details,output-store-stats,output-store-ids,dashboard,dashboard.csv,settings}`, `POST /admin/{quotas,credits/grant,wipe-all,settings/promote-prompt}`, `PUT /admin/settings`, `DELETE /admin/job-outputs`. (`GET /admin/credits` = ledger rows enriched with type + computed balance for the dashboard Users tab; `POST /admin/credits/grant` `{ credit_key, amount }` adds free top-up credits to an existing row.) `routes/admin-test-run.js`: `POST /admin/{test-run,test-run-suite}`. BTCPay setup wizard under `/admin/btcpay/*` (`routes/btcpay-setup.js`).
|
||||
|
||||
### `/admin/internal-meetings/*` (cookie-gated; `routes/internal-meetings.js`)
|
||||
|
||||
@@ -141,14 +143,14 @@ this. When unsure whether a change is contract-affecting, assume it is and check
|
||||
- **Never edit a `startos/versions/<v>.ts` that's already been built/installed** — add a new version file.
|
||||
- **Don't push to GitHub by default** — remote is self-hosted Gitea.
|
||||
|
||||
## Current state — post-eval security pass landed (2026-06-13)
|
||||
## Current state — Users tab + webhook-dedup/P2 batch landed (2026-06-15)
|
||||
|
||||
- **Box, local tree, git aligned at relay `0.2.124`** (app `0.2.155`); `current: v_0_2_124`. Git is local-only (no remote). Working tree clean. **Suite green at 60 tests** (`cd server && npm test`); server boots clean.
|
||||
- **Full independent eval done** (evaluator + security-auditor + exerciser + doc-auditor + start9-spec-checker) → `EVALUATION.md` (overwritten in place each run, so re-running diffs cleanly).
|
||||
- **All P0/P1 fixed** this session (commits `8ad7c54`/`d2caa98`/`3a601e1`): SSRF guard on caller-supplied media URLs (new `server/safe-url.js`), the early-renewal credit-reset money-leak (`extendUserTier`/`setUserTier` `resetCycle`), and the `multer`→`^2.0.1` DoS bump. None touch the `../recap` client contract.
|
||||
- **Three P2 fixed** (commits `cbd9748`/`da1bba2`/`693d724`): meeting-`:id` path-traversal guard (`meetingPath()`), constant-time operator-key compare, and a JSON error handler that closes the malformed-body stack-trace leak.
|
||||
- **Next (open P2), in priority order:**
|
||||
1. Persist webhook dedup so a restart can't double-credit/double-extend — `routes/credits.js:63`, `zaprite-webhook.js:27`.
|
||||
2. **Needs operator decision:** is BTCPay a hard requirement or truly optional? It's `optional:false`/`kind:'running'` despite "optional" comments, so StartOS won't start the relay without BTCPay co-installed — `startos/manifest/index.ts:38-49` + `dependencies.ts`. Then make manifest/deps/comment agree.
|
||||
3. Money-path unit tests (`commitCredit`/`refundCredit`/`applyTierPromotion`/grant handlers); scope `cors()` off `/admin/*` (`index.js`); split the 2225-line `routes/internal-meetings.js`; fix the two AGENTS.md auth-doc drifts (Stack-line `/relay/*` auth; missing `/admin/btcpay/callback` exempt path).
|
||||
- **Risks/notes:** SSRF guard leaves a DNS-rebinding TOCTOU open (acceptable for a private box; revisit if exposed). P3+ deferred tail + pre-existing speaker-tool/empty-section backlog → `ROADMAP.md` / `docs/issues-backlog.md`.
|
||||
- **Box, local tree, git aligned at relay `0.2.126`** (app `0.2.155`); `current: v_0_2_126`. Gitea remote `origin` (`ssh://git@immense-voyage.local:59916/grant/recap-relay.git`); `master` tracks `origin/master`. Working tree clean. **Suite green at 79 tests** (`cd server && npm test`); server boots clean.
|
||||
- **Users dashboard tab** (`0.2.125`): new cookie-gated tab — every credit-ledger row (typed cloud/license/install) with computed remaining/total balances, key filter, and a per-row "grant free credits" action. `GET /admin/credits` (enriched read) + `POST /admin/credits/grant {credit_key, amount}` (free top-up via `addPurchasedCredits`, guards: positive int ≤1M, must be an existing row). Admin-only; no `../recap` contract change.
|
||||
- **Webhook dedup now persistent** (`0.2.126`): new `server/webhook-dedup.js` (JSON store at `/data/processed-webhooks.json`, atomic writes, 180-day prune) replaces the in-memory Sets in `routes/credits.js` + `zaprite-webhook.js` (and the rescan path) — a duplicate delivery straddling a restart can no longer double-credit/double-extend. Keys namespaced `<storeId>|<invoiceId>` vs `zaprite:<orderId>`.
|
||||
- **BTCPay is REQUIRED** (operator decision, 2026-06-15): config was already `optional:false`/`kind:'running'`; corrected the contradictory "optional" comment in `startos/manifest/index.ts`. It's the only paid rail, so the relay shouldn't run without it.
|
||||
- **CORS scoped to `/relay/*`** (`index.js`) — off `/admin/*` + dashboard (same-origin). Plus money-path unit tests (`commitCredit`/`refundCredit`/`applyTierPromotion`) and the two AGENTS.md auth-doc drift fixes.
|
||||
- **Next (open P2 / deferred):**
|
||||
1. Split the 2225-line `routes/internal-meetings.js` — **deferred as likely overkill** for a private service; do only if it becomes painful to work in.
|
||||
2. P3+ deferred tail (no `/relay/*` rate limiting, container-as-root, dashboard `innerHTML` XSS surface, prune 126 version files, `/relay/health` stale `0.2.11`, etc.) + speaker-tool/empty-section backlog → `ROADMAP.md` / `docs/issues-backlog.md`.
|
||||
- **Risks/notes:** webhook dedup keeps the pre-existing check-then-mark race for *truly simultaneous* duplicate deliveries (vanishingly rare on a private box; would need locking). SSRF guard leaves a DNS-rebinding TOCTOU open (acceptable for a private box). Full prior eval → `EVALUATION.md`.
|
||||
|
||||
+179
-1
@@ -1044,10 +1044,15 @@
|
||||
activeTab: (() => {
|
||||
try {
|
||||
const saved = localStorage.getItem("recap-relay-active-tab");
|
||||
if (saved === "jobs" || saved === "settings" || saved === "overview") return saved;
|
||||
if (saved === "jobs" || saved === "settings" || saved === "overview" || saved === "users") return saved;
|
||||
} catch {}
|
||||
return "overview";
|
||||
})(),
|
||||
// Users-tab state — enriched credit-ledger rows from /admin/credits.
|
||||
creditsData: null,
|
||||
creditsLoading: false,
|
||||
creditsSort: { col: "last_active_at", dir: "desc" },
|
||||
creditsQuery: "",
|
||||
// Jobs-tab state.
|
||||
jobsData: null,
|
||||
jobsLoading: false,
|
||||
@@ -1176,6 +1181,9 @@
|
||||
if (state.authed && state.activeTab === "meetings" && !state.meetingsList) {
|
||||
loadMeetingsList();
|
||||
}
|
||||
if (state.authed && state.activeTab === "users" && !state.creditsData) {
|
||||
loadCredits();
|
||||
}
|
||||
// Start the Overview auto-refresh poll on boot regardless of
|
||||
// current tab — cheap (one fetch every 10s) and ensures the
|
||||
// tab is current the moment the operator switches to it.
|
||||
@@ -1460,6 +1468,8 @@
|
||||
renderSettingsTab();
|
||||
} else if (state.activeTab === "meetings") {
|
||||
renderMeetingsTab();
|
||||
} else if (state.activeTab === "users") {
|
||||
renderUsersTab();
|
||||
} else {
|
||||
renderDashboard();
|
||||
}
|
||||
@@ -1471,6 +1481,7 @@
|
||||
return '<div class="tabs">' +
|
||||
t("overview", "Overview") +
|
||||
t("jobs", "Jobs") +
|
||||
t("users", "Users") +
|
||||
t("meetings", "Internal Meetings") +
|
||||
t("settings", "Settings") +
|
||||
'</div>';
|
||||
@@ -1561,6 +1572,9 @@
|
||||
if (tab === "meetings") {
|
||||
loadMeetingsList();
|
||||
}
|
||||
if (tab === "users") {
|
||||
loadCredits();
|
||||
}
|
||||
// Overview tab: re-fetch the dashboard data on EVERY entry (so
|
||||
// the summaries / errors / perf tables show current state, not
|
||||
// whatever was last cached) AND start the 10-second auto-refresh
|
||||
@@ -6662,6 +6676,170 @@
|
||||
'</div>';
|
||||
}
|
||||
|
||||
// ── Users tab ───────────────────────────────────────────────────
|
||||
// A flat view of the credit ledger: every user/install + their
|
||||
// current balance, with a per-row "grant free credits" action.
|
||||
// Balances are COMPUTED server-side (/admin/credits) since the
|
||||
// ledger stores consumed counters, not a remaining number.
|
||||
async function loadCredits() {
|
||||
state.creditsLoading = true;
|
||||
if (state.activeTab === "users") render();
|
||||
try {
|
||||
const r = await fetch("/admin/credits", { cache: "no-store" });
|
||||
if (!r.ok) throw new Error("HTTP " + r.status);
|
||||
const data = await r.json();
|
||||
state.creditsData = Array.isArray(data.rows) ? data.rows : [];
|
||||
} catch (err) {
|
||||
state.creditsData = [];
|
||||
console.warn("loadCredits failed:", err);
|
||||
}
|
||||
state.creditsLoading = false;
|
||||
if (state.activeTab === "users") render();
|
||||
}
|
||||
|
||||
// Credits consumed in the current cycle: Core spends a lifetime
|
||||
// budget, paid tiers spend a monthly one.
|
||||
function usedCount(r) {
|
||||
return (r.tier_snapshot === "pro" || r.tier_snapshot === "max")
|
||||
? (r.monthly_consumed || 0)
|
||||
: (r.lifetime_consumed || 0);
|
||||
}
|
||||
// null = unlimited tier (no cap). Render it as a word, not a number.
|
||||
function fmtCredits(v) {
|
||||
return v == null ? '<span class="dim">Unlimited</span>' : fmtInt(v);
|
||||
}
|
||||
|
||||
const USERS_COLS = [
|
||||
{ key: "credit_key", label: "Key", val: (r) => r.credit_key },
|
||||
{ key: "type", label: "Type", val: (r) => r.type },
|
||||
{ key: "tier_snapshot", label: "Tier", val: (r) => r.tier_snapshot || "core" },
|
||||
{ key: "remaining", label: "Remaining", num: true, val: (r) => r.remaining == null ? Infinity : r.remaining },
|
||||
{ key: "purchased", label: "Purchased", num: true, val: (r) => r.purchased || 0 },
|
||||
{ key: "total", label: "Total", num: true, val: (r) => r.total == null ? Infinity : r.total },
|
||||
{ key: "used", label: "Used (cycle)", num: true, val: (r) => usedCount(r) },
|
||||
{ key: "last_active_at", label: "Last active", val: (r) => new Date(r.last_active_at || 0).getTime() },
|
||||
];
|
||||
|
||||
function sortUsers(col) {
|
||||
const s = state.creditsSort;
|
||||
if (s.col === col) { s.dir = s.dir === "asc" ? "desc" : "asc"; }
|
||||
else { s.col = col; s.dir = (col === "credit_key" || col === "type") ? "asc" : "desc"; }
|
||||
refreshUsersTable();
|
||||
}
|
||||
function filterUsers(v) {
|
||||
state.creditsQuery = v;
|
||||
refreshUsersTable();
|
||||
}
|
||||
// Re-render ONLY the table (sort/filter) so the search box outside
|
||||
// the wrap keeps focus + caret between keystrokes.
|
||||
function refreshUsersTable() {
|
||||
const wrap = document.getElementById("users-table-wrap");
|
||||
if (wrap) wrap.innerHTML = usersTableHtml();
|
||||
}
|
||||
|
||||
function usersTableHtml() {
|
||||
const rows = (state.creditsData || []).slice();
|
||||
const q = (state.creditsQuery || "").trim().toLowerCase();
|
||||
const filtered = q
|
||||
? rows.filter((r) => (r.credit_key || "").toLowerCase().includes(q))
|
||||
: rows;
|
||||
if (filtered.length === 0) {
|
||||
return '<div class="empty">' + (q ? "No users match “" + esc(q) + "”." : "No users yet.") + '</div>';
|
||||
}
|
||||
const { col, dir } = state.creditsSort;
|
||||
const colDef = USERS_COLS.find((c) => c.key === col) || USERS_COLS[0];
|
||||
const mul = dir === "asc" ? 1 : -1;
|
||||
filtered.sort((a, b) => {
|
||||
const av = colDef.val(a), bv = colDef.val(b);
|
||||
if (typeof av === "number" && typeof bv === "number") return (av - bv) * mul;
|
||||
return String(av).localeCompare(String(bv)) * mul;
|
||||
});
|
||||
|
||||
const thead = USERS_COLS.map((c) => {
|
||||
const ind = c.key === col ? (dir === "asc" ? " ▲" : " ▼") : "";
|
||||
return '<th class="' + (c.num ? "num " : "") + '" style="cursor:pointer;" onclick="sortUsers(\'' + c.key + '\')">'
|
||||
+ esc(c.label) + '<span style="opacity:0.7;font-size:9px;">' + ind + '</span></th>';
|
||||
}).join("") + '<th>Grant</th>';
|
||||
|
||||
const tbody = filtered.map((r) => {
|
||||
const typeBadge = '<span style="display:inline-block;padding:1px 7px;border-radius:999px;font-size:10px;'
|
||||
+ 'background:rgba(165,180,252,0.12);border:1px solid var(--line-2);color:var(--fg-dim);">'
|
||||
+ esc(r.type) + '</span>';
|
||||
const grantCell = '<div style="display:flex;gap:5px;align-items:center;">'
|
||||
+ '<input type="number" min="1" step="1" placeholder="N" class="grant-amt" '
|
||||
+ 'style="width:62px;padding:4px 6px;font-size:12px;background:var(--bg);border:1px solid var(--line-2);border-radius:5px;color:var(--fg);" />'
|
||||
+ '<button class="grant-btn" data-key="' + esc(r.credit_key) + '" onclick="grantCredits(this)" '
|
||||
+ 'style="padding:4px 10px;font-size:11px;background:transparent;border:1px solid var(--line-2);border-radius:5px;color:var(--accent);cursor:pointer;">Grant</button>'
|
||||
+ '</div>';
|
||||
return '<tr>'
|
||||
+ '<td><code title="' + esc(r.credit_key) + '">' + esc(shortId(r.credit_key)) + '</code></td>'
|
||||
+ '<td>' + typeBadge + '</td>'
|
||||
+ '<td>' + tierPill(r.tier_snapshot) + '</td>'
|
||||
+ '<td class="num">' + fmtCredits(r.remaining) + '</td>'
|
||||
+ '<td class="num">' + fmtInt(r.purchased || 0) + '</td>'
|
||||
+ '<td class="num">' + fmtCredits(r.total) + '</td>'
|
||||
+ '<td class="num" title="' + (r.capped === "monthly" ? "monthly allowance" : "lifetime allowance") + '">' + fmtInt(usedCount(r)) + '</td>'
|
||||
+ '<td><span class="dim">' + esc(fmtTs(r.last_active_at)) + '</span></td>'
|
||||
+ '<td>' + grantCell + '</td>'
|
||||
+ '</tr>';
|
||||
}).join("");
|
||||
|
||||
return '<table><thead><tr>' + thead + '</tr></thead><tbody>' + tbody + '</tbody></table>';
|
||||
}
|
||||
|
||||
async function grantCredits(btn) {
|
||||
const key = btn.dataset.key;
|
||||
const input = btn.parentElement.querySelector(".grant-amt");
|
||||
const amount = parseInt(input && input.value, 10);
|
||||
if (!Number.isInteger(amount) || amount <= 0) {
|
||||
alert("Enter a positive whole number of credits.");
|
||||
return;
|
||||
}
|
||||
if (!confirm("Grant " + amount + " free credit(s) to " + key + "?\n\nThese never expire and are spent AFTER the user's tier allowance.")) {
|
||||
return;
|
||||
}
|
||||
btn.disabled = true;
|
||||
try {
|
||||
const r = await fetch("/admin/credits/grant", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ credit_key: key, amount }),
|
||||
});
|
||||
const data = await r.json().catch(() => ({}));
|
||||
if (!r.ok) throw new Error(data.error || ("HTTP " + r.status));
|
||||
// Refetch enriched rows so Remaining/Total/Purchased reflect the grant.
|
||||
await loadCredits();
|
||||
} catch (err) {
|
||||
alert("Grant failed: " + (err?.message || err));
|
||||
btn.disabled = false;
|
||||
}
|
||||
}
|
||||
|
||||
function renderUsersTab() {
|
||||
const data = state.creditsData;
|
||||
let body;
|
||||
if (data == null) {
|
||||
body = '<div class="loading">' + (state.creditsLoading ? "Loading users…" : "No data.") + '</div>';
|
||||
} else {
|
||||
const counts = data.reduce((acc, r) => { acc[r.type] = (acc[r.type] || 0) + 1; return acc; }, {});
|
||||
const summary = '<div style="font-size:11px;color:var(--fg-dim);margin-bottom:12px;">'
|
||||
+ fmtInt(data.length) + ' user(s) — '
|
||||
+ (counts.cloud || 0) + ' cloud · ' + (counts.license || 0) + ' license · ' + (counts.install || 0) + ' install. '
|
||||
+ 'Grants land in the never-expires top-up bucket (spent after the tier allowance).'
|
||||
+ '</div>';
|
||||
const search = '<input type="text" placeholder="Filter by key…" value="' + esc(state.creditsQuery || "") + '" '
|
||||
+ 'oninput="filterUsers(this.value)" '
|
||||
+ 'style="margin-bottom:12px;width:260px;padding:6px 10px;font-size:12px;background:var(--bg);border:1px solid var(--line-2);border-radius:5px;color:var(--fg);" />';
|
||||
body = summary + search + '<div id="users-table-wrap">' + usersTableHtml() + '</div>';
|
||||
}
|
||||
root.innerHTML =
|
||||
'<h1>Recap Relay — Operator Dashboard</h1>' +
|
||||
tabsHtml() +
|
||||
'<div style="max-width:1100px; padding:12px 0;">' +
|
||||
body +
|
||||
'</div>';
|
||||
}
|
||||
|
||||
function table(headers, rows) {
|
||||
if (rows.length === 0) return '<div class="empty">No data in this range.</div>';
|
||||
const thead = headers.map(h => '<th>' + esc(h) + '</th>').join("");
|
||||
|
||||
+7
-1
@@ -17,6 +17,7 @@ import { initCredits } from "./credits.js";
|
||||
import { initAuditLog } from "./audit-log.js";
|
||||
import { initJobCredits } from "./job-credits.js";
|
||||
import { initOutputStore } from "./output-store.js";
|
||||
import { initWebhookDedup } from "./webhook-dedup.js";
|
||||
import {
|
||||
setupAdminAuthMiddleware,
|
||||
setupAdminAuthRoutes,
|
||||
@@ -49,9 +50,14 @@ await initCredits({ dataDir: DATA_DIR });
|
||||
await initJobCredits({ dataDir: DATA_DIR });
|
||||
await initAuditLog({ dataDir: DATA_DIR });
|
||||
await initOutputStore({ dataDir: DATA_DIR });
|
||||
await initWebhookDedup({ dataDir: DATA_DIR });
|
||||
|
||||
const app = express();
|
||||
app.use(cors());
|
||||
// CORS only on /relay/* — those are the cross-origin clients (the Recaps
|
||||
// app + cloud server). /admin/* and the dashboard are served same-origin
|
||||
// to the operator's browser, so they don't need (and shouldn't get) the
|
||||
// permissive Access-Control-Allow-Origin a global cors() would apply.
|
||||
app.use("/relay", cors());
|
||||
app.use(cookieParser());
|
||||
|
||||
// Admin auth must run BEFORE the admin routes register so the cookie
|
||||
|
||||
+72
-2
@@ -8,8 +8,8 @@
|
||||
// action but reachable from the dashboard)
|
||||
|
||||
import express from "express";
|
||||
import { getConfigSnapshot, getTierPrices } from "../config.js";
|
||||
import { snapshotAll } from "../credits.js";
|
||||
import { getConfigSnapshot, getTierPrices, getTierQuotas } from "../config.js";
|
||||
import { snapshotAll, computeRemaining, addPurchasedCredits } from "../credits.js";
|
||||
import { snapshotCache } from "../keysat-client.js";
|
||||
// snapshotJobs is exported by BOTH ../jobs.js (the in-memory job
|
||||
// tracker) and ../job-credits.js (the credit-ledger). They return
|
||||
@@ -47,6 +47,16 @@ import { getHardwareQueueStatus } from "../hardware-queue.js";
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
|
||||
// Human-facing row category derived from the ledger credit-key prefix.
|
||||
// `user:` → cloud user (recaps.cc), `lic:` → shared license pool,
|
||||
// `inst:` and legacy bare-installId rows → a single install.
|
||||
function creditKeyType(key) {
|
||||
if (typeof key !== "string") return "install";
|
||||
if (key.startsWith("user:")) return "cloud";
|
||||
if (key.startsWith("lic:")) return "license";
|
||||
return "install";
|
||||
}
|
||||
|
||||
export function adminRouter({ dataDir }) {
|
||||
const router = express.Router();
|
||||
|
||||
@@ -58,6 +68,66 @@ export function adminRouter({ dataDir }) {
|
||||
});
|
||||
});
|
||||
|
||||
// ── Users / credit-balance view ──────────────────────────────────
|
||||
// Like /usage but enriched for the dashboard's Users tab: each row
|
||||
// carries a human-facing `type` (derived from the credit-key prefix)
|
||||
// and a COMPUTED balance (remaining tier credits + purchased top-up),
|
||||
// since the ledger stores consumed counters, not a remaining number.
|
||||
router.get("/credits", async (_req, res) => {
|
||||
const quotas = await getTierQuotas();
|
||||
const rows = snapshotAll().map((r) => {
|
||||
const balance = computeRemaining(r, quotas);
|
||||
return {
|
||||
...r,
|
||||
type: creditKeyType(r.credit_key),
|
||||
remaining: balance.remaining, // tier portion; null = unlimited
|
||||
purchased: balance.purchased,
|
||||
total: balance.total, // remaining + purchased; null = unlimited
|
||||
capped: balance.capped, // "monthly" | "lifetime"
|
||||
gemini_remaining: balance.gemini_remaining,
|
||||
};
|
||||
});
|
||||
res.json({ count: rows.length, rows });
|
||||
});
|
||||
|
||||
// Grant free credits to one user. Lands in the never-expires
|
||||
// `purchased_balance` bucket (spent AFTER the tier allotment), so this
|
||||
// is a pure top-up — it doesn't touch the user's monthly/lifetime
|
||||
// allowance. Only grants to an EXISTING ledger row: a typo'd key must
|
||||
// not spawn a ghost row, so we check snapshotAll() first.
|
||||
router.post("/credits/grant", express.json(), async (req, res) => {
|
||||
const creditKey =
|
||||
typeof req.body?.credit_key === "string" ? req.body.credit_key.trim() : "";
|
||||
const amount = Number(req.body?.amount);
|
||||
if (!creditKey) {
|
||||
return res.status(400).json({ error: "credit_key required" });
|
||||
}
|
||||
if (!Number.isInteger(amount) || amount <= 0) {
|
||||
return res.status(400).json({ error: "amount must be a positive integer" });
|
||||
}
|
||||
// Fat-finger guard — a manual top-up in the millions is almost
|
||||
// certainly a typo, not intent. Operator can repeat the grant if
|
||||
// they genuinely mean to add more.
|
||||
if (amount > 1_000_000) {
|
||||
return res.status(400).json({ error: "amount too large (max 1,000,000 per grant)" });
|
||||
}
|
||||
const exists = snapshotAll().some((r) => r.credit_key === creditKey);
|
||||
if (!exists) {
|
||||
return res.status(404).json({ error: "unknown credit_key" });
|
||||
}
|
||||
const newBalance = await addPurchasedCredits({ creditKey, amount });
|
||||
console.log(
|
||||
`[admin/credits] manual grant: +${amount} free credit(s) to ${creditKey} ` +
|
||||
`(new purchased_balance: ${newBalance})`
|
||||
);
|
||||
res.json({
|
||||
ok: true,
|
||||
credit_key: creditKey,
|
||||
granted: amount,
|
||||
purchased_balance: newBalance,
|
||||
});
|
||||
});
|
||||
|
||||
router.get("/config", async (_req, res) => {
|
||||
const cfg = await getConfigSnapshot();
|
||||
const hw = await (await import("../hardware-config.js")).resolveHardwareConfig(cfg);
|
||||
|
||||
+17
-19
@@ -49,18 +49,16 @@ import {
|
||||
BtcPayError,
|
||||
} from "../btcpay-client.js";
|
||||
import { recordCall } from "../audit-log.js";
|
||||
import { isWebhookProcessed, markWebhookProcessed } from "../webhook-dedup.js";
|
||||
import { envelope, errorEnvelope } from "./envelope.js";
|
||||
|
||||
// In-memory dedup of processed BTCPay invoice ids. BTCPay retries
|
||||
// webhook deliveries on non-2xx responses or network errors, so the
|
||||
// same InvoiceSettled event can land more than once. We don't want
|
||||
// to grant credits twice for one paid invoice.
|
||||
//
|
||||
// Cleared on relay restart, which means an unlucky webhook duplicate
|
||||
// straddling a restart would double-credit. Acceptable tradeoff for
|
||||
// v1 (operator can manually adjust the ledger if it happens), but
|
||||
// worth swapping for a persistent set once the relay sees real load.
|
||||
const processedInvoices = new Set();
|
||||
// Dedup of processed BTCPay invoice ids. BTCPay retries webhook
|
||||
// deliveries on non-2xx responses or network errors, so the same
|
||||
// InvoiceSettled event can land more than once — we don't want to grant
|
||||
// credits twice for one paid invoice. Backed by the persistent
|
||||
// webhook-dedup store (../webhook-dedup.js) so a duplicate straddling a
|
||||
// relay restart can't double-credit. Keys are namespaced
|
||||
// `<storeId>|<invoiceId>` to share the store with the Zaprite rail.
|
||||
|
||||
export function creditsRouter() {
|
||||
const router = express.Router();
|
||||
@@ -452,7 +450,7 @@ export function creditsRouter() {
|
||||
}
|
||||
|
||||
const dedupKey = `${cfg.relay_btcpay_store_id}|${invoiceId}`;
|
||||
if (processedInvoices.has(dedupKey)) {
|
||||
if (isWebhookProcessed(dedupKey)) {
|
||||
return res
|
||||
.status(200)
|
||||
.json({ ok: true, ignored: "already_processed", invoiceId });
|
||||
@@ -487,7 +485,7 @@ export function creditsRouter() {
|
||||
const subTier = meta.tier;
|
||||
const periodDays = Number(meta.period_days) || 30;
|
||||
if (!subUserId || (subTier !== "pro" && subTier !== "max")) {
|
||||
processedInvoices.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
return res
|
||||
.status(200)
|
||||
.json({ ok: true, ignored: "bad_tier_metadata", invoiceId });
|
||||
@@ -498,7 +496,7 @@ export function creditsRouter() {
|
||||
tier: subTier,
|
||||
periodDays,
|
||||
});
|
||||
processedInvoices.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
console.log(
|
||||
`[btcpay/webhook] ${subTier} +${periodDays}d for user ${subUserId.slice(0, 8)}… (invoice ${invoiceId}) → expires ${row.subscription_expires_at}`,
|
||||
);
|
||||
@@ -529,7 +527,7 @@ export function creditsRouter() {
|
||||
// Not ours — could be an invoice from another product on the
|
||||
// same store. Mark processed so we don't keep retrying, and
|
||||
// 200 so BTCPay stops calling.
|
||||
processedInvoices.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
return res
|
||||
.status(200)
|
||||
.json({ ok: true, ignored: "no_recap_metadata" });
|
||||
@@ -541,7 +539,7 @@ export function creditsRouter() {
|
||||
creditKey,
|
||||
amount: credits,
|
||||
});
|
||||
processedInvoices.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
await recordCall({
|
||||
install_id: installId,
|
||||
license_fingerprint: buyerFp,
|
||||
@@ -614,8 +612,8 @@ function rewriteCheckoutUrl(url, browserBase) {
|
||||
// Recovery: when the BTCPay webhook URL was broken and paid
|
||||
// invoices never got credited, this scans BTCPay's recent settled
|
||||
// invoices and grants credits for ones the relay hasn't processed.
|
||||
// Idempotent via the same processedInvoices dedup the webhook uses,
|
||||
// so re-running is safe.
|
||||
// Idempotent via the same persistent webhook-dedup store the webhook
|
||||
// uses, so re-running is safe.
|
||||
//
|
||||
// Exported so the admin route in btcpay-setup.js can call it. Not
|
||||
// exposed via /relay/* because it's operator-initiated, not buyer.
|
||||
@@ -660,7 +658,7 @@ export async function rescanSettledInvoices() {
|
||||
const details = [];
|
||||
for (const invoice of invoices || []) {
|
||||
const dedupKey = `${cfg.relay_btcpay_store_id}|${invoice.id}`;
|
||||
if (processedInvoices.has(dedupKey)) {
|
||||
if (isWebhookProcessed(dedupKey)) {
|
||||
alreadyProcessed++;
|
||||
continue;
|
||||
}
|
||||
@@ -684,7 +682,7 @@ export async function rescanSettledInvoices() {
|
||||
creditKey,
|
||||
amount: credits,
|
||||
});
|
||||
processedInvoices.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
await recordCall({
|
||||
install_id: installId,
|
||||
license_fingerprint: buyerFp,
|
||||
|
||||
@@ -15,16 +15,14 @@ import express from "express";
|
||||
import { extendUserTier } from "../credits.js";
|
||||
import { getZapriteConfig } from "../config.js";
|
||||
import { getOrder, isOrderPaid, orderIdFromWebhook } from "../zaprite-client.js";
|
||||
import { isWebhookProcessed, markWebhookProcessed } from "../webhook-dedup.js";
|
||||
|
||||
// In-memory dedup of fully-processed orders (mirrors the BTCPay handler's
|
||||
// processedInvoices). Zaprite retries on non-200, and may fire multiple
|
||||
// events per order, so we guard the grant. Cleared on restart — a
|
||||
// re-delivered webhook after restart re-fetches + re-grants, but
|
||||
// extendUserTier is keyed per order via this set within a process; a
|
||||
// duplicate grant across a restart is the same harmless "extend by one
|
||||
// period" the operator-comp path already tolerates. (Acceptable: card
|
||||
// double-fires across a restart are vanishingly rare.)
|
||||
const processedZaprite = new Set();
|
||||
// Dedup of fully-processed orders (mirrors the BTCPay handler). Zaprite
|
||||
// retries on non-200, and may fire multiple events per order, so we
|
||||
// guard the grant. Backed by the persistent webhook-dedup store
|
||||
// (../webhook-dedup.js) so a re-delivered webhook straddling a relay
|
||||
// restart can't double-extend a subscription. Keys are namespaced
|
||||
// `zaprite:<orderId>` to share the store with the BTCPay rail.
|
||||
|
||||
export function zapriteWebhookRouter() {
|
||||
const router = express.Router();
|
||||
@@ -49,7 +47,7 @@ export function zapriteWebhookRouter() {
|
||||
return res.status(200).json({ ok: true, ignored: "no_order_id" });
|
||||
}
|
||||
const dedupKey = `zaprite:${orderId}`;
|
||||
if (processedZaprite.has(dedupKey)) {
|
||||
if (isWebhookProcessed(dedupKey)) {
|
||||
return res
|
||||
.status(200)
|
||||
.json({ ok: true, ignored: "already_processed", orderId });
|
||||
@@ -91,7 +89,7 @@ export function zapriteWebhookRouter() {
|
||||
|
||||
const meta = order.metadata || {};
|
||||
if (meta.product !== "recap_tier_subscription") {
|
||||
processedZaprite.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
return res
|
||||
.status(200)
|
||||
.json({ ok: true, ignored: "not_a_tier_order", orderId });
|
||||
@@ -100,7 +98,7 @@ export function zapriteWebhookRouter() {
|
||||
const subTier = meta.tier;
|
||||
const periodDays = Number(meta.period_days) || 30;
|
||||
if (!subUserId || (subTier !== "pro" && subTier !== "max")) {
|
||||
processedZaprite.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
return res
|
||||
.status(200)
|
||||
.json({ ok: true, ignored: "bad_tier_metadata", orderId });
|
||||
@@ -112,7 +110,7 @@ export function zapriteWebhookRouter() {
|
||||
tier: subTier,
|
||||
periodDays,
|
||||
});
|
||||
processedZaprite.add(dedupKey);
|
||||
await markWebhookProcessed(dedupKey);
|
||||
console.log(
|
||||
`[zaprite/webhook] ${subTier} +${periodDays}d for user ${subUserId.slice(0, 8)}… (order ${orderId}) → expires ${row.subscription_expires_at}`,
|
||||
);
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
// The /admin/credits read view + /admin/credits/grant action behind the
|
||||
// Users dashboard tab. Mounts the REAL admin router (sans the cookie
|
||||
// auth middleware, which lives in index.js) over an ephemeral HTTP
|
||||
// server so the handler's validation runs for real, not just the
|
||||
// underlying ledger primitives.
|
||||
|
||||
import { test, describe, before, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import express from "express";
|
||||
|
||||
import { initCredits, setUserTier, addPurchasedCredits } from "../credits.js";
|
||||
import { adminRouter } from "../routes/admin.js";
|
||||
|
||||
let baseUrl;
|
||||
let server;
|
||||
|
||||
const getCredits = async () => {
|
||||
const r = await fetch(`${baseUrl}/admin/credits`, { cache: "no-store" });
|
||||
return { status: r.status, body: await r.json() };
|
||||
};
|
||||
const grant = async (payload) => {
|
||||
const r = await fetch(`${baseUrl}/admin/credits/grant`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(payload),
|
||||
});
|
||||
return { status: r.status, body: await r.json().catch(() => ({})) };
|
||||
};
|
||||
const rowFor = (body, key) => body.rows.find((r) => r.credit_key === key);
|
||||
|
||||
describe("/admin/credits + /admin/credits/grant", () => {
|
||||
before(async () => {
|
||||
await initCredits({ dataDir: mkdtempSync(path.join(tmpdir(), "relay-admincred-")) });
|
||||
// Seed one row of each ledger-key shape so the type derivation and
|
||||
// the per-tier balance math are both exercised. Default quotas:
|
||||
// Core lifetime=10, Pro monthly=50.
|
||||
await setUserTier({ userId: "alice", tier: "pro" }); // user:alice
|
||||
await addPurchasedCredits({ creditKey: "inst:bob", amount: 5 }); // Core install
|
||||
await addPurchasedCredits({ creditKey: "lic:deadbeefdeadbeef", amount: 2 }); // license pool
|
||||
|
||||
const app = express();
|
||||
app.use("/admin", adminRouter({ dataDir: "/tmp" }));
|
||||
await new Promise((resolve) => { server = app.listen(0, resolve); });
|
||||
baseUrl = `http://127.0.0.1:${server.address().port}`;
|
||||
});
|
||||
|
||||
after(() => { server?.close(); });
|
||||
|
||||
test("GET /credits derives a type from the credit-key prefix", async () => {
|
||||
const { status, body } = await getCredits();
|
||||
assert.equal(status, 200);
|
||||
assert.equal(rowFor(body, "user:alice").type, "cloud");
|
||||
assert.equal(rowFor(body, "inst:bob").type, "install");
|
||||
assert.equal(rowFor(body, "lic:deadbeefdeadbeef").type, "license");
|
||||
});
|
||||
|
||||
test("GET /credits computes remaining/total (not raw counters)", async () => {
|
||||
const { body } = await getCredits();
|
||||
const alice = rowFor(body, "user:alice"); // Pro, monthly 50, nothing spent
|
||||
assert.equal(alice.remaining, 50);
|
||||
assert.equal(alice.purchased, 0);
|
||||
assert.equal(alice.total, 50);
|
||||
const bob = rowFor(body, "inst:bob"); // Core lifetime 10 + 5 purchased
|
||||
assert.equal(bob.remaining, 10);
|
||||
assert.equal(bob.purchased, 5);
|
||||
assert.equal(bob.total, 15);
|
||||
});
|
||||
|
||||
test("POST /grant adds to the purchased bucket and returns the new balance", async () => {
|
||||
const { status, body } = await grant({ credit_key: "inst:bob", amount: 7 });
|
||||
assert.equal(status, 200);
|
||||
assert.equal(body.ok, true);
|
||||
assert.equal(body.granted, 7);
|
||||
assert.equal(body.purchased_balance, 12); // 5 seeded + 7 granted
|
||||
|
||||
// ...and the read view reflects it (purchased 12, total 10 + 12).
|
||||
const { body: after } = await getCredits();
|
||||
const bob = rowFor(after, "inst:bob");
|
||||
assert.equal(bob.purchased, 12);
|
||||
assert.equal(bob.total, 22);
|
||||
});
|
||||
|
||||
test("POST /grant rejects non-positive / non-integer amounts", async () => {
|
||||
for (const amount of [0, -5, 1.5, "x"]) {
|
||||
const { status } = await grant({ credit_key: "inst:bob", amount });
|
||||
assert.equal(status, 400, `amount=${amount} should be 400`);
|
||||
}
|
||||
});
|
||||
|
||||
test("POST /grant rejects an absurdly large amount (fat-finger guard)", async () => {
|
||||
const { status } = await grant({ credit_key: "inst:bob", amount: 2_000_000 });
|
||||
assert.equal(status, 400);
|
||||
});
|
||||
|
||||
test("POST /grant 400s on missing credit_key, 404s on an unknown one", async () => {
|
||||
assert.equal((await grant({ amount: 5 })).status, 400);
|
||||
const unknown = await grant({ credit_key: "inst:nobody", amount: 5 });
|
||||
assert.equal(unknown.status, 404);
|
||||
// The unknown key must NOT have been created as a side effect.
|
||||
const { body } = await getCredits();
|
||||
assert.equal(rowFor(body, "inst:nobody"), undefined);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,111 @@
|
||||
// Core billing primitives: commitCredit (debit), refundCredit (inverse),
|
||||
// and applyTierPromotion (the Core→paid upgrade bookkeeping). Default
|
||||
// quotas (no config file → getTierQuotas falls back): Core lifetime=10,
|
||||
// geminiCapLifetime=5; Pro monthly=50, geminiCapMonthly=25.
|
||||
|
||||
import { test, describe, before } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import {
|
||||
initCredits,
|
||||
getOrCreateRow,
|
||||
getUserCreditRow,
|
||||
setUserTier,
|
||||
commitCredit,
|
||||
refundCredit,
|
||||
applyTierPromotion,
|
||||
} from "../credits.js";
|
||||
|
||||
before(async () => {
|
||||
await initCredits({ dataDir: mkdtempSync(path.join(tmpdir(), "relay-money-")) });
|
||||
});
|
||||
|
||||
describe("commitCredit", () => {
|
||||
test("Core debits the lifetime bucket; Gemini tracked separately", async () => {
|
||||
const creditKey = "inst:core-commit";
|
||||
await commitCredit({ creditKey, tier: "core", backend: "gemini" });
|
||||
let row = await getOrCreateRow({ creditKey });
|
||||
assert.equal(row.lifetime_consumed, 1);
|
||||
assert.equal(row.lifetime_gemini_consumed, 1);
|
||||
|
||||
// A hardware call bumps lifetime but NOT the Gemini sub-counter.
|
||||
await commitCredit({ creditKey, tier: "core", backend: "hardware" });
|
||||
row = await getOrCreateRow({ creditKey });
|
||||
assert.equal(row.lifetime_consumed, 2);
|
||||
assert.equal(row.lifetime_gemini_consumed, 1);
|
||||
});
|
||||
|
||||
test("paid tier debits the monthly bucket", async () => {
|
||||
await setUserTier({ userId: "p-commit", tier: "pro" });
|
||||
await commitCredit({ creditKey: "user:p-commit", tier: "pro", backend: "gemini" });
|
||||
const row = await getUserCreditRow("p-commit");
|
||||
assert.equal(row.monthly_consumed, 1);
|
||||
assert.equal(row.monthly_gemini_consumed, 1);
|
||||
});
|
||||
|
||||
test("spend order: once the tier bucket is exhausted, debit purchased", async () => {
|
||||
const creditKey = "inst:core-exhausted";
|
||||
const row = await getOrCreateRow({ creditKey });
|
||||
row.lifetime_consumed = 10; // at the Core lifetime cap
|
||||
row.purchased_balance = 3;
|
||||
await commitCredit({ creditKey, tier: "core", backend: "hardware" });
|
||||
const after = await getOrCreateRow({ creditKey });
|
||||
assert.equal(after.lifetime_consumed, 10, "tier counter must not exceed the cap");
|
||||
assert.equal(after.purchased_balance, 2, "overflow comes out of purchased");
|
||||
});
|
||||
});
|
||||
|
||||
describe("refundCredit", () => {
|
||||
test("mirrors a Core commit (tier bucket first, Gemini sub-counter too)", async () => {
|
||||
const creditKey = "inst:core-refund";
|
||||
await commitCredit({ creditKey, tier: "core", backend: "gemini" });
|
||||
await refundCredit({ creditKey, tier: "core", backend: "gemini" });
|
||||
const row = await getOrCreateRow({ creditKey });
|
||||
assert.equal(row.lifetime_consumed, 0);
|
||||
assert.equal(row.lifetime_gemini_consumed, 0);
|
||||
});
|
||||
|
||||
test("refund with an empty tier bucket credits the purchased bucket", async () => {
|
||||
const creditKey = "inst:refund-to-purchased";
|
||||
const row = await getOrCreateRow({ creditKey });
|
||||
row.lifetime_consumed = 0;
|
||||
row.purchased_balance = 0;
|
||||
await refundCredit({ creditKey, tier: "core", backend: "hardware" });
|
||||
const after = await getOrCreateRow({ creditKey });
|
||||
assert.equal(after.lifetime_consumed, 0, "must floor at 0, not go negative");
|
||||
assert.equal(after.purchased_balance, 1, "refund lands in purchased when tier is already 0");
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyTierPromotion", () => {
|
||||
test("Core→paid transfers leftover Core credits to purchased and resets the cycle", async () => {
|
||||
const row = await getOrCreateRow({ creditKey: "inst:promo" });
|
||||
row.lifetime_consumed = 4; // 6 of 10 Core credits unused
|
||||
row.monthly_consumed = 2; // stale paid-counter noise that must be zeroed
|
||||
const promoted = await applyTierPromotion(row, "pro");
|
||||
assert.equal(promoted, true);
|
||||
assert.equal(row.tier_snapshot, "pro");
|
||||
assert.equal(row.purchased_balance, 6, "leftover Core credits carry forward as durable top-up");
|
||||
assert.equal(row.purchased_total_ever, 6);
|
||||
assert.equal(row.monthly_consumed, 0, "promotion starts a fresh monthly cycle");
|
||||
});
|
||||
|
||||
test("idempotent: a second promotion on an already-paid row is a no-op", async () => {
|
||||
const row = await getOrCreateRow({ creditKey: "inst:promo-again" });
|
||||
row.lifetime_consumed = 4;
|
||||
await applyTierPromotion(row, "pro"); // first: fires
|
||||
const purchasedAfterFirst = row.purchased_balance;
|
||||
const promoted = await applyTierPromotion(row, "max"); // second: must bail
|
||||
assert.equal(promoted, false);
|
||||
assert.equal(row.purchased_balance, purchasedAfterFirst, "no second leftover transfer");
|
||||
assert.equal(row.tier_snapshot, "pro", "bails before flipping tier again");
|
||||
});
|
||||
|
||||
test("promoting to Core is a no-op", async () => {
|
||||
const row = await getOrCreateRow({ creditKey: "inst:promo-core" });
|
||||
assert.equal(await applyTierPromotion(row, "core"), false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,70 @@
|
||||
// Persistent webhook dedup — the store that stops a settled-payment
|
||||
// webhook duplicate from double-crediting (BTCPay) or double-extending
|
||||
// (Zaprite) when the duplicate straddles a relay restart.
|
||||
|
||||
import { test, describe } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import {
|
||||
initWebhookDedup,
|
||||
isWebhookProcessed,
|
||||
markWebhookProcessed,
|
||||
} from "../webhook-dedup.js";
|
||||
|
||||
// The module is a singleton; each test inits a fresh dir to reset state.
|
||||
const freshDir = () => mkdtempSync(path.join(tmpdir(), "relay-dedup-"));
|
||||
|
||||
describe("webhook dedup (persistent)", () => {
|
||||
test("unknown key is not processed; marking makes it processed", async () => {
|
||||
await initWebhookDedup({ dataDir: freshDir() });
|
||||
assert.equal(isWebhookProcessed("store1|inv1"), false);
|
||||
await markWebhookProcessed("store1|inv1");
|
||||
assert.equal(isWebhookProcessed("store1|inv1"), true);
|
||||
});
|
||||
|
||||
// Headline guarantee for this change.
|
||||
test("processed keys survive a restart (reload from disk)", async () => {
|
||||
const dir = freshDir();
|
||||
await initWebhookDedup({ dataDir: dir });
|
||||
await markWebhookProcessed("store1|inv-restart");
|
||||
// Simulate a relay restart: re-init from the SAME data dir.
|
||||
await initWebhookDedup({ dataDir: dir });
|
||||
assert.equal(
|
||||
isWebhookProcessed("store1|inv-restart"),
|
||||
true,
|
||||
"a key marked before restart must still be deduped after restart"
|
||||
);
|
||||
});
|
||||
|
||||
test("BTCPay and Zaprite keys share the store without colliding", async () => {
|
||||
await initWebhookDedup({ dataDir: freshDir() });
|
||||
await markWebhookProcessed("store1|inv9");
|
||||
await markWebhookProcessed("zaprite:order9");
|
||||
assert.equal(isWebhookProcessed("store1|inv9"), true);
|
||||
assert.equal(isWebhookProcessed("zaprite:order9"), true);
|
||||
assert.equal(isWebhookProcessed("zaprite:inv9"), false);
|
||||
});
|
||||
|
||||
test("marking is idempotent", async () => {
|
||||
await initWebhookDedup({ dataDir: freshDir() });
|
||||
await markWebhookProcessed("k");
|
||||
await markWebhookProcessed("k"); // must not throw or duplicate
|
||||
assert.equal(isWebhookProcessed("k"), true);
|
||||
});
|
||||
|
||||
test("stale entries past the retention window are pruned on load", async () => {
|
||||
const dir = freshDir();
|
||||
const ancient = new Date(Date.now() - 365 * 24 * 60 * 60 * 1000).toISOString();
|
||||
const recent = new Date().toISOString();
|
||||
writeFileSync(
|
||||
path.join(dir, "processed-webhooks.json"),
|
||||
JSON.stringify({ keys: { "store1|old": ancient, "store1|new": recent } })
|
||||
);
|
||||
await initWebhookDedup({ dataDir: dir });
|
||||
assert.equal(isWebhookProcessed("store1|old"), false, "1y-old key should be pruned");
|
||||
assert.equal(isWebhookProcessed("store1|new"), true, "recent key should survive");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,93 @@
|
||||
// Persistent dedup of settled-payment webhook deliveries — BTCPay
|
||||
// invoices AND Zaprite orders share this one store. It replaces the
|
||||
// per-process in-memory Sets the two webhook handlers used to keep:
|
||||
// those were cleared on restart, so a duplicate delivery that straddled
|
||||
// a relay restart would double-credit (BTCPay) or double-extend a
|
||||
// subscription (Zaprite). Persisting the processed-keys to disk closes
|
||||
// that window.
|
||||
//
|
||||
// JSON-file backed (single file at <dataDir>/processed-webhooks.json),
|
||||
// same rationale as the credit ledger in credits.js: at most one
|
||||
// mutation per settled payment, so a plain JSON file with serial writes
|
||||
// is plenty.
|
||||
//
|
||||
// Keys are namespaced by their caller so the two rails can't collide in
|
||||
// the shared store: BTCPay uses `<storeId>|<invoiceId>`, Zaprite uses
|
||||
// `zaprite:<orderId>`. We store key → ISO timestamp (not a bare set) so
|
||||
// entries far older than any processor's retry window can be pruned on
|
||||
// load, keeping the file bounded.
|
||||
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
|
||||
// Payment processors retry webhook delivery for hours-to-days, never
|
||||
// months. 180 days is safely beyond any retry window, so pruning past
|
||||
// it cannot re-open a double-grant gap.
|
||||
const RETENTION_MS = 180 * 24 * 60 * 60 * 1000;
|
||||
|
||||
let storePath = null;
|
||||
let processed = {}; // key → ISO timestamp
|
||||
let writing = null; // serializes concurrent writes
|
||||
|
||||
export async function initWebhookDedup({ dataDir } = {}) {
|
||||
const dir = dataDir || "/data";
|
||||
storePath = path.join(dir, "processed-webhooks.json");
|
||||
await fs.mkdir(dir, { recursive: true }).catch(() => {});
|
||||
try {
|
||||
const raw = await fs.readFile(storePath, "utf8");
|
||||
const parsed = JSON.parse(raw);
|
||||
processed =
|
||||
parsed && typeof parsed === "object" && parsed.keys ? parsed.keys : {};
|
||||
} catch (err) {
|
||||
if (err.code !== "ENOENT") {
|
||||
console.warn(
|
||||
`[webhook-dedup] failed to read ${storePath}: ${err.message} — starting empty`
|
||||
);
|
||||
}
|
||||
processed = {};
|
||||
}
|
||||
// Prune anything older than the retention window so the file stays
|
||||
// bounded over the relay's lifetime.
|
||||
const cutoff = Date.now() - RETENTION_MS;
|
||||
let pruned = 0;
|
||||
for (const [k, ts] of Object.entries(processed)) {
|
||||
const t = Date.parse(ts);
|
||||
if (!Number.isFinite(t) || t < cutoff) {
|
||||
delete processed[k];
|
||||
pruned += 1;
|
||||
}
|
||||
}
|
||||
if (pruned > 0) await persist();
|
||||
console.log(
|
||||
`[webhook-dedup] loaded ${Object.keys(processed).length} processed key(s)` +
|
||||
`${pruned ? `, pruned ${pruned} stale` : ""} from ${storePath}`
|
||||
);
|
||||
}
|
||||
|
||||
async function persist() {
|
||||
// Coalesce concurrent writes — mirrors credits.js persist().
|
||||
if (writing) await writing;
|
||||
writing = (async () => {
|
||||
const tmp = storePath + ".tmp";
|
||||
await fs.writeFile(tmp, JSON.stringify({ keys: processed }), { mode: 0o600 });
|
||||
await fs.rename(tmp, storePath);
|
||||
})();
|
||||
try {
|
||||
await writing;
|
||||
} finally {
|
||||
writing = null;
|
||||
}
|
||||
}
|
||||
|
||||
export function isWebhookProcessed(key) {
|
||||
return Object.prototype.hasOwnProperty.call(processed, key);
|
||||
}
|
||||
|
||||
// Mark a webhook key processed and persist immediately. Callers invoke
|
||||
// this AFTER the grant/extend succeeds, so a failed grant leaves the key
|
||||
// unmarked and a processor retry can complete it. Idempotent.
|
||||
export async function markWebhookProcessed(key) {
|
||||
if (isWebhookProcessed(key)) return;
|
||||
processed[key] = new Date().toISOString();
|
||||
await persist();
|
||||
}
|
||||
@@ -31,15 +31,15 @@ export const manifest = setupManifest({
|
||||
start: null,
|
||||
stop: null,
|
||||
},
|
||||
// Relay has no REQUIRED dependencies — Gemini is internet-fronted
|
||||
// and the optional Parakeet/Gemma backends are at user-configured
|
||||
// URLs (typically a separate machine on the operator's LAN).
|
||||
//
|
||||
// BTCPay Server is declared optional so the dashboard's "Connect
|
||||
// BTCPay" flow can auto-discover its URL via
|
||||
// sdk.serviceInterface.getAll() when both are installed on the
|
||||
// same Start9 box. When BTCPay is not installed, the relay still
|
||||
// runs fine — only the credit-purchase flow is disabled.
|
||||
// BTCPay Server is a REQUIRED running dependency (optional: false
|
||||
// here, kind: 'running' in dependencies.ts). It's the only payment
|
||||
// rail through which the operator actually gets paid, so the relay
|
||||
// shouldn't run without it. The dependency also wires up the internal
|
||||
// docker hostname (btcpayserver.startos:23000) the relay-to-BTCPay API
|
||||
// calls rely on, and lets the dashboard's "Connect BTCPay" flow
|
||||
// auto-discover the URL via sdk.serviceInterface.getAll(). (Gemini is
|
||||
// internet-fronted and the Parakeet/Gemma backends are at
|
||||
// user-configured LAN URLs, so neither is a StartOS dependency.)
|
||||
dependencies: {
|
||||
btcpayserver: {
|
||||
description: {
|
||||
|
||||
@@ -125,8 +125,10 @@ import { v_0_2_121 } from './v0.2.121'
|
||||
import { v_0_2_122 } from './v0.2.122'
|
||||
import { v_0_2_123 } from './v0.2.123'
|
||||
import { v_0_2_124 } from './v0.2.124'
|
||||
import { v_0_2_125 } from './v0.2.125'
|
||||
import { v_0_2_126 } from './v0.2.126'
|
||||
|
||||
export const versionGraph = VersionGraph.of({
|
||||
current: v_0_2_124,
|
||||
other: [v_0_2_123, v_0_2_122, v_0_2_121, v_0_2_120, v_0_2_119, v_0_2_118, v_0_2_117, v_0_2_116, v_0_2_115, v_0_2_114, v_0_2_113, v_0_2_112, v_0_2_111, v_0_2_110, v_0_2_109, v_0_2_108, v_0_2_107, v_0_2_106, v_0_2_105, v_0_2_104, v_0_2_103, v_0_2_102, v_0_2_101, v_0_2_100, v_0_2_99, v_0_2_98, v_0_2_97, v_0_2_96, v_0_2_95, v_0_2_94, v_0_2_93, v_0_2_92, v_0_2_91, v_0_2_90, v_0_2_89, v_0_2_88, v_0_2_87, v_0_2_86, v_0_2_85, v_0_2_84, v_0_2_83, v_0_2_82, v_0_2_81, v_0_2_80, v_0_2_79, v_0_2_78, v_0_2_77, v_0_2_76, v_0_2_75, v_0_2_74, v_0_2_73, v_0_2_72, v_0_2_71, v_0_2_70, v_0_2_69, v_0_2_68, v_0_2_67, v_0_2_66, v_0_2_65, v_0_2_64, v_0_2_63, v_0_2_62, v_0_2_61, v_0_2_60, v_0_2_59, v_0_2_58, v_0_2_57, v_0_2_56, v_0_2_55, v_0_2_54, v_0_2_53, v_0_2_52, v_0_2_51, v_0_2_50, v_0_2_49, v_0_2_48, v_0_2_47, v_0_2_46, v_0_2_45, v_0_2_44, v_0_2_43, v_0_2_42, v_0_2_41, v_0_2_40, v_0_2_39, v_0_2_38, v_0_2_37, v_0_2_36, v_0_2_35, v_0_2_34, v_0_2_33, v_0_2_32, v_0_2_31, v_0_2_30, v_0_2_29, v_0_2_28, v_0_2_27, v_0_2_26, v_0_2_25, v_0_2_24, v_0_2_23, v_0_2_22, v_0_2_21, v_0_2_20, v_0_2_19, v_0_2_18, v_0_2_17, v_0_2_16, v_0_2_15, v_0_2_14, v_0_2_13, v_0_2_12, v_0_2_11, v_0_2_10, v_0_2_9, v_0_2_8, v_0_2_7, v_0_2_6, v_0_2_5, v_0_2_4, v_0_2_3, v_0_2_2, v_0_2_1, v_0_2_0, v_0_1_0],
|
||||
current: v_0_2_126,
|
||||
other: [v_0_2_125, v_0_2_124, v_0_2_123, v_0_2_122, v_0_2_121, v_0_2_120, v_0_2_119, v_0_2_118, v_0_2_117, v_0_2_116, v_0_2_115, v_0_2_114, v_0_2_113, v_0_2_112, v_0_2_111, v_0_2_110, v_0_2_109, v_0_2_108, v_0_2_107, v_0_2_106, v_0_2_105, v_0_2_104, v_0_2_103, v_0_2_102, v_0_2_101, v_0_2_100, v_0_2_99, v_0_2_98, v_0_2_97, v_0_2_96, v_0_2_95, v_0_2_94, v_0_2_93, v_0_2_92, v_0_2_91, v_0_2_90, v_0_2_89, v_0_2_88, v_0_2_87, v_0_2_86, v_0_2_85, v_0_2_84, v_0_2_83, v_0_2_82, v_0_2_81, v_0_2_80, v_0_2_79, v_0_2_78, v_0_2_77, v_0_2_76, v_0_2_75, v_0_2_74, v_0_2_73, v_0_2_72, v_0_2_71, v_0_2_70, v_0_2_69, v_0_2_68, v_0_2_67, v_0_2_66, v_0_2_65, v_0_2_64, v_0_2_63, v_0_2_62, v_0_2_61, v_0_2_60, v_0_2_59, v_0_2_58, v_0_2_57, v_0_2_56, v_0_2_55, v_0_2_54, v_0_2_53, v_0_2_52, v_0_2_51, v_0_2_50, v_0_2_49, v_0_2_48, v_0_2_47, v_0_2_46, v_0_2_45, v_0_2_44, v_0_2_43, v_0_2_42, v_0_2_41, v_0_2_40, v_0_2_39, v_0_2_38, v_0_2_37, v_0_2_36, v_0_2_35, v_0_2_34, v_0_2_33, v_0_2_32, v_0_2_31, v_0_2_30, v_0_2_29, v_0_2_28, v_0_2_27, v_0_2_26, v_0_2_25, v_0_2_24, v_0_2_23, v_0_2_22, v_0_2_21, v_0_2_20, v_0_2_19, v_0_2_18, v_0_2_17, v_0_2_16, v_0_2_15, v_0_2_14, v_0_2_13, v_0_2_12, v_0_2_11, v_0_2_10, v_0_2_9, v_0_2_8, v_0_2_7, v_0_2_6, v_0_2_5, v_0_2_4, v_0_2_3, v_0_2_2, v_0_2_1, v_0_2_0, v_0_1_0],
|
||||
})
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
import { VersionInfo } from '@start9labs/start-sdk'
|
||||
|
||||
export const v_0_2_125 = VersionInfo.of({
|
||||
version: '0.2.125:0',
|
||||
releaseNotes: {
|
||||
en_US: 'Add Users tab to the operator dashboard: per-user credit balances with a grant-free-credits action',
|
||||
},
|
||||
migrations: {
|
||||
up: async ({ effects }) => {},
|
||||
down: async ({ effects }) => {},
|
||||
},
|
||||
})
|
||||
@@ -0,0 +1,12 @@
|
||||
import { VersionInfo } from '@start9labs/start-sdk'
|
||||
|
||||
export const v_0_2_126 = VersionInfo.of({
|
||||
version: '0.2.126:0',
|
||||
releaseNotes: {
|
||||
en_US: 'Persist payment-webhook dedup across restarts (no double-credit/double-extend); declare BTCPay a required dependency; scope CORS to /relay/*; add money-path + webhook-dedup tests; doc fixes',
|
||||
},
|
||||
migrations: {
|
||||
up: async ({ effects }) => {},
|
||||
down: async ({ effects }) => {},
|
||||
},
|
||||
})
|
||||
Reference in New Issue
Block a user