Compare commits

..

68 Commits

Author SHA1 Message Date
Keysat f2188fa797 Handoff: correct CSS-token convention; refresh Current state for 0.2.161
- Conventions: inline styles are now var()-ified (Phase 2), and the
  SHARE_PAGE_* export is pure literal hex with no :root (corrects the
  prior "three surfaces each carry a :root copy" claim). Records the
  CSS-value-position scoping rule for future sweeps.
- Current state: tighten the design-system entry to a snapshot now that
  detail lives in Conventions + ROADMAP.
2026-06-17 10:57:26 -05:00
Keysat 64f3e6628e Ignore the server-minted install-id file
In local dev the server writes its per-install UUID to the repo root
(on the box it lives at /data/install-id), so it kept showing up as
untracked. It's machine-local runtime identity — never commit it.
2026-06-17 09:23:31 -05:00
Keysat 82e544af47 Var-ify inline styles and snap off-scale type/radii (design Phase 2)
Phase 2 of the design-contract cleanup:
- 346 inline-style hexes (+7 #475569, mapped by property) -> var(--token),
  scoped to CSS-value position so JS-logic/quoted hex, the meta theme-color,
  SVG attrs, and the no-:root share-export region stay literal; #fff and
  no-token hexes left as-is.
- Snap off-scale font sizes (9/10.5->10, 11.5/12.5->12, 15->16, 24->22) and
  radii (3->4, 5->6, 7->6, 11->12, 9->8|10) to the scale.
- Bump to 0.2.161, which also ships the previously-uninstalled 0.2.160
  share-page HTML export.
2026-06-17 08:22:48 -05:00
Keysat d3ab281baa Record CSS-token convention; refresh Current state for the design system 2026-06-17 07:41:36 -05:00
Keysat 211287aed5 Consolidate UI colors behind CSS custom properties; fix design drift
Phase 1 of the design-contract conformance cleanup. Add a canonical
:root token block (single source of truth, mirroring
design/tokens.tokens.json) to public/index.html's stylesheet and migrate
the whole <style> block to var(--token); give public/auth.html its own
subset :root and migrate it too.

Fix all color + weight drift across every surface (stylesheet, inline
styles, JS handlers, the SHARE_PAGE_CSS export):
- legacy indigos #6366f1/#4f46e5/#4338ca + rgba(99,102,241) ->
  #818cf8/#a5b4fc/rgba(129,140,248)
- blue #3b82f6 interactive buttons (incl. the whole auth screen) -> indigo
- legacy darks #0a0e17/#0b1120/#020617/#121828/#1f2942 -> the surface ladder
- #f5f9ff -> #f1f5f9, #312e81 -> #1e293b, weights 650->600 / 680->700

The meta theme-color stays a literal #0a0e1a. Verified: 144 tests pass,
both pages serve 200, all var() references resolve. Phase 2 (var-ifying
the long-tail inline styles, snapping off-scale font/radius) is in
ROADMAP.md.
2026-06-16 23:36:42 -05:00
Keysat 1741fb11a5 Add design/ contract extracted from the as-built UI
Inventory the as-built recaps.cc look and distill it into a durable,
vendor-neutral design contract: design/DESIGN.md (nine-section brand
brief) + design/tokens.tokens.json (W3C DTCG tokens), plus brand icon
and provenance notes. Canonical calls reconciled with the owner:
indigo #818cf8 as the single interactive accent, purple #a855f7 for
premium only, the #0a0e1a->#111827->#0f172a surface ladder, and a
normalized type scale. Wire the AGENTS.md Design line and record the
contract-vs-code drift as a cleanup backlog in ROADMAP.md.
2026-06-16 23:08:41 -05:00
Keysat c9ad731860 Add Gemini 3.5 model-selection idea to backlog
Triaged from the cross-project inbox. Needs a research pass to confirm
available stable Gemini versions before wiring; flag the server+client
model-list duplication and the matching relay-side capture.
2026-06-16 21:43:39 -05:00
Keysat 621af7ca14 Add self-contained shareable HTML export for YouTube recaps
New 'Share page (HTML)' entry in the Export menu generates a single
standalone .html file: the recap record inlined as JSON plus a small
baked-in renderer that reproduces the embedded YouTube player and the
expandable timestamped summaries. The recipient opens it with no account
and no calls back to the server. On mobile it hands the file to the
native share sheet (navigator.share with files); on desktop or where
unsupported it downloads.

YouTube only by design — podcasts have no portable audio and are
rejected with a toast. The generator runs inside index.html's own
<script>, so closing tags are split and inlined JSON escapes '<' to
avoid premature script termination.

Ship as 0.2.160.
2026-06-16 07:57:30 -05:00
Keysat f38ecc6c86 Handoff: record server/client URL-parser duplication; bump Current state to 0.2.159 2026-06-15 23:33:33 -05:00
Keysat cb961cd2d9 Accept YouTube /live/ and /shorts/ URLs in extractVideoId
The video-id regex only matched /watch?v=, youtu.be, /embed/, and /v/
forms, so youtube.com/live/<id> and youtube.com/shorts/<id> links were
rejected with "Invalid YouTube URL". Add both forms to the server and
frontend extractors (kept in sync) and cover them with tests.

Ship as 0.2.159.
2026-06-15 23:29:57 -05:00
Keysat f9367c2ae5 Handoff: record operator-absorbed relay convention; condense Current state 2026-06-15 20:10:45 -05:00
Keysat be9692daa7 Record 0.2.158 Daily Digest live on the box 2026-06-15 19:53:39 -05:00
Keysat b4fa5d7be8 Add opt-in Daily Digest (daily email of last 24h of library recaps)
Multi-mode, off by default. Each new recap is synthesized into a 1-2
paragraph overview via the relay (operator-absorbed) and cached onto the
session JSON; a daily 08:00 scan emails opted-in users their fresh
recaps, deduped by a per-user watermark that never skips a failed or
over-cap recap. One-click tokenized unsubscribe; settings-modal toggle;
admin test trigger. Bumps to 0.2.158.
2026-06-15 19:50:48 -05:00
Keysat 962423ca10 Add Daily Digest plan; record render-loop invariants + deploy model in AGENTS.md 2026-06-15 18:31:08 -05:00
Keysat 693bb981ff Fix mobile/UX bug cluster: video minimize, audio interrupt, scroll reset, redundant box
Four fixes in public/index.html, all reported against recaps.cc on mobile:

- Video minimize no longer shows a black frame on expand. toggleVideoMinimize()
  used to call render(), rebuilding the YouTube iframe inside the display:none
  minimized container, which wedged the IFrame API. Minimize now toggles the
  .results-left.minimized class in place; a !videoMinimized guard on render()'s
  needsMount plus a new ensureYtMounted() (called from the expand paths) keep the
  player from ever being created in a hidden container.

- Background processing no longer interrupts podcast audio or resets the
  transcript scroll. The ~60s relay-credit poll calls render(), which rebuilt the
  <audio> element and chunks-scroll. render() now preserves the live <audio> node
  across the innerHTML swap (replaceWith when the src matches) and restores
  chunks-scroll scrollTop; initPodcastPlayer() is idempotent so the preserved node
  doesn't get duplicate listeners.

- Removed the redundant centered "Processing..." box; the staged pizza-tracker
  breadcrumb already covers that window.

- Added -webkit-overflow-scrolling/overscroll-behavior to .chunks-scroll for the
  mobile can't-scroll-to-top report (best-effort, needs on-device verification).

Ships as 0.2.157. Reviewer pass clean; inline JS syntax checked with node --check.
2026-06-15 17:38:32 -05:00
Keysat 91af0b711e Harden iOS sign-in against stale-connection POST failures
iPad users hit a spurious "network error" on the first tap of
"Send sign-in link", with a second tap succeeding. Cause is iOS
Safari dispatching the POST onto a pooled keep-alive socket the
server/proxy already closed; unlike a GET it isn't transparently
re-sent, so it surfaces as a transport TypeError. The single 500ms
auto-retry was too quick and reused the same dead socket.

Both sign-in entry points (auth.html postWithRetry, index.html
fetchWithRetry) now retry 3x with growing backoff (0 -> +400ms ->
+1.6s) to outlast Safari evicting the socket. Frontend-only.

Ships as 0.2.156.
2026-06-15 16:35:13 -05:00
Keysat aca2ba9e2e Handoff: record P0/P1 fixes as done, move eval debt to ROADMAP
Refresh AGENTS.md conventions (client IP via req.ip; safeFilename exported),
rewrite Current state to a lean snapshot, move the P2 known-debt detail to
ROADMAP.md.
2026-06-15 13:48:20 -05:00
Keysat d0e98424c1 Fix five P0/P1 security & correctness findings from the full-eval
- Arbitrary file write (P0): validate import keys in /api/library/import via
  a now-exported safeFilename(); a ../../ key is skipped, not written out of
  the scope dir.
- SSRF (P0): guard downloadPodcastAudio — reject non-HTTP(S) schemes, block
  IP-literal and DNS-resolved private/link-local/loopback/reserved/multicast
  and embedded-IPv4 IPv6 targets (closes DNS rebinding), cap + resolve redirects.
- ESM require (P1): top-level import of randomBytes in license-purchase.js
  (the inner require threw on the anon purchase-settle path).
- Concurrency lock (P1): skip the process-global free-tier slot in multi-mode
  so it no longer serializes every cloud tenant onto one job.
- X-Forwarded-For bypass (P1): set Express trust proxy from
  RECAP_TRUSTED_PROXY_HOPS (default 1); getClientIp now reads req.ip instead
  of a client-spoofable XFF entry.

Tests added for safeFilename, the SSRF guard, and getClientIp (119 pass).
Registry blockers deferred (ROADMAP); leaked-key history purge queued.
2026-06-15 13:36:40 -05:00
Keysat 755b100f00 Triage P3 eval findings into deferred backlog
Defer the low-severity full-eval findings (request caps, invoice-ID claim,
root container, rate-limit buckets, repo hygiene, StartOS polish, bulk doc
reconciliation) for later batching. P0/P1 and P2 live in AGENTS.md.
2026-06-15 12:30:58 -05:00
Keysat 6bb7e69141 Add inbox-check line; align .gitignore with canonical .claude policy
Cross-repo git-hygiene audit remediation: surface ~/Projects/standards/INBOX.md items at session start, and switch .gitignore to the deny-by-default .claude/* block (shared wiring allow-listed) plus the canonical secrets/env lines — per standards/portability.md.
2026-06-14 12:17:16 -05:00
Keysat d4c742d6e7 Add full-evaluation report
Independent six-lens evaluation (evaluator, security-auditor, exerciser,
doc-auditor, start9-spec-checker). Surfaces three P0s on the cloud surface
and a registry-submission block; full priority queue in the file.
2026-06-14 09:39:46 -05:00
Keysat 982e1b0d66 Retrofit: fix stale command docs, extract relay-client guide
- Replace the Commands-table Lint/Type-check TODOs with the real, verified
  commands: `npm run check` (tsc --noEmit over startos/) and `npm run prettier`.
  There is no ESLint/linter; server/ JS is untooled.
- Move the client-side relay contract (env vars, /relay/* endpoints, X-Recap-*
  headers, file map) out of AGENTS.md into docs/guides/relay-client.md with
  paths: frontmatter, lazy-loaded via a .claude/rules symlink; AGENTS.md keeps
  a one-line pointer.
- Un-ignore .claude/rules/ so the guide auto-attaches in any clone, while
  .claude/ local state (worktrees, plans) stays ignored.
2026-06-13 14:58:05 -05:00
Keysat 0ae59f3550 Add multi-tenant cloud mode: self-serve purchase, credit metering, core-decoupling
Introduces RECAP_MODE=multi alongside single-mode self-host:
- Tenant auth + accounts (magic-link via System SMTP), per-tenant credit pool,
  anonymous trial minting with per-IP/-64 caps
- Self-serve Pro/Max purchase: inline Lightning (BTCPay) + card (Zaprite),
  prepaid 30-day periods, expiry-reminder emails
- Core-decoupling: relay owns cloud tier/expiry keyed by Recaps user-id
- SQLite (better-sqlite3) schema for multi-mode; filesystem unchanged for single
- StartOS actions/versions through 0.2.155
2026-06-13 14:25:05 -05:00
Keysat db580abad7 Add cross-repo change-impact convention 2026-06-13 12:18:40 -05:00
Keysat 5b7df2f073 Complete client-side relay contract in AGENTS.md
Add the /relay/* endpoints the app actually calls that were omitted (capabilities, policy, tts, jobs/:id, credits/*); fix the Files attribution (add relay-capabilities.js + credits-purchase.js; the /relay/policy proxy lives in index.js only).
2026-06-13 12:02:36 -05:00
Keysat 4bba466665 Tidy relay-internal detail; add client-side relay contract
- Add Client-side contract with the relay sub-section: env vars
  (RECAP_RELAY_BASE_URL, RECAP_RELAY_OPERATOR_KEY ↔ relay_cloud_operator_key),
  auth direction the client SENDS, the 12 /relay/* endpoints the consumer
  actually calls (verified against providers/relay.js + billing-routes.js +
  subscription-reminders.js).
- Drop two relay-internal references now canonical in ../recap-relay/AGENTS.md:
  the extendUserTier function name and the Adjacent-repo bullet's
  "Private; ships via make install only" sentence.
2026-06-13 11:13:21 -05:00
Keysat b906b8a5c4 Add agent docs (AGENTS.md, ROADMAP.md, CLAUDE.md symlink) 2026-06-13 10:38:51 -05:00
Keysat 373d10595b Pluggable AI providers, relay credit system, picker UX overhaul
Captures roughly forty version bumps (v0.2.6 → v0.2.47) of work that
accumulated without commits.

- Pluggable provider system under server/providers/: gemini, anthropic,
  openai, openai-compatible, ollama, whisper-compatible, relay. Mix and
  match transcription + analysis per request via the picker UI.
- Relay backend integration. Hardcoded relay URL in server/relay-default.js
  (operator-controlled at build time, not user-configurable). New
  /api/relay/{status,policy} endpoints proxy to the relay; balance pings
  populate a cached credit display.
- Per-install identity in server/install-id.js for relay credit accounting.
  Sent to the relay as X-Recap-Install-Id; persists across upgrades, lost
  on a full uninstall + reinstall. Not surfaced in the UI.
- Admin login gate (server/admin-auth.js + setAdminPassword action). Scrypt
  password hash + HMAC-signed session cookie.
- Entitlement scheme rename: pro / max (each paired with subscriptions and
  relay_pro / relay_max), replacing the misleading "core" entitlement
  that conflicted with the user-facing "Core" tier name.
- Activation screen: dynamic credit count pulled from /api/relay/policy,
  "Skip — use free mode" button, accurate paid-feature list.
- Top toolbar: inline credit-balance pill (or "BYO configured" fallback),
  Upgrade + "I have a key" buttons.
- Picker UI: per-provider sections with Save/Test/Delete buttons, sections
  collapsible by chevron, default-collapsed unless currently selected,
  "Use comped credits (reset to relay)" link when the user has strayed,
  green hint under inputs whose values are server-configured.
- Activity log: chevron-collapsible groups per video, refresh-survival via
  localStorage + a 500-entry server-side buffer, explicit Clear button.
- YouTube captions fast-path with user toggle (skips audio download + AI
  transcription when captions are available — uncheck for speaker labels).
- Cancel button: AbortController plumbed through every provider SDK call;
  retryAPI short-circuits on AbortError; cancellation events surface in
  the activity log instead of silent retries.
- Long-video analysis: auto-coalesce transcript entries before building the
  analysis prompt so local-model context windows (32k-ish) don't overflow.
  Original entries preserved for transcript display via an index map; the
  analyzer sees a coarser view but click-to-seek timestamps stay precise.
- StartOS action grouping (Setup / AI Providers) so the actions list is
  navigable.
- Manifest description rewritten to reflect multi-provider support and
  free-tier relay credits.
- Smaller fixes: summarize-button enablement no longer requires a Gemini
  key when other providers are configured; analysis fallback chain handles
  context-length and 503 capacity errors; single-segment expansion for
  providers that don't return per-segment timestamps (Parakeet et al.);
  many other UX polish items.
2026-05-11 23:46:20 -05:00
Keysat 2544cf7dde Repoint FileBrowser upload from /websites/packages to /websites/keysat-registry
Aligns the upload path with the keysat.xyz registry-domain rebrand —
the directory name now matches the registry it serves.
2026-05-09 20:03:19 -05:00
Keysat 068335f43c Repoint deploy script defaults from satsflows.com to keysat.xyz
Both REGISTRY_URL (registry index re-index endpoint) and
REGISTRY_PUBLIC_URL (the URL embedded in the registry entry pointing
at the .s9pk download) now default to keysat.xyz. The hosts route to
the same data via StarTunnel; this just stops embedding the legacy
satsflows.com hostname in the registered package metadata.
2026-05-09 19:52:18 -05:00
Keysat 25ec40f345 Bump version to 0.2.5
Tighter license-poll cadence + opportunistic online refresh.
2026-05-09 19:36:47 -05:00
Keysat 9439154c25 Tighten license-poll cadence; add opportunistic online refresh
Three changes that together make license state changes feel
near-instant in the UI without burning real I/O / network budget:

  1. File-poll interval: 30s → 5s
     Action-set keys (via "Set Recap License") get picked up almost
     immediately. Cost: a single stat per file every 5s, negligible.

  2. Online validation interval: 6h → 30min
     A license revoked on Keysat now flips to invalid within 30 min
     worst-case, instead of sitting unnoticed for hours. Bounded
     latency makes revocation usable in production.

  3. Opportunistic online refresh on /api/license-status
     If the cached LIC was last validated more than 10 min ago, fire
     validateOnline() in the background (non-blocking) when the web
     UI hits the status endpoint. Since the UI hits status on every
     page load, revocations get caught the next time anyone opens
     the app — usually well under the scheduled 30 min tick.

Three new env vars for tuning:
  RECAP_LICENSE_FILE_POLL_MS         (default 5000)
  RECAP_VALIDATE_INTERVAL_MS         (default 1800000)
  RECAP_OPPORTUNISTIC_REFRESH_MS     (default 600000)
2026-05-09 19:36:46 -05:00
Keysat e5a779ced2 Strip prepare/build scripts from vendored keysat-client package.json
When npm install runs in server/ and processes the file: dep at
vendor/keysat-licensing-client/, it fires the vendor's 'prepare'
script even with --ignore-scripts (npm bug for file/git deps).
The prepare script calls tsup which isn't installed in the build
container, so the build fails with 'sh: 1: tsup: not found'.

The vendored copy ships its prebuilt dist/ in the repo, so we don't
need tsup. Drop the scripts entirely.
2026-05-09 11:59:55 -05:00
Keysat 823b9e0375 Bump version to 0.2.4
Critical fix for v0.2.3 startup crash (missing modules in Docker
image). Also refreshes vendored Keysat client to v0.2.0 ahead of
in-app buy flow.
2026-05-09 11:57:58 -05:00
Keysat 495b4aef36 Fix Dockerfile to copy all server/*.js modules; refresh vendor to v0.2.0
The runtime crash on v0.2.3:

  Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/app/server/util.js'
  imported from /app/server/index.js

happened because the Dockerfile's stage-2 COPY only listed server/
index.js + server/license.js explicitly. When I started extracting
modules in v0.2.3 (util.js, gemini-helpers.js, audio.js, ytdlp.js,
cookies.js, config.js, license-middleware.js, history.js, library.js)
I forgot to update the COPY list, so those files were never copied
into the runner image. Local 'node' tests passed because the modules
exist on disk; the .s9pk container had only the two original files
and crashed on first import.

Fix:

  COPY server/*.js ./server/

Glob picks up all top-level .js files automatically, including any
future extractions, while still skipping server/test/ and server/
node_modules/. This is the simplest forward-compatible form.

Bonus: refresh the vendored @keysat/licensing-client from 0.1.0 to
0.2.0. The new SDK adds:

  • policySlug field on StartPurchaseOptions (so we can drive Core/
    Pro tier selection programmatically from our backend)
  • client.listPublicPolicies(productSlug) for fetching the tier
    cards' data without auth

Both are prerequisites for the in-app buy flow planned in
~/.claude/plans/in-app-buy-flow.md. The vendor's own node_modules
(@noble/ed25519, @noble/hashes) is gitignored as before — Docker
builds re-install via `npm install --omit=dev --ignore-scripts` in
the vendor dir during stage 1.

Also includes the license-middleware update from earlier in the day:
a 30s license-file poll so a key set via the "Set Recap License"
StartOS action is picked up within seconds (instead of waiting for
the 6h scheduled validateOnline tick).
2026-05-09 11:57:41 -05:00
Keysat c06ffbbdf4 Module split: library export/import → server/library.js
• setupLibraryRoutes(app) — registers GET /api/library/export and
                              POST /api/library/import

The library module reads through history.js helpers (getHistoryDir,
loadMeta, saveMeta) and reads/writes subscriptions.json directly.
Subscriptions integration is via raw fs because (a) the library merge
logic is library-specific (skip-if-already-exists semantics), and (b)
the subscriptions module hasn't been extracted yet — the only thing
the import path needs is to merge dedupe-by-URL into the file.

server/index.js: 2079 → 1971 lines.

Smoke tested: server boots; /api/license-status, /api/health respond;
/api/library/export still returns 402 license_required for unlicensed
(unchanged Pro-gate behavior). 69 unit tests still pass.
2026-05-09 10:39:09 -05:00
Keysat a09ad9c429 Add unit tests for util / gemini-helpers / license / history modules
69 tests across 16 suites, ~120 ms total. Uses node:test (built into
Node 20+) — no new dependency, no Docker rebuild churn. Run with:

  cd server && npm test

Coverage:
  • util.js               extractVideoId, formatTime,
                          parseTimestampedTranscript, safeText,
                          retryGemini (incl. 503 retry, network-error
                          retry, non-retryable passthrough), sendEvent
  • gemini-helpers.js     PRICING table integrity, calcCost (model-
                          specific rates, default fallback, missing
                          fields, sub-cent ¢ formatting, totalTokens
                          precedence), buildAnalysisPrompt
  • license.js            checkLicense (no key, malformed, fallback to
                          startos-config.json, license.txt priority),
                          activate (bad-format throw, file write),
                          deactivate (file removal, idempotent),
                          publicView (no raw key leak, sorted
                          entitlements, ISO dates), has()
  • history.js            initHistory + getHistoryDir, saveToHistory
                          (id shape, defaults, podcast guid encoding),
                          loadMeta + saveMeta round-trip, corrupt-file
                          tolerance

Tests that need module-private file paths (license, history) use a
mkdtemp'd tmp dir as DATA_DIR + dynamic import() so each suite starts
clean. No test mocks the filesystem — they read/write real files
inside the tmp dir, matching production behavior.

Deliberately not yet covered (need an Express app harness or external
binaries): license-middleware (gate behavior), config (live-reload
poll), audio (ffmpeg/ffprobe), ytdlp (yt-dlp + git), cookies (state
mutation routes), the /api/process pipeline. Worth a follow-up after
the current refactor settles.
2026-05-09 10:36:12 -05:00
Keysat fe07580a12 Bump version to 0.2.3
New 'Set Recap License' StartOS action; split server/index.js into
8 focused modules (util, gemini-helpers, audio, ytdlp, cookies,
config, license-middleware, history); 'Recap license' copy fix.
2026-05-09 07:06:36 -05:00
Keysat 29282f8dcc Add 'Set Recap License' StartOS action + s/Keysat license/Recap license/
Two related changes:

1. New StartOS action: 'Set Recap License'

   Symmetric with the existing 'Set Gemini API Key' action — paste a
   LIC1-... key into the StartOS Actions menu and it gets persisted.
   Added because some users prefer the StartOS form for credentials
   over the in-app activation modal.

   Implementation:
     • startos/file-models/config.json.ts: schema gains recap_license_key
     • startos/actions/setLicense.ts: input form (masked, regex-checks
       for the LIC1- prefix), persists via configFile.merge()
     • startos/actions/index.ts: registers the new action
     • server/license.js: readLicenseString() falls back to
       startos-config.json after the legacy license.txt path. Resolution
       order: env → license.txt → startos-config.json
     • server/license-middleware.js: faster license-file poll (30 s,
       env-overridable RECAP_LICENSE_FILE_POLL_MS) re-runs checkLicense
       so action-set keys take effect within seconds, not the 6 h online
       cycle. If the new key parses as 'licensed', kicks an immediate
       online check to confirm.

2. Copy fix: 'Keysat license' → 'Recap license' in user-facing text

   Keysat is the licensing system underneath, but customers buy a
   'Recap license'. Updated:
     • Activation screen subtitle (public/index.html)
     • 402 message in the activation gate (server/license-middleware.js)

   Internal references (PRODUCT_SLUG, KEYSAT_BASE_URL, the issuer.pub
   filename, the 'Issuer: licensing.keysat.xyz' display in the
   activation card) stay as Keysat — those are accurate.

Smoke tested locally: starting the server with no license, then
writing a fake LIC1-... key into startos-config.json, the
license-file poll picks it up within ~2 s and transitions state from
'unlicensed' to 'invalid' (since the fake key fails Ed25519
verification, as expected). With a real key, the same path would land
in 'licensed'.
2026-05-09 07:06:21 -05:00
Keysat 85cb641044 Module split: history storage + meta + 9 routes → server/history.js
• initHistory({ dataDir })          — boot setup; mkdir + path init
  • saveToHistory(...)                — write one summary file
  • loadMeta() / saveMeta(meta)       — _meta.json folder structure
  • setupHistoryRoutes(app, deps)     — registers GET /api/history,
                                        GET/PUT/DELETE /api/history/:id,
                                        PUT /api/history/:id/title,
                                        PUT /api/history/meta,
                                        POST/PUT/DELETE /api/history/folders[/:id],
                                        PUT /api/history/folders/:id/collapsed,
                                        PUT /api/history/move
  • getHistoryDir()                   — exposes the directory for code
                                        that hasn't been extracted yet
                                        (subscriptions / library / process
                                        pipeline)

The DELETE route needs to add the deleted videoId to the skip list so
subscriptions don't re-queue it. That's a cross-module concern, so
setupHistoryRoutes takes addToSkipList as a dependency. For now it's
late-bound to the still-local function in index.js (lambda captures
the scope, not the value); when subscriptions are extracted next, the
import flips cleanly.

server/index.js: 2300 → 2079 lines.

Smoke tested: /api/license-status, /api/health, and /api/history (402
license-gated for unlicensed) all respond as expected.
2026-05-08 17:09:10 -05:00
Keysat 5540b71446 Module split: license gate + Pro gates + license routes → server/license-middleware.js
• LIC                              — exported live binding (ESM)
  • setupLicenseMiddleware(app)      — registers activation gate + Pro
                                       feature gates (must run before any
                                       /api/* route)
  • setupLicenseRoutes(app)          — /api/license-status, /api/license/
                                       activate, /api/license/deactivate
  • startLicenseRefresh()            — startup + 6h periodic online check
  • refreshLicenseOnline(reason)     — ad-hoc refresh (e.g., during activate)
  • isFreeUser()                     — 'no license || no core entitlement'
  • tryAcquireFreeSlot() / releaseFreeSlot() — the free-tier concurrency
                                                 lock previously open-coded
                                                 in /api/process

Local 'const isFreeUser = ...' in /api/process renamed to 'isFree' to
avoid shadowing the imported helper. Open-coded freeJobInFlight reads/
writes replaced with the slot helpers.

server/index.js: 2461 → 2300 lines.

Smoke tested: server boots; /api/license-status, /api/health, /api/
process (rejects with 400 'No API key' as expected for unlicensed +
no key) all behave as before.
2026-05-08 17:05:35 -05:00
Keysat 7ab2a3249a Module split: extract API key + live reload to server/config.js
• initConfig({ dataDir })           — boot-time setup; mkdir's configDir,
                                        reads initial value, kicks off the
                                        3 s poll loop
  • serverApiKey                      — exported as a 'let' binding (ESM
                                        live binding) so importers see the
                                        current value
  • resolveApiKey(clientKey)          — picks per-request key (BYO vs
                                        server)
  • getEnvPath()                      — exposes /data/.env path so the
                                        cookies module can read its own
                                        legacy YT_COOKIES_FROM setting

Bug fix uncovered during this extraction: /api/health was directly
referencing ytCookiesFileExists / ytCookiesFilePath (module-scoped
vars I'd already moved to cookies.js). The route silently 500'd on the
first request after the cookies extraction. Now uses ytCookieMethod()
and getCookieFilePath() instead.

server/index.js: 2510 → 2461 lines.

Smoke tested: server boots; /api/health responds; updating
/data/config/startos-config.json flips hasServerKey from false → true
within 3 s ([config] server API key loaded log line confirms).
2026-05-08 17:01:45 -05:00
Keysat 2c655dc9ee Module split: extract cookies state + helpers + routes to server/cookies.js
• initCookies({ dataDir, envPath })  — boot-time setup; reads .env's
                                         YT_COOKIES_FROM and probes for
                                         cookies.txt
  • ytCookieArgs() / ytExtraArgs()     — yt-dlp arg builders
  • ytCookieMethod()                   — human-readable active method
  • setupCookieRoutes(app)             — registers the four /api/cookies/*
                                         routes (upload / delete / test /
                                         status)

Module owns its private state (browser-name, file-exists, file-path).
Upload and delete routes flip the file-exists flag inside the same
module, so subsequent yt-dlp calls reflect the change immediately
without callers re-reading.

server/index.js: 2614 → 2510 lines.

Smoke tested: server boots; /api/license-status and /api/health
respond. /api/cookies/status returns the existing license-gated 402
for unlicensed callers (unchanged behavior).
2026-05-08 16:57:03 -05:00
Keysat 9a82fede7a Module split: extract yt-dlp lifecycle helpers to server/ytdlp.js
• checkYtdlp()                  — version + GitHub-latest check (24h-cached)
  • autoUpdateYtdlp(dataDir)      — multi-strategy update (-U, pip, brew, binary)

Module-private memoization (ytdlpVersion, ytdlpLastCheck) now stays
inside ytdlp.js where it belongs. autoUpdateYtdlp gained an explicit
dataDir parameter — strategy 4 (StartOS binary download to /data/bin/)
needs it; passing it in keeps the module pure of caller-side state.

server/index.js: 2694 → 2614 lines.

Smoke tested: server boots; /api/license-status, /api/health respond.
No behavior change.
2026-05-08 16:54:51 -05:00
Keysat 4c3cb6a077 Module split: extract audio I/O helpers to server/audio.js
• getAudioDuration(path)         — ffprobe wrapper, returns seconds | null
  • splitAudioFile(in, dir, secs)  — ffmpeg -acodec copy chunking
  • downloadPodcastAudio(url, dst) — streams HTTP audio to disk

Also moved fetchUrl into util.js (alongside the other stateless
helpers) — it's a generic HTTP-GET-with-redirects used by RSS parsing
and channel discovery, not strictly audio.

server/index.js: 2758 → 2694 lines.

Smoke tested: server boots; /api/license-status, /api/health, /
respond. No behavior change.
2026-05-08 16:53:06 -05:00
Keysat 1c78e46ebd Module split: extract Gemini-specific helpers to server/gemini-helpers.js
• PRICING table             — per-1M-token rates by model
  • calcCost(model, usage)    — Gemini usage object → cost record
  • buildAnalysisPrompt(...)  — JSON-output topic-analysis prompt

These all share the Gemini contract — pricing schema, usage shape, and
prompt format. When we add other providers, each gets its own
provider-specific helpers file; this becomes the basis of the Gemini
provider implementation.

server/index.js: 2828 → 2758 lines.

Smoke tested: server boots; /api/license-status, /api/health, and /
(frontend) all respond. No behavior change.
2026-05-08 16:50:34 -05:00
Keysat ffc8c31130 Module split: extract pure helpers to server/util.js
First step of breaking up the 2914-line server/index.js. Pulled out the
zero-state, no-side-effects helpers:

  • sendEvent(res, event, data)         — writes one SSE frame
  • extractVideoId(url)                 — YouTube URL → 11-char id
  • formatTime(seconds)                 — seconds → 'M:SS' or 'H:MM:SS'
  • parseTimestampedTranscript(text)    — Gemini transcript text → entries[]
  • safeText(result)                    — robust .text getter for Gemini responses
  • retryGemini(fn, opts)               — 503/429/network retry with linear backoff

server/index.js: 2914 → 2828 lines.
server/util.js : new, 113 lines.

Smoke tested: server boots, /api/license-status responds. No behavior
change.
2026-05-08 16:48:40 -05:00
Keysat 3bd50f8429 Bump version to 0.2.2
Live-reload of the Gemini API key, and fix a startup-crash bug in
0.2.0 / 0.2.1 where the vendored keysat client could not resolve its
@noble/* deps inside the running container.
2026-05-08 16:38:44 -05:00
Keysat b5a066750a Live-reload Gemini API key config + fix vendor module resolution
Two related changes that ship together because the second was uncovered
while testing the first.

1. Live config reload (the ostensible feature):

   The "Set Gemini API Key" StartOS action writes to /data/config/
   startos-config.json. The server used to read that file once at
   startup (and via a separate Python read in docker_entrypoint.sh
   before that), which meant a key change required a service restart
   to take effect. Now the server polls the file every 3 s
   (RECAP_CONFIG_POLL_MS, env-overridable) and updates serverApiKey
   in place. fs.watch was tried first and dropped — it's flaky on
   macOS (FSEvents single-file quirks) and behaves inconsistently with
   atomic-rename writes the SDK file model uses. Polling is dead
   simple and a stat call every 3 s is free.

   Also dropped the Python config read from docker_entrypoint.sh; the
   server now handles it natively. Entrypoint still loads /data/.env
   for arbitrary env vars (RECAP_*, etc.).

2. Vendor module resolution (the silently-broken thing):

   The earlier vendor change (move @keysat/licensing-client from a
   git+https dep to a file: dep at vendor/) created a symlink in
   server/node_modules. That symlink to the vendor dir was getting
   resolved by Node, so the keysat client tried to import @noble/
   ed25519 from /app/vendor/keysat-licensing-client/dist/, walked up
   to /app/vendor/, then /app/, neither of which had node_modules.

   Result: v0.2.0 and v0.2.1 would crash at startup with
   ERR_MODULE_NOT_FOUND on @noble/ed25519. The Docker BUILD succeeded
   because npm install with file: deps doesn't pull transitive deps
   into the parent node_modules — but the runtime would have failed
   the moment server/license.js ran.

   Fix:
     • Dockerfile builder now `npm install`s inside vendor/keysat-
       licensing-client/ so @noble/* lands in its own node_modules,
       where Node's resolver finds it.
     • Dockerfile runner now COPYs vendor/ to the runner image
       (previously not copied — the symlink in server/node_modules
       would have pointed at nothing).
     • vendor/keysat-licensing-client/package-lock.json is committed
       so the in-Docker install is reproducible.
2026-05-08 16:38:33 -05:00
Keysat eb152cc97c Bump version to 0.2.1
Reword short description on the registry card.
2026-05-08 14:13:44 -05:00
Keysat 64a04cdd7f Reword short description to lead with the output, not the tech
Old: Download, transcribe, and summarize YouTube videos and podcasts
     with AI.
New: Turn videos and podcasts into structured topic summaries with
     clickable timestamps.

Updated in both the StartOS manifest (startos/manifest/i18n.ts) and
the registry card (startos-registry/packages/recap/package.json).
2026-05-08 14:13:44 -05:00
Keysat 8aaa405843 Vendor @keysat/licensing-client to avoid private-repo auth in Docker build
The keysat-client-ts repo is private. Previous builds were succeeding
purely because Docker layer caching reused a node_modules from when
the repo had been accessible — once anything invalidated the
server/package.json or server/package-lock.json hash (the rename did),
npm in a fresh container hit github with no credentials and 404'd.

Fix: copy the built dist/ from server/node_modules/@keysat/licensing-
client/ into vendor/keysat-licensing-client/, strip the prepare/build
scripts (we already have the compiled output), and switch the server
package.json dep to a file: path:

  "@keysat/licensing-client": "file:../vendor/keysat-licensing-client"

Dockerfile now COPY's vendor/ before npm ci. No git, no SSH, no
credentials needed in the build container — and the npm step is
pure-local so it's deterministic.

Side cleanup: dropped the apt-install-git + url.insteadOf gymnastics
that existed solely to work around the now-removed git+https resolution.
The image is slightly smaller (no git in the builder stage). Switched
the npm flag to the modern --omit=dev (the legacy --production printed
a warning).

If keysat-client-ts updates, regenerate vendor/ by:

  cp -r server/node_modules/@keysat/licensing-client/{dist,package.json,LICENSE,README.md} \
        vendor/keysat-licensing-client/
  # then strip prepare/build scripts and devDeps from the copied package.json
  # (or just hand-edit if the upstream package.json hasn't changed)
2026-05-08 13:45:12 -05:00
Keysat 2c2ccfae05 Bump version to 0.2.0
Renamed to Recap. New StartOS package id, new Keysat product slug,
new display name everywhere user-visible. StartOS will treat this as
a fresh install.
2026-05-08 13:35:47 -05:00
Keysat 9282440143 Rename project: youtube-summarizer → recap
The product was always more than YouTube — it handles podcast feeds
too, and the upcoming multi-provider work makes it less Gemini-
specific. New name: Recap.

This is a coordinated identity change across:

  • StartOS package id: youtube-summarizer → recap
    (manifest.id; the .s9pk filename, Docker image namespace, and
    install path under StartOS all derive from this automatically)
  • Display name: "YouTube Summarizer" → "Recap"
    (manifest title, activation screen heading, page <title>, console
    log on boot, i18n strings, ABOUT.md, Dockerfile header,
    docker_entrypoint banner)
  • Keysat product slug: youtube-summarizer → recap
    (server/license.js PRODUCT_SLUG; frontend fallback strings)
  • Daemon subscription id: youtube-summarizer-sub → recap-sub
  • Env var prefix: YT_SUMMARIZER_* → RECAP_*
    (LICENSE_KEY, LICENSE_KEY_PATH, MAX_OFFLINE_DAYS,
    VALIDATE_INTERVAL_MS)
  • localStorage keys: yt-summarizer-* → recap-*
    (gemini-key, activation-skipped, clips)
  • Library export filename: youtube-summarizer-library.json →
    recap-library.json
  • npm package names: youtube-summarizer-{startos,server} → recap-*
  • Deploy paths: youtube-summarizer_x86_64.s9pk → recap_x86_64.s9pk
    (default values in bin/deploy.sh; .deploy.env on dev machine
    needs the same update before next push)
  • Self-hosted registry directory: startos-registry/packages/
    youtube-summarizer → .../recap (with package.json + INSTRUCTIONS
    rewritten)

What does NOT change:
  • Filesystem repo path (still /Users/.../youtube-summarizer/)
  • Git history / commit messages
  • Existing version files in startos/versions/ (kept as-is — the
    version chain belongs to the package's own history regardless of
    its display name)

User-side follow-ups required:
  1. Create "recap" product in Keysat admin, set up Core/Pro tier
     policies (same entitlements as before), mint a fresh test
     license. Old "youtube-summarizer" licenses won't activate
     against the new slug.
  2. Update .deploy.env (gitignored) so FILEBROWSER_PATH and
     REGISTRY_PUBLIC_URL point at recap_x86_64.s9pk.

StartOS will treat this as a brand-new app on install — existing
youtube-summarizer installs will not auto-migrate (acknowledged
intentional given no real users yet).
2026-05-08 13:35:27 -05:00
Keysat 1aaa7a453a Bump version to 0.1.18
Fix Settings modal failing to open for licensed users.
2026-05-08 13:00:29 -05:00
Keysat 8a519ee25d Fix Settings modal crash: send licenseId as string, not Uint8Array
The Keysat client's payload exposes both:
  • licenseId    — raw 16-byte UUID as Uint8Array
  • licenseUuid  — same value, canonical string form

server/license.js was sending licenseId (the Uint8Array). After JSON
serialization that turned into an object like {"0":1,"1":2,...} on
the wire. The frontend's renderLicenseBlock() calls
`lic.licenseId.slice(0, 8)` to abbreviate the ID for display — .slice
doesn't exist on that object, so the template threw, the
app.innerHTML assignment silently aborted, and clicking the gear
looked like a no-op.

The render error guard added in 0.1.17 caught it on first repro:
  Render error: lic.licenseId.slice is not a function

Fix: switch to payload.licenseUuid (the string form).
2026-05-08 13:00:29 -05:00
Keysat a226113a10 Drop transcription/analysis cost lines from logs
The cost breakdowns (token counts + dollar figures) are tied to the
hardcoded Gemini PRICING table, which won't make sense once we add
OpenAI/Claude/local providers — different APIs report tokens
differently, some are free, and the pricing table can't keep up. Drop
the cost and token-count lines from the activity log so the per-step
timing is the durable signal we keep.

Removed:
  • "Total cost: <in> in / <out> out — cost: $X" (chunked transcription)
  • "Transcription tokens: <in>/<out> — cost: $X" (single-shot)
  • "Analysis tokens: <in>/<out>/<thinking> — cost: $X"
  • "Pipeline finished in Ns — total cost: $X (Y tokens)" → now just
    "Pipeline finished in Ns"

Cost calculation helpers (calcCost, PRICING) stay in the codebase for
now — they may come back via a per-provider plugin layer later. They
just no longer write to the activity-log stream.
2026-05-08 12:50:10 -05:00
Keysat f6c1a1e830 Bump version to 0.1.17
See pending notes
2026-05-08 12:26:53 -05:00
Keysat c8b3300904 UI: persistent upgrade banner; surface render errors
Two bugs reported in 0.1.16 testing:

1. Library button shows a toast "Library is a paid feature ... Tap
   upgrade to unlock" but no Upgrade button is visible anywhere on the
   page. The free-tier banner only rendered when isLicensed() was false,
   so a licensed-but-missing-entitlements user got the toast directing
   them at an invisible button.

2. Clicking the gear (Settings) icon does nothing — the modal never
   appears. The Activity Log button works on the same page.

Fix 1: replace the unlicensed-only renderFreeBanner with renderUpgrade
       Banner driven by shouldShowUpgradeBanner() = !isProTier(). Copy
       adapts by tier:
         • Free: "Free mode · one video at a time · no library, no subs"
         • Limited license (missing core entitlements): "Limited license
           — your license is missing some Core features"
         • Core tier: "Core tier — upgrade to Pro for subs, auto-queue,
           clips"
       Pro tier shows nothing.

Fix 1b: rewordtoasts on Library / Subscribe clicks to drop the
        "tap Upgrade" instruction — the persistent banner is the action
        path now, no need to direct users at a maybe-not-there button.

Fix 2 (diagnostic): wrap the main app.innerHTML build in try/catch.
       A thrown exception in any of the ${...} template substitutions
       silently aborts the innerHTML assignment, leaving the previous
       DOM in place — which looks exactly like a button doing nothing
       when it actually fired and called render(). Now an exception
       lands in a visible error-box with the message + console trace,
       so the next reproduction tells us what's actually throwing.
2026-05-08 12:26:29 -05:00
Keysat 8d9b384e32 Bump version to 0.1.16
Release notes:
  Add free mode for unlicensed users (one video at a time, no library).
  Online license revocation check via Keysat /v1/validate.
  Fix blank-screen bug after processing long videos.
  Ship deploy-script polish (no double-bump prompt).
2026-05-08 11:34:28 -05:00
Keysat 1e030a24c6 Free tier: drop spurious BYO key gate; clarify bundled vs BYO
The previous free-tier commit (c0975fe) blocked USE_SERVER_KEY for
unlicensed users on the theory that this protected a "bundled key."
That conflated two different things:

  • USE_SERVER_KEY = the user's OWN Gemini key, just stored server-side
    via the StartOS configuration action (vs. browser localStorage).
    Both paths are BYO — the user pays Google directly either way.

  • Bundled key = a future relay where paid users' /api/process requests
    are proxied through the operator's service and the operator absorbs
    the API cost. Sketched in UPGRADE-DESIGN.md (deleted, in git history)
    but NOT YET BUILT.

Blocking USE_SERVER_KEY broke a legitimate flow: a free user installs
the app on their own StartOS, sets their Gemini key via the config
action, then summarizes from the web UI without re-entering it.

This commit:
  • Drops the BYO/USE_SERVER_KEY rejection in /api/process. Free users
    can use a key from either path; the existing `if (!apiKey)` check
    still catches the no-key-anywhere case with a helpful message.
  • Reverts the frontend submit-button and handleSubmit checks to the
    same key requirement for both tiers (state.apiKey OR state.hasServerKey).
  • Drops "bundled API key" from the activation-screen subtitle and
    "bring your own Gemini key" from the free-mode banner. Until the
    relay is built, paid users still BYO too — promising otherwise in
    upgrade copy is misleading.
  • Keeps the parts that ARE legitimate free-vs-paid differentiators:
    the one-at-a-time concurrency lock and skipping saveToHistory.

Also fixes the `make deploy` redundancy:
  • bin/bump-version.sh accepts --from-deploy. When set, if there is no
    .release-notes-pending.txt (consumed by a prior bump or never
    written), exit 0 without prompting — the current version is already
    fresh.
  • Makefile passes --from-deploy from the deploy target. Standalone
    `make bump` is unchanged (always prompts).

Result: `make bump` then `make deploy` no longer double-prompts. And
calling `make deploy` twice in a row (no new work) is idempotent on
the bump step.
2026-05-08 11:32:30 -05:00
Keysat 25b1c3a366 Add free tier (unlicensed users get one-at-a-time summarization)
Unlicensed users can now summarize a single video at a time using their
own Gemini API key. The result renders in the UI exactly like a paid
summary, but is not persisted — there's no library entry, no history,
and a second submission while one is in flight is rejected.

Server (server/index.js):
  • /api/process is now in LICENSE_OPEN_PATHS. The route handler
    distinguishes free users (state !== "licensed" || no "core") and:
      - rejects USE_SERVER_KEY / empty key with 402 byo_key_required
        (so the bundled Gemini key stays paid-only)
      - rejects a second concurrent job with 409 processing_in_progress
        via a module-level freeJobInFlight flag, released in finally
      - skips saveToHistory so the host's library stays clean
  • Pro feature gates (history/library/subscriptions) unchanged — still
    return 402 feature_not_in_tier for unlicensed callers.

Frontend (public/index.html):
  • New state.activationSkipped flag (persisted to localStorage). The
    activation screen still appears on first launch, but now offers a
    "Skip — use free mode" button alongside Activate / Buy a key.
    Once skipped, the main app renders normally.
  • Free-mode upgrade banner under the top bar with Upgrade and "I have
    a key" buttons (the latter routes back to the activation screen).
  • handleLibraryClick / handleSubscribeClick wrappers — for unlicensed
    users, the library (clock) icon and the channel-URL Subscribe
    submission show a toast explaining the upgrade rather than opening
    an empty sidebar / hitting a 402.
  • Submit button enforces BYO key for unlicensed users (the bundled
    state.hasServerKey doesn't enable submit). handleSubmit shows a
    toast when an unlicensed user tries to queue a second video.
2026-05-08 11:16:02 -05:00
Keysat 7d71150439 Fix SSE event type lost across reader chunks (blank screen post-process)
Symptom: after a long video finishes processing, the screen goes blank
even though the video and chunks save correctly to history (visible
after a refresh + library click).

Cause: the manual SSE parser in processUrl declared `let eventType = ""`
inside the `while (await reader.read())` loop, so the variable reset on
every chunk. The server emits each event as a single write of
`event: X\ndata: Y\n\n`, but for a long video the result payload (entries
+ chunks + logs) is tens of KB and gets split across reader chunks by
the browser. When the split landed between the `event: result\n` line
and the `data: ...` line, the event type was lost and `handleSSE("",
data)` matched no branch — the result event was silently dropped, and
state.chunks stayed at the empty array set during processUrl reset.

Fix: hoist eventType outside the loop so it persists across chunks, and
reset it after each dispatch (per the SSE spec, event type returns to
default after an event is fired).

Short videos with small result payloads fit in a single chunk and so
were unaffected — which is why the bug looked intermittent.
2026-05-08 11:04:50 -05:00
Keysat 2621f2cdbe Add online license revocation check (Keysat /v1/validate)
Without this, a license revoked in the Keysat admin UI keeps unlocking
the app on the customer's machine — Ed25519 signatures are perpetually
valid, so the offline-only check never sees the revocation.

What this adds:

  • license.js: validateOnline() calls licensing.keysat.xyz/v1/validate
    via @keysat/licensing-client's Client. Hard rejections (revoked,
    suspended, expired, not_found, product_mismatch, fingerprint_mismatch,
    too_many_machines, invalid_state) immediately flip state to "invalid"
    and persist the verdict to <license>.state.json so it survives
    restarts. rate_limited and unknown reasons are treated as transient.

  • Network errors keep the prior state for up to MAX_OFFLINE_DAYS
    (default 7, env-overridable) since the last successful validate.
    Past the ceiling, lock out with reason=validation_overdue. This
    avoids breaking customers when Keysat is briefly down while still
    catching revocations on machines that go offline forever.

  • license.js: deactivate() helper that removes both license.txt and
    its sidecar state file (idempotent). publicView() now exposes
    lastValidatedAt, serverStatus, graceUntil for the UI.

  • index.js: refreshLicenseOnline() runs on startup (async, non-
    blocking), every 6h thereafter (env-overridable), and at activation
    time with an 8s timeout cap so a slow Keysat doesn't hang the
    activation UI. State changes are logged.

  • index.js: /api/license/activate now awaits an online confirmation
    after the offline signature check passes. A revoked key pasted into
    the activation modal fails fast instead of working until the next
    poll.
2026-05-08 10:39:11 -05:00
Keysat 154d692371 Clean up legacy 0.3.5 scaffolding and standalone-mode artifacts
- Removes start9/0.4/ — the StartOS 0.3.5-style YAML manifest folder,
  superseded by the TypeScript-based startos/ package.
- Removes pre-StartOS standalone-mode artifacts: setup.sh, create-app.sh,
  Start Summarizer.command, build-guide-pdf.py, GET-STARTED.md/.pdf.
- Removes conversion-era design docs (START9_PACKAGING_GAMEPLAN.md,
  UPGRADE-DESIGN.md, KEYSAT_INTEGRATION.md). Recoverable from history if
  needed.
- Tightens .gitignore: untracks node_modules, javascript/ build output,
  *.s9pk, history/ user data, cookies.txt, library export, .env, .DS_Store,
  and .claude/ worktree state. Files remain on disk; just removed from
  the index.
2026-05-08 09:41:42 -05:00
Keysat 574a16d9fa Save in-progress keysat integration and StartOS 0.4 work
Snapshot of the working tree before cleanup. Captures:
- Keysat licensing: server/license.js, /api/license/* endpoints in
  server/index.js, activation modal in public/index.html, embedded
  Ed25519 issuer key (assets/issuer.pub).
- StartOS 0.4 expansion: setApiKey action, version files v0.1.1
  through v0.1.15, file-models/config.json.ts, manifest updates.
- Self-hosted registry server (startos-registry/).
- Build/deploy scripts (bin/bump-version.sh, bin/deploy.sh, vendored
  yt-dlp binary), .gitignore, .deploy.env.example.
- Recent design docs (KEYSAT_INTEGRATION.md, UPGRADE-DESIGN.md) —
  retained here so they remain recoverable when removed in the
  follow-up cleanup commit.
2026-05-08 09:39:17 -05:00
MacPro 8298c083c7 Fix StartOS 0.4 TypeScript packaging to match SDK API 2026-04-09 15:10:44 -05:00
MacPro 68ec875ee7 Add StartOS 0.4.0 packaging 2026-04-09 15:03:31 -05:00
258 changed files with 32990 additions and 1318 deletions
+1
View File
@@ -0,0 +1 @@
../../docs/guides/relay-client.md
+13 -2
View File
@@ -19,6 +19,9 @@ image.tar
# Runtime / user data — must never be committed
history/
# Per-install identity (UUID) the server mints on first boot. On the box it
# lives at /data/install-id; in local dev the server writes it to the repo root.
/install-id
cookies.txt
*.txt.bak
youtube-summarizer-library-export*.json
@@ -26,6 +29,14 @@ ytdlp-cache/
# Local dev secrets
.env
.env.*
!.env.example
# Claude Code state (worktrees, plans, etc.)
.claude/
# Claude Code — deny by default (worktrees, plans, local settings stay out),
# allow-list shared wiring (see standards/portability.md).
.claude/*
!.claude/rules/
!.claude/agents/
!.claude/commands/
!.claude/skills/
!.claude/settings.json
+154
View File
@@ -0,0 +1,154 @@
# AGENTS.md — Recaps
YouTube + podcast summarizer + library, served as a single-page app from a Node.js backend. Ships as a StartOS `.s9pk` (single-mode self-host) and as the public `recaps.cc` cloud (multi-mode tenants).
> **Inbox check:** At session start, if `~/Projects/standards/INBOX.md` exists, scan it for
> items tagged `(recap)` and surface them before proposing next steps; triage with `/triage`.
> **Design:** before building or changing any user-facing UI, read `design/DESIGN.md` and
> `design/tokens.tokens.json` and conform to them. Accent is indigo `#818cf8`; purple
> `#a855f7` is premium-only. The same tokens govern three surfaces that must stay in sync:
> the main `public/index.html` stylesheet, its `SHARE_PAGE_CSS` share-export string, and
> `public/auth.html`.
## Stack
- **Server**: Node.js (`type: module`, ES modules). The dev box currently runs `v25.6.1`; container runtime is whatever the `Dockerfile` pins — check before assuming.
- **Frontend**: One file, `public/index.html`, with vanilla JS embedded (no framework, no bundler). Render is a render-string-into-`innerHTML` loop driven by a module-scoped `state` object.
- **DB**: SQLite via `better-sqlite3`. Multi-mode only; single-mode keeps everything on the filesystem.
- **Packaging**: `@start9labs/start-sdk` under `startos/` — version graph at `startos/versions/index.ts`.
- **Deps of note**: `@anthropic-ai/sdk`, `@google/genai`, `openai`, `nodemailer`, `express`, `@keysat/licensing-client` (vendored at `vendor/keysat-licensing-client`).
## Commands
Run from repo root unless noted.
| Action | Command |
|---|---|
| Dev server (single-mode default) | `cd server && npm run dev` |
| Prod server | `cd server && npm start` |
| Run all tests | `cd server && npm test` |
| Run one test file | `cd server && node --test --test-reporter=spec test/<file>.test.js` |
| Run one test by name | `cd server && node --test --test-reporter=spec --test-name-pattern='<substring>' test/<file>.test.js` |
| Build `.s9pk` (x86) | `make x86` |
| Bump version (interactive) | `make bump` |
| Install to local StartOS | `make install` *(see Always/Never — bump first; the binary is `start-cli` under the hood)* |
| Type-check (StartOS TS) | `npm run check` *(repo root; runs `tsc --noEmit` over `startos/**/*.ts`. The `server/` is plain JS and is not type-checked.)* |
| Format (StartOS TS) | `npm run prettier` *(repo root; `prettier --write startos`. There is **no** ESLint/linter — `server/` JS is untooled. Many `startos/versions/*.ts` are currently unformatted.)* |
Mode is selected at boot via the `RECAP_MODE` env var: `single` (default) or `multi`. Other runtime env var of note: `RECAP_TRUSTED_PROXY_HOPS` (default `1`) — how many trusted reverse proxies sit in front of the app, so the anonymous-trial per-IP cap reads the real client IP from `X-Forwarded-For` (set `0` if the app is directly internet-facing, `2`+ behind a CDN/LB; setting it too high re-opens the trial-cap bypass).
## Directory layout
```
server/
index.js main HTTP + SSE entry; mounts every route
providers/ relay.js, gemini.js, openai.js, anthropic.js, ollama.js,
openai-compatible.js, whisper.js — each implements the
provider interface in providers/index.js
anon-trial.js multi-mode trial-cookie minting + IP cap
tenant-credits.js multi-mode signed-in-tenant credit pool
history.js per-scope library save/load + REST handlers
config.js StartOS config snapshot + server-side API-key resolver
db.js SQLite schema apply + getDb() handle (multi-mode only)
billing-routes.js multi-mode self-serve purchase: /api/billing/{plans,buy,status};
Bitcoin (BTCPay inline Lightning) + card (Zaprite) rails
subscription-reminders.js daily expiry-reminder scan → sendMail (multi-mode)
smtp.js StartOS System-SMTP transport (magic links + reminders)
test/ node --test files
public/
index.html the whole single-page app, ~10k lines vanilla JS
auth.html standalone magic-link landing page (multi-mode)
startos/
manifest/ StartOS package manifest
versions/<vN>.ts one file per shipped version + index.ts version graph
actions/ operator-facing StartOS Actions
docs/ design notes; treat as in-progress, not authoritative
bin/bump-version.sh used by `make bump` and `make deploy`
vendor/keysat-licensing-client/ local-link Keysat SDK
```
## Conventions
- **Plain language over jargon**, especially for git / packaging / dev-tooling steps.
- **Don't be sycophantic.** Push back when something doesn't add up.
- **Honest reports.** A failing test/build is a failure, even if pre-existing or unrelated. Don't fold it into a "success" summary.
- **Diff size matches change scope.** Small reviewable diffs, not sweeping rewrites.
- **Comments explain WHY, not what.** No narrating self-evident code. No referencing tasks/PRs/callers in source — that rots.
- **Match the file's own style** over any default of your own. The frontend's vanilla-JS shape is intentional; don't reach for a framework.
- **Write the test alongside the change** when the area already has tests (`server/test/*.test.js`). The repo uses the built-in `node --test`.
- **Plans persist in `docs/`** when scoped + named (e.g., `docs/per-tenant-subscriptions-plan.md`); ephemeral planning lives in conversation/tasks, not Markdown files.
### Conventions for this codebase specifically
- **Relay is the modern default provider.** The legacy "must have a Gemini API key configured" gate is dead — server-side callers should pick `relay` when configured, fall back to `gemini` only if a local key exists, otherwise surface a clear user-facing error. The frontend stores the choice in localStorage; the server can't read it.
- **Sanitize operator-internal language at error boundaries.** Strings like "Spark Control", "parakeet", "vLLM", LAN IPs, `*.local` URLs come from the sibling relay and must not reach cloud users.
- **Multi-mode credit gates fire BEFORE the pipeline.** See `/api/process` for the order — admin → license → free tenant → trial → anonymous-mint. Don't reorder without reading the comment block.
- **Trial IP cap is per-IP for IPv4, per-/64 prefix for IPv6.** Dual-stack home networks would otherwise bypass it via privacy-extension address rotation.
- **Client IP comes from `req.ip`, never a raw `X-Forwarded-For` entry.** Express `trust proxy` is set in `index.js` from `RECAP_TRUSTED_PROXY_HOPS` (default 1); `getClientIp` (`anon-trial.js`) returns `req.ip`. Trusting raw `XFF[0]` let clients spoof the trial-cap IP — don't reintroduce it.
- **`safeFilename()` is exported from `history.js`** — import and use it for any user-content → on-disk path; don't roll your own. It validates against `/^[A-Za-z0-9_-]+$/` and throws on traversal/separators (the library-import file-write hole was a missing call).
- **The relay owns cloud Pro/Max tier + expiry** (core-decoupling; `docs/core-decoupling-plan.md`). In multi-mode, paid status is `users.tier` — cached from the relay, keyed by the Recaps user-id — NOT a per-user Keysat license. Don't gate cloud paid features by `keysat_license`; the license only matters for self-hosted "take it home" portability. Cloud requests carry `X-Recap-User-Id` + the operator key; server-to-server tier reads/writes go through `providers/relay.js`.
- **Server-side background relay calls that the operator should eat go through `resolveProviderOpts("relay", { req: null })`** → the operator install identity, the *same* relay credit pool free signed-in users' summaries already draw from (`providers/index.js` `pickRelayIdentity`). That's the "operator-absorbed" lane — no comped system user-id, no operator action. To bill a specific user instead, pass their cloud identity (`{cloud:true, userId, operatorKey}`). The Daily Digest synthesis (`daily-digest.js`) is the reference use.
- **Self-serve purchase has two rails, both prepaid (no auto-renew yet).** Bitcoin = a BTCPay invoice rendered as an INLINE Lightning QR on-screen — the relay returns the BOLT11 server-to-server, so the buyer never loads BTCPay (replicate the buy-credits inline flow; do NOT redirect to a hosted checkout). Card = a Zaprite one-time hosted order (Zaprite's API has no recurring — see ROADMAP). Both settle webhooks land on the relay (it owns subscription expiry); the frontend just polls `/api/billing/status`. Expiry reminders go out via the existing System-SMTP transport (`smtp.js`): the relay enumerates who's expiring (`GET /relay/expiring-subscriptions`), Recaps maps user-id → email and sends.
- **Tier credit allotments are operator-config-driven, never hardcoded.** The cards' "N relay credits each period" comes from the relay's tier-quota config (`credits_per_period` on `/relay/tier-plans`); `null` → "Unlimited". Don't bake a number or "Unlimited" into the UI.
- **`recaps.cc` IS the operator's StartOS box**, served via Start9 Pages + StartTunnel. So `make install` (after a version bump) updates the public cloud site automatically — there is no separate cloud deploy step. A frontend-only change reaches recaps.cc as soon as the box serves the new `public/` files.
- **`render()` rebuilds the whole view via `innerHTML` — preserve live media + scroll across it.** It re-attaches the live podcast `<audio>` node (`replaceWith`, src-matched) and restores `.chunks-scroll` scrollTop, so a background re-render (e.g. the ~60s relay-credit poll) doesn't stop playback or bounce the reader to the top. YouTube minimize toggles the `.results-left.minimized` CSS class **in place** — never `render()`, because creating the YT iframe inside a `display:none` container wedges the IFrame API (black frame, needs reload); `ensureYtMounted()` + a `!state.videoMinimized` guard on `needsMount` keep the player from ever being built hidden, and `initPodcastPlayer()` is idempotent (`dataset.inited`). Don't reintroduce a full `render()` on minimize or drop these preservation steps.
- **URL parsing is duplicated server + client — edit both in lockstep.** `extractVideoId` (and the sibling classifiers `isChannelUrl` / `isPodcastUrl`) live in BOTH `server/util.js` and `public/index.html` with the same logic; the server copy is the one that actually rejects a submit (`server/index.js` → "Invalid YouTube URL"). When you teach one a new URL form, teach the other or the frontend preview and server will disagree. `extractVideoId` accepts `/watch?v=`, `youtu.be/`, `/embed/`, `/v/`, `/live/`, `/shorts/`, and a bare 11-char id; trailing `?si=…` tracking params are fine (the regex stops after the id). Tests: `server/test/util.test.js`.
- **Colors flow from CSS custom properties — edit the token, not the value.** `public/index.html`'s `<style>` opens with a `:root` token block (the single source of truth, mirroring `design/tokens.tokens.json`); the stylesheet **and the inline `style=` attributes** (var-ified in Phase 2) reference `var(--accent)` / `var(--surface)` / etc. Change a brand color there, once. **Two surfaces use `:root`+`var()` and must stay in sync:** the main stylesheet and `public/auth.html` (a standalone doc with its own `:root` copy). **The `SHARE_PAGE_*` share export is the exception** — a standalone document with **no `:root`**, so it's **pure literal hex**; a token change must be hand-mirrored into its literal values, and you must NOT introduce `var()` there (it won't resolve). **Left literal on purpose even in the main doc (don't blind-sweep):** `#fff`, hexes with no token, hex in JS *logic* (quoted ternary branches, `const c=…`), SVG `fill`/`stroke`, and `<meta theme-color>`. Phase 2 var-ified everything else by scoping to **CSS-value position** (hex preceded by `:`/space/`,`, never a quote) — reuse that rule for any future sweep.
### Client-side contract with the relay
The full client-side relay contract — env vars, the `/relay/*` endpoint list, `X-Recap-*` header directions, and the file map — lives in **`docs/guides/relay-client.md`**. Read it before editing `server/providers/relay.js`, `relay-capabilities.js`, `relay-default.js`, `billing-routes.js`, `credits-purchase.js`, `subscription-reminders.js`, or the relay env-var resolution in `config.js`. Canonical endpoint shapes are in `../recap-relay/AGENTS.md`.
### Cross-repo changes (sibling: `../recap-relay`)
This repo and the relay (`../recap-relay`) share a live client/server contract — the
`/relay/*` endpoints, the `X-Recap-*` headers, request/response shapes, and tier/credit
semantics. **Before finishing any change that touches that boundary, check whether
`../recap-relay` needs a matching change.** If you add/rename/remove a relay call, alter a
payload shape or header, or shift tier/credit/billing behavior, update the relay side too —
and reflect it in BOTH repos' `AGENTS.md` (the contract docs) and `ROADMAP.md` (if it's
staged work). Purely local changes (UI, library handling, packaging) don't need this. When
unsure whether a change is contract-affecting, assume it is and check.
## Always
- **Bump the version before `make install`.** StartOS dedupes sideloads by version string — installing the same version twice silently no-ops. Use `make bump` or edit `startos/versions/index.ts` + add a `vN.ts` file. Applies to EVERY iteration, even a one-line edit.
- **Add new version files to BOTH the import block AND the `other:` list** in `startos/versions/index.ts`, and update `current:` to the new version constant.
- **Ask before `make deploy` / `make redeploy`.** These push to the Start9 community registry — public-facing, attribution-tracked, irreversible from your laptop. `make install` is the safe iteration loop.
- **Verify mDNS resolution before blaming it** when `make install` fails. Substitute the operator's actual StartOS hostname (the `host:` field in `~/.startos/config.yaml`) and run `curl -sk "https://${STARTOS_HOST}/rpc/v1" -X POST -d '{}' -H 'Content-Type: application/json'` — if that reaches the box but `start-cli` doesn't on the same target, it's almost certainly **macOS Local Network privacy** blocking the third-party `start-cli` binary (Apple's `curl`/`ping` are exempt, so the box looks reachable). Tell: `node -e` TCP-connect to `<box-ip>:443` also gives `EHOSTUNREACH` while `curl` gets 200. Fix: System Settings → Privacy & Security → Local Network → enable the Claude app (restart Claude Code if it doesn't take). This often flips off after a Claude Code update. Details: memory `feedback_macos_local_network_install`.
- **Reference env-var names, never values.** Secrets live in `.env` / `.deploy.env` (both gitignored). Examples for new vars belong in `.deploy.env.example`.
## Never
- **No "Co-Authored-By"** trailers on commits, no mention of "Claude" in source files, comments, or commit messages. Commits are authored by the user.
- **Never claim `make install` succeeded without verifying it.** Confirm the `make` exit code is 0 AND the new version actually shows on the box (`start-cli package list`) — not just that the command ran (a `tail`/pipe can mask a non-zero exit). Installs DO work from this agent's shell now; the old "`start-cli` is blocked by the sandbox" framing was a misdiagnosis — it was macOS Local Network privacy, which is fixable (see the Always "verify mDNS" rule).
- **Never `make deploy` to the registry** without explicit per-action approval, even if a prior session ran one.
- **Never edit `startos/versions/<v>.ts` for a version that's already been built and is being tested.** Add a new version file instead — operators may already have the prior `.s9pk` cached.
- **Don't add the relay's `internal-meetings` feature here.** That lives in the sibling `../recap-relay` repo. This repo is the client/library; the relay does diarization + clustering + meeting analysis.
- **Don't push to GitHub by default.** The configured remote is self-hosted Gitea unless the user says otherwise.
- **Don't pull `cookies.txt` into commits** — it's an operational yt-dlp artifact, not source.
- **Never modify `~/.startos/config.yaml`** without authorization (contains host credentials).
## Adjacent repo
- `../recap-relay` — the operator-side credit-metered service this client talks to. Owns Gemini/Parakeet/Sortformer routing, diarization, internal-meeting analysis, and the operator dashboard. See `../recap-relay/AGENTS.md` for its endpoint shapes, build/deploy rules, and roadmap. Reference it but do not change it from inside this repo.
## Current state
**Live on the operator's StartOS box** — app **0.2.161** + relay **0.2.126**. Tests: `cd server && npm test` → **144 pass**.
**Done & live:** self-serve Pro/Max purchase (Bitcoin inline-Lightning + Zaprite card, prepaid, relay owns tier/expiry), core-decoupling, per-tenant subscriptions, expiry-reminder emails (`POST /api/admin/reminders/run {test_email}`), **opt-in Daily Digest** (0.2.158, `b4fa5d7`): off-by-default daily email of a user's last ~24h of library recaps, each synthesized via `/relay/analyze` (operator-absorbed); `daily-digest.js` scan at `SEND_HOUR=8`, per-user watermark dedup, public tokenized unsubscribe, admin trigger `POST /api/admin/digest/run`; **YouTube `/live/` + `/shorts/` URL support** (0.2.159, `cb961cd`): `extractVideoId` now accepts those forms (was rejecting them as "Invalid YouTube URL"); and **self-contained shareable HTML export** (0.2.160, `621af7c`; first installed in 0.2.161): the Export menu offers a standalone `.html` with the embedded video + expandable timestamped summaries baked in, no account needed (native share sheet on mobile, download on desktop). Plans in `docs/*-plan.md`.
**Design system — DONE & live (0.2.161).** The `design/` contract + both conformance-cleanup phases are installed: Phase 1 (canonical `:root` token block; stylesheet + `auth.html` on `var()`; drift fixed) and Phase 2 (var-ified the inline `style=` hexes — 346 + 7 `#475569` — and snapped off-scale fonts/radii to the scale). Verified: 144 tests, both pages serve 200, every `var()` resolves, no off-scale residue. The var-ify scoping rule + the `SHARE_PAGE_*` literal-hex exception now live in **Conventions**; only the Style-Dictionary `palette.css` stretch goal remains (`ROADMAP.md`).
**Only loose end:** the Daily Digest's relay-synthesis + SMTP path can't be exercised off-box, so it's installed but **not yet smoke-tested** — that's operator action #5 below. Everything else (schema/upgrade, scheduler boot, unsubscribe flow) is verified.
**Pending operator actions:**
1. **Verify the mobile can't-scroll-to-top fix on the iPad** — UNVERIFIED in 0.2.157 (iOS-layout-specific, not reproducible off-device); send a screen recording if it persists. Inbox item kept open + annotated.
2. (optional) Rotate the still-live Gemini key in AI Studio, then `rm /Users/macpro/Projects/recap-keyleak-purge-backup.bundle`.
3. Real-world cloud tests: first Bitcoin purchase; enable Zaprite cards (relay "Set Zaprite Connection" + webhook); eyeball a reminder email.
4. If recaps.cc ever gains a CDN/LB hop, set `RECAP_TRUSTED_PROXY_HOPS` or the trial-cap bypass reopens.
5. **Smoke-test the Daily Digest (0.2.158):** (a) `POST /api/admin/digest/run {test_email}` to eyeball the sample render; (b) toggle it on in Settings, add a recap, then `POST /api/admin/digest/run` (no body) to force a real scan — confirms relay synthesis + SMTP send + the unsubscribe link end-to-end. Needs System-SMTP configured.
**Backlog** in `ROADMAP.md`: eval **P2** known-debt (SSE error-string scrub, credit-debit TOCTOU, multi-tenant gemini-key bypass, `GET /api/history` perf, dependency CVEs, integration tests, doc drift) + **P3** cleanup, and standing decisions (Zaprite recurring, "take Recaps home" broken for relay-tier users, cloud paid-only, no CI lint/type-check).
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+18 -4
View File
@@ -26,6 +26,18 @@ WORKDIR /app/server
COPY server/package.json server/package-lock.json* ./
RUN npm ci --omit=dev --ignore-scripts 2>/dev/null || npm install --omit=dev --ignore-scripts
# better-sqlite3 is a native (C++) module — `--ignore-scripts` above
# skips the postinstall hook that fetches its prebuilt binary for our
# platform. Rebuild it explicitly so prebuild-install runs. python3 +
# make + g++ are the fallback toolchain if no prebuilt matches (e.g.
# on uncommon arches); on linux-x64/arm64 the prebuild downloads in
# seconds and the compiler is never invoked. This stage is discarded
# from the final image, so the install footprint doesn't matter.
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 make g++ \
&& npm rebuild better-sqlite3 \
&& rm -rf /var/lib/apt/lists/*
# ── Stage 2: Final runtime image ───────────────────────────
FROM node:20-slim AS runner
@@ -53,11 +65,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
COPY --from=builder /app/vendor ./vendor/
COPY --from=builder /app/server/node_modules ./server/node_modules/
COPY server/package.json ./server/
# Top-level *.js files only — picks up index.js, license.js, util.js,
# gemini-helpers.js, audio.js, ytdlp.js, cookies.js, config.js,
# license-middleware.js, history.js, library.js, and any future
# extractions automatically. (Glob skips subdirs like server/test/.)
# Top-level *.js files (index.js, license.js, util.js, gemini-helpers.js,
# audio.js, ytdlp.js, cookies.js, config.js, license-middleware.js,
# history.js, library.js, admin-auth.js, …) PLUS the providers/
# subdirectory (multi-provider AI adapters). Anything new added in
# `server/<subdir>/` needs its own COPY line — the glob does not recurse.
COPY server/*.js ./server/
COPY server/providers/ ./server/providers/
COPY public/ ./public/
COPY assets/ ./assets/
+102
View File
@@ -0,0 +1,102 @@
# Evaluation — recap (Recaps) — 2026-06-13
Intent: Recaps is a YouTube + podcast summarizer and personal library, served as a single-page vanilla-JS app from a Node.js (ES module) backend — shipping both as a single-mode StartOS `.s9pk` self-host package and as the multi-tenant `recaps.cc` cloud (SQLite, credit-metered, talking to a sibling "relay" service).
Agents run: evaluator, security-auditor, exerciser, doc-auditor, start9-spec-checker. Reviewer **skipped** (working tree clean — no diff to review). All five completed; none failed.
## Verdict
The pure-logic core is genuinely well-built — 103 passing tests, parameterized SQL everywhere, scrypt + `timingSafeEqual` auth, hashed single-use magic-link tokens, server-derived tenant identity that resists `X-Recap-User-Id` spoofing, and a correctly server-to-server billing/settlement path. But three independent agents surfaced **three P0 issues** that the test suite can't see because the riskiest files are untested: an **arbitrary file write** via `/api/library/import` (reproduced live — `../../` in a session key escapes the scope dir), an **SSRF-with-read-back** in the podcast download path (reachable by an anonymous trial), and a **live, still-active Gemini API key committed to git history**. On top of that the multi-tenant claim is undercut by a process-global lock that serializes every concurrent cloud summarize to one-at-a-time. None of these is hard to fix, but the cloud is **not ready for untrusted users** until the three P0s are closed and the key is rotated. Self-host single-mode is in much better shape (the file-write and SSRF matter far less with one trusted operator), but a StartOS **community-registry submission is BLOCKED** on packaging gaps regardless.
## Cross-referenced findings (corroboration / merges)
- **SSE error-boundary leak — TWO agents, ONE finding.** The evaluator (`index.js:3432`, `:3003`, `:4246`) and the security-auditor (`relay.js:135-144``index.js:~4246`) independently flagged that relay backend errors are forwarded to cloud users verbatim, violating the AGENTS.md "scrub operator-internal language" contract. The evaluator's surprise sharpens it: the pipeline's own regex at `index.js:3419` detects "Parakeet/Gemma/CUDA" in the error string and then forwards that exact string two lines later. **One P2, two evidence kinds.**
- **The file-write P0 and a doc finding share a root cause.** The exerciser reproduced arbitrary file write in `library.js` because the session key is used as a filename without `safeFilename()`. The doc-auditor independently found that `safeFilename()` is *module-private in `history.js`*, not an exported shared util — which is *why* `library.js` couldn't reuse it. AGENTS.md tells contributors "`safeFilename()` already exists — use it," but it isn't importable. Fix the code and the doc together.
- **The concurrency lock cuts both ways.** The evaluator's P1 (global `currentFreeJob` lock serializes all tenants) and the security-auditor's P2 (concurrent credit over-spend *when the lock is absent*, i.e. on licensed installs) are two faces of the same `isFreeUser()`/`currentFreeJob` mechanism: when the lock applies it breaks multi-tenant concurrency; when it doesn't, it lets a 1-credit user fan out N parallel requests before any debit lands.
- **"Riskiest files are untested" — evaluator + exerciser agree.** Both note zero coverage on `/api/process` gating, `relay.js`, `tenant-auth.js`, and billing. Every P0/P1 code bug above lives in an untested file. The exerciser further notes the real summarize→save→debit happy path could not be run end-to-end (no Gemini key / relay credits) — a shared blind spot, below.
- **Secrets — evaluator + security-auditor.** Both flag `cookies.txt` as sensitive plaintext in the repo root; the security-auditor escalates with the far more serious committed-and-active Gemini key in history `d5046a0:.env`. Working tree itself is clean.
- **Version-file sprawl — evaluator + spec-checker.** Both independently noted 175 StartOS version files with only ~3 carrying real migrations.
## Priority queue
**P0 — block the cloud; fix before any untrusted use**
- [P0] Arbitrary file write via `/api/library/import``../../` in a session key escapes scope; reproduced writing `/tmp/rce_test.json` & `recap/evil.json``server/library.js:131-139` (key used verbatim, no `safeFilename()`) — **exerciser** *(confirmed in single-mode; confirm multi-mode tenant reachability — it's per-scope so any authed tenant should reach it)*
- [P0] SSRF with read-back via podcast URL — anon trial can POST `{type:"podcast", url:"http://169.254.169.254/..."}`; unguarded `http.get` follows redirects to any host and the body is transcribed back to the attacker — `server/audio.js:78-97` (`downloadPodcastAudio`), gate `server/index.js:2747`, reached `:3455`**security-auditor**
- [P0] Live Gemini API key committed to git history and still the active key — `git show d5046a0:.env`, pushed to `origin/master`**security-auditor** *(rotate now; purge from history)*
**P1 — correctness / availability / submission blockers**
- [P1] `require("crypto")` inside an ESM module throws `ReferenceError` on the anon license-purchase settle path — `server/license-purchase.js:423` (called from `:353-354`) — **evaluator**
- [P1] Global single-flight `currentFreeJob` lock serializes the *entire* multi-tenant cloud (a 2nd concurrent `/api/process` from any user gets 409); `isFreeUser()` returns true for every tenant in multi-mode — `server/index.js:2621`, `server/license-middleware.js:143`**evaluator**
- [P1] Trial IP-cap + magic-link rate-limit bypass via spoofed `X-Forwarded-For` (no `trust proxy` / XFF normalization) — unlimited free trial credits if the edge proxy appends rather than replaces XFF — `server/anon-trial.js:50-57`, `server/auth-routes.js:209`**security-auditor** *(deployment-dependent on the recaps.cc proxy)*
- [P1] StartOS community-registry submission **BLOCKED** (3 blockers): missing root `instructions.md`; `packageRepo`/`upstreamRepo` point to `https://ten31.xyz` (a homepage, not a source repo); `license: 'Proprietary'` fails the "source available" gate — `startos/manifest/index.ts`, repo root — **start9-spec-checker** *(only blocks registry submission; does NOT affect `make install`)*
**P2 — real risk / quality, not release-blocking for self-host**
- [P2] Operator-internal language (Parakeet/Gemma/CUDA/LAN IPs/`*.local`) leaks to cloud users at the SSE error boundary; no scrub exists — `server/index.js:3432,3003,4246` + `server/providers/relay.js:135-144`**evaluator + security-auditor**
- [P2] Concurrent credit over-spend (TOCTOU) on licensed installs — N parallel requests pass the `total>0` check before any blind `debitOne` lands — gate `server/index.js:2497-2550` vs debit `:3158,:4197` (`anon-trial.js:299`) — **security-auditor**
- [P2] Multi-mode tenant can spend the operator's *server* Gemini key directly (bypasses relay metering) by selecting `transcriptionProvider:"gemini"` with an empty key — `server/providers/index.js:104-114`**security-auditor**
- [P2] `GET /api/history` reads + `JSON.parse`s every full session file (transcript+summary, MB each) just to list ~8 metadata fields — `server/history.js:418-437`**evaluator**
- [P2] Dependency CVEs: nodemailer 6.10.1 (high — SMTP injection/DoS, low practical reach here), ws 8.20.0, qs/express, protobufjs 7.5.6 (moderate) — `npm audit` in `server/`**security-auditor**
- [P2] Unsanitized IDs persisted into `_meta.json` via array-form import and `PUT /api/history/move` (corrupts the listing; no direct file-path escape) — `server/library.js`, history move handler — **exerciser**
- [P2] `PUT /api/history/meta` accepts arbitrary JSON shapes with no schema (a wrong-typed body, e.g. `{"folders":"not-an-array"}`, can break the listing) — **exerciser**
- [P2] `server/index.js` is 4351 lines mixing routing, the full `/api/process` pipeline, yt-dlp orchestration, and SSE (`:2463-4270`) — **evaluator**
- [P2] Zero tests on the highest-risk files (`/api/process` gating, `relay.js`, `tenant-auth.js`, billing) — every code P0/P1 above lives here — **evaluator + exerciser**
**P2/P3 — documentation drift (doc-auditor)**
- [P2] Documented credit-gate order in AGENTS.md omits the "paid cloud user" bypass state (between license and free-tenant) — `AGENTS.md:77` vs `server/index.js:2464-2472`**doc-auditor**
- [P2] Operator-facing copy is stale Gemini-first / Gemini-only; relay is the default provider — `startos-registry/packages/recap/INSTRUCTIONS.md:5,23`, `assets/ABOUT.md:3,18`**doc-auditor**
- [P3] AGENTS.md directory layout omits ~25 server modules (audio, library, credits-purchase, tts-routes, relay-default/-capabilities/-state, url-resolver, ytdlp, …) — `AGENTS.md:34-60`**doc-auditor**
- [P3] `relay-client.md` Authorization header missing the `Bearer` scheme — says `Authorization: <license>`, code sends `Authorization: Bearer <license_key>``docs/guides/relay-client.md:17` (== `.claude/rules/relay-client.md:17`) — **doc-auditor**
- [P3] `index.html` line count stated `~10k`, actually 12,515 — `AGENTS.md:51`**doc-auditor**
- [P3] AGENTS.md `safeFilename()` convention implies an importable util, but it's module-private in `history.js``AGENTS.md:79`**doc-auditor** *(see P0 file-write)*
**P3 — hardening / hygiene**
- [P3] 100MB JSON body limit on all routes is a cheap memory-exhaustion lever for unauthenticated `/api/process``server/index.js:203`**evaluator**
- [P3] `/api/credits/claim`: a leaked anon BTCPay invoice ID lets any signed-in account claim those credits — `server/credits-purchase.js:342`**security-auditor**
- [P3] `downloadPodcastAudio` has no size/time cap (disk/memory fill) — `server/audio.js`**security-auditor**
- [P3] Container runs as root (no `USER` in `Dockerfile`) — acceptable under StartOS isolation, add non-root for the cloud image — **security-auditor**
- [P3] In-memory auth rate-limit buckets reset on restart — `server/auth-routes.js:106`**security-auditor + exerciser**
- [P3] Stale `youtube-summarizer_x86_64.s9pk` (~223MB) + root `package.json` still named `youtube-summarizer-startos` — leftover from the package-ID rename — **start9-spec-checker**
- [P3] Manifest `docsUrls: []` empty; multi-tenant cloud actions exposed in the single-mode StartOS menu must be verified to run cleanly (`enableMultiTenantMode` et al.) — `startos/actions/index.ts`**start9-spec-checker**
- [P3] `cookies.txt` (119KB, sensitive yt-dlp plaintext) sits in the repo root and is expiring imminently (`/api/health` already reports `fileExpiring:true`) — correctly gitignored — **evaluator + exerciser**
- [P3] 175 StartOS version files, only ~3 with real migrations (172 empty `up`/`down` stubs) — maintenance surface — **evaluator + start9-spec-checker**
## Scorecard
The evaluator's six-lens table, with two lenses adjusted on cross-agent evidence:
| Lens | Evaluator | Adjusted | Why adjusted |
|---|---|---|---|
| Architecture | 3 | 3 | — |
| Security | 4 | **2** | The evaluator sampled `index.js`/`relay.js` via grep and found no P0; the exerciser (file write, reproduced) and security-auditor (SSRF read-back + committed live key) each found a P0 the evaluator's sample missed. Three P0s on the cloud surface contradict a 4. |
| Performance | 3 | 3 | — |
| Testing | 3 | 3 | Corroborated by the exerciser; every code P0/P1 lives in an untested file. Borderline 2. |
| Code quality | 3 | 3 | — |
| Documentation | 4 | **3** | The doc-auditor found moderate drift the evaluator's "candid docs" read didn't weigh: a missing gate-order state, a directory layout missing half the server, and stale Gemini-first operator instructions. |
Not a lens, but tracked separately: **StartOS packaging = BLOCKED for registry submission** (3 hard blockers), fine for `make install`.
## Disagreements & gaps
- **Disagreement on Security posture.** The evaluator rated Security 4 ("fundamentals done right"); the security-auditor and exerciser found three P0s. Both are partly right — the *fundamentals* (SQL, auth, billing trust model) genuinely are strong, but the *user-controlled-input → side-effect* paths (`library.js` filename, `audio.js` URL fetch) were the evaluator's blind spot because it sampled the big files by grep rather than reading `library.js`/`audio.js` in full. Adjusted Security to 2 to reflect the reproduced exploit.
- **Shared blind spot — the real pipeline was never run end-to-end.** No agent had a Gemini key or relay credits, so the actual summarize → `saveToHistory` → credit-debit happy path under concurrency is unverified by *anyone*. The credit-TOCTOU (P2) and the concurrency lock (P1) are reasoned from code, not observed. This is the single highest-value gap to close with a live integration test.
- **Deployment-dependent finding.** The `X-Forwarded-For` bypass (P1) can't be confirmed from this repo — it hinges on whether the recaps.cc edge proxy appends or overwrites XFF. Self-hosted StartOS (tunnel sets XFF) is unaffected.
- **Spec-checker couldn't fully verify two things read-only:** whether the local `start-cli` pack hard-fails without `instructions.md` (its `list-ingredients` omits it — ambiguous), and whether the multi-tenant actions run cleanly in single mode (needs an on-device run).
## Surprises (carried forward from every agent)
- The pipeline detects relay errors containing "Parakeet/Gemma/CUDA" (`index.js:3419`) and then forwards that exact string to cloud users two lines later — it knows the secret and leaks it anyway. (evaluator)
- The SSRF guard the codebase clearly knows it needs — it strips `::ffff:` and validates IPs for the trial cap — is entirely absent from the one place that fetches a user-controlled URL. (security-auditor)
- The billing/settlement trust model (a common place for funds bugs) is correctly server-to-server, relay-verified, and idempotent — a *positive* surprise. (security-auditor)
- The first `/api/process` after boot triggers a yt-dlp + Homebrew self-update, adding 510s latency before the request completes. (exerciser)
- AGENTS.md's server directory layout documents only a skeleton — more than half the `server/*.js` files are unlisted; it reads like it was written when the server was much smaller. (doc-auditor)
- `start-cli s9pk list-ingredients` does not include `instructions.md` even though the docs say the build fails without it, and the root `package.json` is still named `youtube-summarizer-startos`. (start9-spec-checker)
## Suggested order of work
1. **Rotate the leaked Gemini key now**, then purge `d5046a0:.env` from history (BFG/filter-repo) and force-push. Independent of all code — do it first. *(P0 secret)*
2. **Close the two reachable code P0s:** add an SSRF guard to `downloadPodcastAudio` (reject private/link-local/loopback IPs, https-only, re-validate on redirect, size/time cap — folds in the P3 cap), and validate the import session key. Export `safeFilename()` as a shared util and call it from `library.js` + the history-move/import paths (this also closes the P2 `_meta.json` pollution). *(P0 file-write + SSRF)*
3. **Fix the two P1 correctness bugs:** the ESM `require()` in `license-purchase.js`, and scope the concurrency lock per-identity (or skip it entirely) in multi-mode. *(P1)*
4. **Add a live integration test** for `/api/process` gating + library import + a happy-path summarize-with-debit (with a stub/real key) — this is the shared blind spot and the regression net for steps 23. Then close the credit TOCTOU and the SSE error-boundary scrub against that test. *(P2)*
5. **Patch dependency CVEs** (`npm audit fix`; nodemailer is a major bump) and the multi-mode server-key fallback. *(P2)*
6. **Reconcile the docs:** add the "paid cloud user" gate-order state, rewrite the operator INSTRUCTIONS.md / ABOUT.md as relay-default (not Gemini-first), fix the directory layout and the Bearer/line-count/`safeFilename` nits. *(P2/P3)*
7. **Only if submitting to the registry:** add `instructions.md`, point `packageRepo`/`upstreamRepo` at a public source repo, choose a source-available license, and remove the stale `youtube-summarizer_x86_64.s9pk`. Otherwise these are non-issues for `make install`. *(P1 conditional)*
+99
View File
@@ -0,0 +1,99 @@
# ROADMAP
Longer-term backlog for Recaps. Near-term in-flight work and known issues live in `AGENTS.md` under **Current state**.
## Near-term backlog
- **Persist provider preference server-side.** `processItemInternally` currently runtime-detects (relay-if-configured / gemini-fallback) because the user's choice lives only in the client's `localStorage`. Persist it so a fresh-container rebuild or any non-browser caller (cron, background processor) picks the right path. Probably a single key in the StartOS config blob + a small migration to seed it from the first authenticated client.
- **Apply Export ▾ menu to the clip-collection panel.** The main view and history rows already have it; the clip collection still has the single legacy "Export PDF" button. Reuse the existing menu component.
- **CI lint + type-check.** No `lint` script in `server/package.json`; top-level `tsconfig.json` exists but the server is pure `.js`. Decide: add ESLint, adopt JSDoc-driven TS checking, or remove the empty `tsconfig.json`.
- **Surface failed auto-queue items in the dashboard.** Currently hidden by default behind a "Show all" toggle. Worth a small banner / count chip when failures exist so operators notice without hunting.
- **Zaprite recurring card billing (BLOCKED on Zaprite).** Grant wants card payments to DEFAULT to recurring (buyer can opt out at checkout). Zaprite's public API (`api.zaprite.com/openapi.json`) only creates one-time `/v1/orders` — recurring is a hosted/dashboard feature with no per-buyer metadata, no renewal webhook, and no billing-portal URL via API. The shipped card rail is one-time prepaid. UNBLOCK by confirming with Zaprite support whether the account can: (a) attach a per-buyer reference/metadata to a recurring checkout (so a payment maps to a Recaps user), (b) fire a webhook on each renewal charge (so we extend the tier each period), (c) expose a customer/billing-portal URL (for the chosen "link to Zaprite portal" cancel path). Decisions already made: no reminder emails for auto-renewing cards; a failed charge = lapse at period end (the relay's expiry-enforcement already does this — a missed renewal just doesn't extend `expires_at`).
- **Close the architecture-simplification gaps** (`docs/architecture-simplification-plan.md`). After core-decoupling + self-serve, these steps remain OPEN: **(8) "Take Recaps home"** — mint a fresh Keysat token on demand at click time; likely BROKEN today because relay-tier cloud users have no `keysat_license` for `/api/account/license-key` to return. **(10) cloud paid-only** — the free signed-in tier + signup-grant credits are still live; the plan wanted cloud to be paid-only with self-hosted as the free path (product call — confirm intent before building). **(5, partial) anon signup→Pro** still routes through `/api/license/purchase` + `pending_signups` (Keysat license) instead of the relay tier like the signed-in flow does. **(6, partial) tokenized renew** — the reminder email's renew link is `?renew=1` (requires sign-in); the plan wanted a one-time-token `/renew?token=…` for friction-free renewal. NOTE: the doc's Zaprite-*recurring* / cancel-button / Recaps-DB-owns-expiry parts were intentionally SUPERSEDED by the prepaid + relay-owns-tier model — don't build those.
- **Decide the Max tier-quota default.** The relay code default is `max.monthly: null` (unlimited) → cards render "Unlimited" on a fresh install. The operator set `max.monthly: 120` on their box via the Adjust-Tier-Quotas action (so cards show 120 there). Decide whether a metered number (e.g. 120) should be the shipped default in `recap-relay/server/config.js` — note it also enforces the ceiling, not just the card label.
- **Add Gemini 3.5 to model selection.** First have a research agent confirm which stable Gemini model versions are actually available and the correct model id/name before wiring anything. The model list is duplicated server + client (provider config under `server/providers/` + the model picker in `public/index.html`) — add the option in lockstep, like the URL-parser convention. Coordinate with the matching relay-side capture (the relay routes Gemini, so its model list must agree). — captured 2026-06-16
## Design-contract conformance cleanup (from the 2026-06-16 `/design` extract)
The `design/` contract (`design/DESIGN.md` + `design/tokens.tokens.json`) was extracted
from the as-built UI and reconciled with Grant on 2026-06-16. The code was structurally
aligned but a set of legacy values had survived as off-contract drift.
**Phase 1 — DONE 2026-06-16 (not yet deployed to the box).** Introduced a canonical `:root`
token block (the single source of truth, mirroring `tokens.tokens.json`) at the top of the
`public/index.html` `<style>` block and migrated the whole stylesheet to `var(--token)`;
`public/auth.html` got its own subset `:root` and was migrated too. Fixed **all** color +
weight drift across every surface (stylesheet, ~447 inline styles, JS handlers, the
`SHARE_PAGE_CSS` export): legacy indigos `#6366f1`/`#4f46e5`/`#4338ca` + `rgba(99,102,241,…)`
`#818cf8`/`#a5b4fc`/`rgba(129,140,248,…)`; blue `#3b82f6` interactive buttons (incl. the
whole auth screen) → indigo; legacy darks `#0a0e17`/`#0b1120`/`#020617`/`#121828`/`#1f2942`
→ the ladder; `#f5f9ff``#f1f5f9`; `#312e81``#1e293b`; weights `650→600`, `680→700`.
Verified: 144 tests pass, both pages serve 200, all 426+27 `var()` references resolve, no
undefined vars. (`SHARE_PAGE_CSS` and `auth.html` are standalone documents that each carry
their own copy — kept in sync; the meta `theme-color` stays a literal `#0a0e1a`.)
**Phase 2 — DONE 2026-06-17 (shipped in app 0.2.161).**
- **Var-ified the long-tail inline `style=` attributes** — 346 inline-style hexes (+7
`#475569`, mapped by property to `--text-faint`/`--border-strong`) → `var(--token)`. Scoped
to CSS-value position (hex preceded by `:`/space/`,`, never a quote), which cleanly dodged
the non-`var()` spots: the `<meta theme-color>`, SVG `fill`/`stroke` attrs, and hex held in
JS *logic* (quoted ternary branches like `${cond ? "#1e293b" : ...}`, `const colour = …`).
Left as literals on purpose: `#fff` (its uses split between on-accent button text and
functional white — `--on-accent` doesn't cleanly cover both, zero visual gain), no-token
hexes (`#e0e7ff`/`#c7d2fe`/`#a78bfa`/`#04210f`/etc.), and the entire `SHARE_PAGE_*` export
region (a standalone doc with no `:root``var()` wouldn't resolve there).
- **Snapped off-scale font sizes** (21 occ): `9/10.5→10`, `11.5/12.5→12`, `15→16`, `24→22`.
Left `40px`/`56px` display glyphs (success numeral, buy spinner) — off-scale by design.
- **Snapped off-scale radii** (18 occ): `3→4`, `5→6`, `7→6`, `11→12`; `9→10` for the two 18px
capsules (`.menu-badge`, `.rc-spk` — radius clamps at 9 on an 18px box, so on-scale and
visually identical) and `9→8` for `.icon-btn`/`.buy-select-btn`/`.buy-discount-input`. Left
the `1px` hamburger-bar radius. Verified: 144 tests pass, both pages serve 200, every
introduced `var()` resolves against `:root`, no off-scale residue.
- **(stretch, NOT done) Generate `design/brand/palette.css` from the tokens** (Style
Dictionary) and `@import`/inline it, so the `:root` block isn't hand-maintained in three
places. Still open.
## Known debt (P2, from the 2026-06-14 full-eval — `EVALUATION.md`)
Real but not release-blocking for self-host. The P0/P1 findings from the same eval were fixed 2026-06-15 (see git log + `EVALUATION.md`).
- **Operator-internal strings leak to cloud users at the SSE error boundary** (Parakeet/Gemma/CUDA/LAN IPs) — no scrub exists, violating the scrub convention in `AGENTS.md`. `server/index.js:3432,3003,4246` + `providers/relay.js:135-144`. (Sharp edge: `index.js:3419` *detects* these strings, then forwards them anyway.)
- **Credit over-spend TOCTOU on licensed installs** — N parallel requests pass the `total>0` check before any blind `debitOne` lands. Make check+debit atomic (reserve up front, refund on failure). `index.js:2497-2550` vs `:3158,:4197`.
- **Multi-mode tenant can spend the operator's server Gemini key** via `transcriptionProvider:"gemini"` + empty key (bypasses relay metering) — `providers/index.js:104-114`. Refuse the operator-key fallback for non-admin tenants.
- **`GET /api/history` parses every full session file** (transcript+summary, MB each) just to list ~8 metadata fields — cache them into `_meta.json` on save. `history.js:418-437`.
- **Dependency CVEs** — nodemailer 6.10.1 (high; low practical reach here), ws/qs/express/protobufjs (moderate). `npm audit fix` (nodemailer is a major bump).
- **No tests on the riskiest files** (`/api/process` gating, `relay.js`, `tenant-auth.js`, billing) — the real summarize→save→debit path can't run end-to-end without a key/credits. Add an integration test as the regression net.
- **Smaller hardening:** unsanitized IDs persisted to `_meta.json` (array-form library import + `PUT /api/history/move`) — no file-path escape (read-time `safeFilename` guards the load), but sanitize at write too; `PUT /api/history/meta` accepts arbitrary JSON shapes with no schema; `index.js` is 4351 lines mixing routing/pipeline/yt-dlp/SSE.
- **Doc drift (high-value):** AGENTS.md credit-gate order omits the "paid cloud user" bypass state (`:77` vs `index.js:2464-2472`); operator-facing `startos-registry/.../INSTRUCTIONS.md` + `assets/ABOUT.md` are stale Gemini-first (relay is the default provider).
## Deferred hardening & cleanup (P3, from the 2026-06-14 full-eval — `EVALUATION.md`)
Low-severity; batch when convenient. None block release. (P0/P1 work queue and P2 known debt live in `AGENTS.md` → Current state.)
- **Request-size / fetch caps.** `express.json({limit:"100mb"})` on every route (`server/index.js:203`) is a cheap memory-exhaustion lever; tighten it. `downloadPodcastAudio` also needs a size/time cap — folds into the P0 SSRF fix.
- **`/api/credits/claim` invoice-ID hijack.** A leaked anon BTCPay invoice ID is claimable by any signed-in account (`server/credits-purchase.js:342`); bind claims to the buyer's email. Random IDs keep this low-risk.
- **Container runs as root** (no `USER` in `Dockerfile`) — acceptable under StartOS isolation; add a non-root user for the cloud image.
- **In-memory auth rate-limit buckets reset on restart** (`server/auth-routes.js:106`) — fine for self-host single-operator; note for cloud HA.
- **Repo hygiene.** Delete the stale `youtube-summarizer_x86_64.s9pk` (~223MB, old package ID) and rename the root `package.json` (still `youtube-summarizer-startos`). `cookies.txt` is sensitive plaintext in the repo root and is expiring (`/api/health` already reports `fileExpiring:true`) — gitignored, but rotate/move it.
- **StartOS community-registry submission — deferred (decision 2026-06-15: self-host + cloud only for now).** Hard blockers if/when we submit: add a root `instructions.md`; point `packageRepo`/`upstreamRepo` at a public source repo (currently `https://ten31.xyz`, a homepage); choose a source-available license for the wrapper (currently `Proprietary`). Softer polish: empty `manifest.docsUrls`; verify the multi-tenant cloud actions (`enableMultiTenantMode` et al., `startos/actions/index.ts`) run cleanly — not stack-trace — in single mode; 172 empty version-file migration stubs are a growing maintenance surface. None of this affects `make install`.
- **Doc reconciliation (bulk).** AGENTS.md directory layout omits ~25 server modules; `docs/guides/relay-client.md:17` Authorization header is missing the `Bearer` scheme; `index.html` is stated as `~10k` lines but is 12.5k (`AGENTS.md:51`); the `safeFilename()` convention (`AGENTS.md:79`) becomes accurate once the function is exported (the P0 fix).
## Larger plans (already drafted in `docs/`)
- `docs/architecture-simplification-plan.md` — broader simplification arc
- `docs/core-decoupling-plan.md` — separating the core summarize pipeline from billing / multi-tenant concerns
- `docs/per-tenant-subscriptions-plan.md` — moving subscription state into the per-user scope
- `docs/self-serve-purchase-plan.md` — buyer flow for Pro/Max and a la carte credits
- `docs/path-2b-and-path-1-interweave.md` — sequencing for the multi-tenant cloud meetings work (depends on the relay's Path 2A)
Treat the `docs/` plans as the source of truth for those items; cross-reference rather than restating here.
## Adjacent (lives in `../recap-relay`)
The relay now has its own `AGENTS.md` + `ROADMAP.md` — track relay work there; this is just what the client surfaces or waits on.
- **Speaker MERGE + re-run detection + re-polish — SHIPPED relay-side** (operator dashboard, live on the box at relay 0.2.124, 2026-06-13). Merge folds two clusters into one; re-run re-clusters at a new strictness to split over-merged speakers; re-polish rewrites topic summaries to corrected names. App-side UI for these is now unblocked if wanted. *(The relay tree is at 0.2.124 but uncommitted to git — see `../recap-relay/ROADMAP.md`.)*
- Cross-call speaker fingerprint memory (recognize the same voice across meetings) — not yet shipped.
- Phase 3 of Path 2A: multiple operator-editable meeting prompt sets (1on1 / all-hands / customer-interview / standup) selectable per upload — not yet shipped.
Avoid building app-side UI for the unshipped items until the relay-side pieces land.
+2297
View File
File diff suppressed because it is too large Load Diff
+4 -4
View File
@@ -9,8 +9,8 @@
#
# If the file .release-notes-pending.txt exists in the project root, its
# contents are shown as the default release notes (just press Enter to accept).
# This is the convention Claude uses after making code changes. The file is
# deleted on a successful bump.
# Drop a note in that file before running this script to pre-fill the
# prompt. The file is deleted on a successful bump.
#
# Flags:
# --from-deploy Treat the absence of .release-notes-pending.txt as the
@@ -84,8 +84,8 @@ fi
NEW_VAR="v_$(echo "$NEW_VERSION" | tr '.' '_')"
# --- Prompt for release notes ---
# If Claude (or you) left suggested notes in .release-notes-pending.txt, show
# them as the default. Press Enter to accept, or type something different.
# If suggested notes are sitting in .release-notes-pending.txt, show them
# as the default. Press Enter to accept, or type something different.
SUGGESTED_NOTES=""
if [ -f "$PENDING_NOTES_FILE" ]; then
# Read the file, trim leading/trailing whitespace, collapse interior newlines
+6 -6
View File
@@ -13,10 +13,10 @@
# START9_SERVER — your Start9 server, e.g. https://immense-voyage.local:62185
#
# Optional config (sensible defaults):
# FILEBROWSER_PATH — path on FileBrowser to overwrite. Default: /websites/packages/recap_x86_64.s9pk
# REGISTRY_URL — registry JSON-RPC URL. Default: https://registry.satsflows.com
# FILEBROWSER_PATH — path on FileBrowser to overwrite. Default: /websites/keysat-registry/recap_x86_64.s9pk
# REGISTRY_URL — registry JSON-RPC URL. Default: https://registry.keysat.xyz
# REGISTRY_PUBLIC_URL — public .s9pk URL registered with start-cli.
# Default: https://files.satsflows.com/recap_x86_64.s9pk
# Default: https://files.keysat.xyz/recap_x86_64.s9pk
set -euo pipefail
@@ -35,9 +35,9 @@ fi
: "${FILEBROWSER_PASS:?FILEBROWSER_PASS is required}"
: "${START9_SERVER:?START9_SERVER is required (e.g. https://immense-voyage.local:62185)}"
FILEBROWSER_PATH="${FILEBROWSER_PATH:-/websites/packages/recap_x86_64.s9pk}"
REGISTRY_URL="${REGISTRY_URL:-https://registry.satsflows.com}"
REGISTRY_PUBLIC_URL="${REGISTRY_PUBLIC_URL:-https://files.satsflows.com/recap_x86_64.s9pk}"
FILEBROWSER_PATH="${FILEBROWSER_PATH:-/websites/keysat-registry/recap_x86_64.s9pk}"
REGISTRY_URL="${REGISTRY_URL:-https://registry.keysat.xyz}"
REGISTRY_PUBLIC_URL="${REGISTRY_PUBLIC_URL:-https://files.keysat.xyz/recap_x86_64.s9pk}"
S9PK_FILE="$PROJECT_ROOT/recap_x86_64.s9pk"
+213
View File
@@ -0,0 +1,213 @@
# Recap — Design brief
The durable brand contract for Recap's user-facing UI. Read this and
`design/tokens.tokens.json` before building or changing any UI. This was
**extracted** from the as-built `recaps.cc` interface (no prior guidelines), then
**reconciled** with the owner on 2026-06-16 — so where the live code still
disagrees with a value here, *this file is the intent* and the code is the cleanup
backlog (see `ROADMAP.md`). Provenance: there was no design tool or export; the
source is the shipped `public/index.html`, `public/auth.html`, and the app icon.
---
## 1. Visual theme
A **dense, dark, information-first developer-tool aesthetic.** Restraint over
decoration: small type, tight radii, thin hairline borders, flat panels on a
near-black navy field, with the indigo accent used sparingly to mark what's
interactive or active. It should read like a fast, technical instrument — closer
to a code editor or an ops dashboard than a consumer media app. Calm, quiet, and
legible at high density; never glossy, never playful.
The brand mark (app icon) is a **play-triangle filled with a blue→purple gradient
over four light "transcript" lines**, on the dark navy field — the product in one
glyph: *press play on a video, get the text back.* That blue→purple gradient is
the origin of the whole accent story (§2).
Voice: precise, plain-spoken, unhyped. The UI explains tersely and trusts the
user. What it is **not**: not gradient-heavy, not glassmorphic beyond a light
overlay blur, not rounded-and-bubbly, not light-mode, not framework-flashy.
## 2. Color palette
Built almost entirely on the **Tailwind Slate + Indigo** ramps over a custom
near-black navy base. Canonical roles below; full values in
`tokens.tokens.json`.
**Surface ladder (warm/cool split — the agreed system).**
| Role | Value | Where |
|---|---|---|
| Base | `#0a0e1a` | page, body, sticky top bars, full-height side panels (history sidebar, log drawer). Also the PWA `theme_color`. |
| Card | `#111827` (warm gray-900) | raised cards, modals, popovers, toasts, icon-buttons, pipeline steps, skeletons. |
| Inset | `#0f172a` (cool slate-900) | recessed fields & list rows: settings/key inputs, queue items, subscription items, model buttons, expanded-chunk bg. |
| Raised field | `#1e293b` | the **primary URL input only** — deliberately lighter than the inset so the hero field pops. |
> Legacy darks **`#0a0e17`, `#0b1120`, `#020617`** are near-duplicates that
> leaked in; fold them into the nearest rung (`#0a0e17`/`#0b1120` → Base,
> `#020617` → Inset). Don't add new background darks.
**Accent — indigo (THE single interactive accent).**
- `#818cf8` (indigo-400) — **the accent.** Fills (submit button, active tab/icon,
processing badge) and marks (links, focus ring, active state, timestamps,
expanded-chunk arrow).
- `#a5b4fc` (indigo-300) — accent hover (lighter), active-line emphasis.
- Indigo tints `rgba(129,140,248, .06.20)` — hover washes, active backgrounds,
the `0 0 0 3px …/.15` focus ring, the `0 4px 24px …/.3` submit glow.
- **Demoted:** `#6366f1` (indigo-500) and its hovers `#4f46e5`/`#4338ca` are
legacy dups — migrate them to `#818cf8`; do not introduce new ones.
**Premium — purple (RESERVED for paid/upgrade only).**
- `#a855f7` (purple-500) — upgrade button, highlighted tier, buy badge, primary
buy CTA. `#c084fc` hover; `#9333ea` deep.
- `#c4b5fd`/`#d8b4fe` (purple-300/200) — tier-badge & Pro text.
- Purple appears **nowhere** in non-premium UI.
**Text ramp (slate).**
`#e2e8f0` primary · `#f1f5f9` strong/headings (canonical near-white — fold the
stray `#f5f9ff` into this) · `#cbd5e1` running body copy · `#94a3b8` muted/
secondary · `#64748b` labels & placeholders · `#475569` faint meta/timestamps ·
`#334155` dimmest (doubles as the hover-border). `#fff` only on filled
accent/premium buttons.
**Borders.** `#1e293b` (slate-800) default hairline → `#334155` (slate-700) on
hover/active → `#475569` (slate-600) strongest hover. Borders, not shadows, are
the primary way surfaces are separated.
**Semantic status (consistent, intentional — keep).**
- Success/green: `#22c55e` base · `#4ade80` text · `#86efac` soft · `#16a34a` deep.
- Error/red: `#ef4444` base · `#f87171` text · `#fca5a5` soft · `#dc2626` deep.
- Warning/amber: `#fbbf24` base/dot · `#fcd34d` · `#fde68a` · `#f59e0b` (update btn).
- Info/blue: `#3b82f6` base · `#60a5fa` · `#93c5fd`. **Blue is status/info only**
(and the legacy auth-screen accent, which should migrate to indigo) — never a
primary interactive color in-app.
**Speaker chips (8-hue categorical set — intentional, keep as-is).** A→H:
red/blue/green/amber/purple/sky/pink/slate, each as the *tinted triplet*
(background α≈.18, light-shade text, border α≈.35).
## 3. Typography
- **Sans (everything):** `-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
sans-serif`. System stack — **no web fonts.**
- **Mono (timestamps, license keys, URLs, code):** `ui-monospace, "SF Mono",
Menlo, Consolas, monospace`. One canonical stack — snap the near-variants to it.
- **Scale (px):** `10 · 11 · 12 · 13 · 14 · 16 · 18 · 20 · 22 · 28`.
- 1011 — micro labels, badges, meta, section labels, pills, hints.
- 12 — secondary UI default (buttons, labels, stats, list titles).
- 13 — body default (descriptions, list items, transcript-adjacent).
- 14 — emphasis / larger body, drawer & settings headers.
- 16 — input text, modal `h2`, section heads.
- 18 — sub-headers, logo wordmark, large numerals.
- 2022 — headings, modal/activation titles, tier names.
- 28 — display (price). Snap strays `15→16`, `24→22`, fractionals
`12.5/11.5→12`, `10.5/9→10`.
- **Weights:** 400 normal · 500 medium · **600 semibold (the UI default)** · 700
bold (headings, badges) · 800 extrabold (tier badge, big numerals). Snap
`650→600`, `680→700`.
- **Line-height:** 1.5 body · 1.55 denser reading blocks · 1.31.4 titles · 1.25
large titles · 1 single-line badges.
- **Letter-spacing:** 0 default. Uppercase micro-labels track `0.05em`; badges
`0.04em`; tier/section labels `0.060.08em`; one display title tightens
`-0.01em`.
## 4. Component styling
- **Buttons.** *Primary:* filled `#818cf8`, white text, radius 810px, weight
600, hover lifts `translateY(-1px)` + indigo glow; disabled → `#1e293b` bg /
`#475569` text. *Secondary:* `#1e293b` fill, `#334155` border, `#94a3b8`→
`#cbd5e1` text. *Icon/ghost:* `#111827` (or transparent), `#1e293b` border,
muted glyph; hover lightens bg + border; `.active` → accent fill. *Premium:*
filled `#a855f7`.
- **Inputs.** Inset `#0f172a` (or `#1e293b` for the hero URL input), `#334155`/
`#1e293b` border; focus → accent border + `0 0 0 3px rgba(129,140,248,.15)`
ring. Mono font for keys/codes; placeholders `#64748b`/`#475569`.
- **Cards.** `#111827` bg, `#1e293b` border, radius 1014px; border lightens on
hover; expanded/active → accent border + faint `0 2px 16px …/.06` accent glow.
- **Pills / badges / chips — the "tinted triplet".** Background at α≈.1,
text in the light shade, border at α≈.2.45 of the same hue. Status pills,
tier badges, `queue-from` tags, clip badges, and speaker chips all follow this
one rule.
- **Modals.** `#111827` (settings) / `#0f172a` (buy) bg, radius 16px, sticky
header with `1px #1e293b` bottom border, `slideUp` entrance, scrim
`rgba(0,0,0,.6)` + `backdrop-filter: blur(4px)`.
- **Toasts.** Top-right stack, `#1e293b` bg, `#334155` border, slide-in from
right, auto-fade.
- **Spinner.** 3px ring, `#1e293b` track, `#818cf8` top, `spin 0.8s linear`.
- **Pipeline / tracker.** Stepped pills: idle neutral, `.active` → accent
tint+border, `.done` → green tint+border.
## 5. Layout
- **Single fluid column.** `.container` `max-width:100%`, padding `36px 24px`
(landing) tightening to `16px 24px` (results) and down on mobile.
- **Results = split screen.** `.results-left` 58% (video/player, sticky top) +
`.results-right` (scrolling chunk list), 16px gap; stacks vertically <900px.
- **Persistent left history sidebar**, 320px, *pushes* content on desktop /
overlays on mobile. **Right log drawer**, 440px, slides in.
- **Dense vertical rhythm.** Card margin 14px, chunk margin 6px, gaps 616px.
- **Spacing steps (de-facto):** `4 · 6 · 8 · 10 · 12 · 14 · 16 · 20 · 24` for
controls/gaps, `28 · 32 · 36` for section padding. Not a strict 4/8 grid —
dense and organic by design; prefer these steps over new in-between values.
## 6. Depth / elevation
**Flat by default — separation comes from 1px borders, not shadows.** Shadows are
reserved for things that genuinely float:
- Overlays/drawers/modals: `0 8px 24px …/.3` (toast) · `0 8px 32px …/.4` (video,
side drawers `±8px 0 32px`) · `0 12px 32px …/.5` (menu) · `0 20px 60px …/.5`
and `0 24px 64px …/.5.6` (panel, settings/buy modal). All `rgba(0,0,0,α)`.
- **Accent glows** signal primary/active: submit `0 4px 24px rgba(129,140,248,.3)`;
focus ring `0 0 0 3px rgba(129,140,248,.15)`; expanded chunk `0 2px 16px …/.06`;
premium tier `0 12px 40px rgba(168,85,247,.25)`.
- Overlay scrims use `rgba(0,0,0,.4.65)` + `backdrop-filter: blur(46px)`.
- Drag/drop affordance is an accent line: `box-shadow: 0 ±2px 0 0 #818cf8`.
## 7. Do's and don'ts
**Do**
- Use `#818cf8` as the *single* interactive accent; reserve purple `#a855f7`
strictly for premium/upgrade.
- Separate surfaces with 1px `#1e293b` borders that lighten to `#334155` on hover;
keep shadows for true overlays only.
- Build every status/category chip as the tinted triplet (bg α.1 / light text /
border α.2).
- Keep type dense — 1213px body, 1011px meta — with 600 as the default weight.
- Follow the surface ladder: base `#0a0e1a` → card `#111827` → inset `#0f172a`.
- Use the system font stack; mono only for timestamps / keys / URLs.
**Don't**
- Don't introduce new `#6366f1`/`#4f46e5` indigos — they migrate to `#818cf8`.
- Don't put blue `#3b82f6` on primary interactive elements; blue is info/status
(and legacy auth) only — auth should move to indigo.
- Don't add new background darks (`#0a0e17`/`#0b1120`/`#020617` are legacy dups).
- Don't use purple for non-premium UI, or the accent indigo for premium.
- Don't add fractional font sizes (12.5/11.5/10.5) or off-scale weights (650/680).
- Don't reach for a framework, web font, or heavy drop shadows — vanilla JS,
system fonts, flat-with-borders is the intentional language.
## 8. Responsive behavior
- **Breakpoints:** `900px` (primary: split→stack, sidebar push→overlay), `880px`
(tablet: top breadcrumb swaps to a hoisted mobile copy), `600px` (phone:
icon-only submit, hamburger menu, desktop toolbar/pills hidden), `640px` (share
export). Plus `≤900px landscape` → fullscreen video.
- **Mobile rules:** form inputs forced to `16px` (iOS auto-zoom guard); touch
targets 4448px; sticky top bar; full-width drawers/panels; hover-gated
controls pinned visible (touch has no hover). Use `100dvh` for full-height.
## 9. Agent prompt guide
When building or changing UI in this repo:
- Read `design/tokens.tokens.json` and pull values from it — don't eyeball new hex.
- Match the **dense dark vanilla-JS** style. The frontend is one file,
`public/index.html`, with two `<style>` blocks + ~447 inline `style=` attrs and
a render-into-`innerHTML` loop. No framework, no bundler, no web fonts — keep it
that way.
- Accent is `#818cf8`; purple is premium-only; follow the surface ladder; build
chips as the tinted triplet; system font for text, mono for timestamps/keys.
- **Three surfaces stay in sync:** the main app stylesheet, the `SHARE_PAGE_CSS`
string (the self-contained share export), and `public/auth.html`. A token change
must be reflected in all three. Auth currently uses the legacy blue accent —
new auth work should adopt indigo.
- Sanitize operator-internal strings at error boundaries (per `AGENTS.md`) — they
must never reach cloud users, design surfaces included.
+17
View File
@@ -0,0 +1,17 @@
# Brand assets
- **`icon.png`** — the app icon (1024×1024). A rounded-square (squircle) on the
dark navy field, with a **play-triangle filled by a blue→purple gradient** over
four light "transcript" lines. Used as the PWA icon (`manifest.json`
`/assets/icon.png`, also `assets/icon.png` / `icon.png` at repo root) and the
auth-screen logo. The gradient's blue end and purple end are the source of the
app's blue (info) and purple (premium) accents; the indigo `#818cf8` interactive
accent sits between them.
**Fonts:** none shipped — Recap uses the system font stack (`-apple-system, …`)
and a system monospace stack. No web-font files to vendor here.
**Palette:** the machine-readable source of truth is `../tokens.tokens.json`. No
generated `palette.css` yet — styles live inline in `public/index.html`;
consolidating them behind CSS custom properties is tracked as cleanup in the
repo `ROADMAP.md`.
Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

+18
View File
@@ -0,0 +1,18 @@
# Inspiration / provenance
This is an **extract** run (Case B): Recap had no prior design guidelines and no
design tool was used. There are no external reference images — the *de-facto* look
**is** the source, harvested directly from the shipped code on 2026-06-16:
- `public/index.html` — the whole app + landing (two `<style>` blocks + ~447
inline `style=` attributes).
- `public/index.html``SHARE_PAGE_CSS` — the self-contained share-export styles.
- `public/auth.html` — the magic-link auth screen.
- `public/manifest.json` — PWA name/colors/icons (`theme_color` = `#0a0e1a`).
- `design/brand/icon.png` — the app icon (the only first-class brand asset). Its
blue→purple gradient play-triangle is the origin of the accent story.
The conflict reconciliation (which dark, which indigo, the type scale, the
accent-hue strategy) was decided with the owner and is recorded in `../DESIGN.md`.
If real reference images are gathered later, drop them here with a one-line note
on what's liked about each.
+194
View File
@@ -0,0 +1,194 @@
{
"$schema": "https://tr.designtokens.org/format/",
"$description": "Recap design tokens (W3C DTCG). Extracted from the as-built recaps.cc UI and reconciled 2026-06-16. Canonical roles only; see design/DESIGN.md for usage. Composite shadows and tint scales are documented strings, not strict DTCG primitives.",
"color": {
"$type": "color",
"surface": {
"$description": "Warm/cool elevation ladder over the navy base. Legacy darks #0a0e17/#0b1120 fold into base; #020617 folds into inset.",
"base": { "$value": "#0a0e1a", "$description": "Page, body, sticky bars, full-height side panels. Also PWA theme_color/background_color." },
"card": { "$value": "#111827", "$description": "Warm gray-900. Raised cards, modals, popovers, toasts, icon-buttons, steps, skeletons." },
"inset": { "$value": "#0f172a", "$description": "Cool slate-900. Recessed fields & list rows: inputs, queue/sub items, model buttons, expanded chunk." },
"raised-field": { "$value": "#1e293b", "$description": "Primary URL input only — lighter than inset so the hero field pops." }
},
"border": {
"default": { "$value": "#1e293b", "$description": "Slate-800. Default hairline — primary surface separator." },
"hover": { "$value": "#334155", "$description": "Slate-700. Border on hover/active." },
"strong": { "$value": "#475569", "$description": "Slate-600. Strongest hover border." }
},
"accent": {
"$description": "Indigo — THE single interactive accent.",
"default": { "$value": "#818cf8", "$description": "Indigo-400. Submit/active fills, links, focus ring, timestamps, active marks." },
"hover": { "$value": "#a5b4fc", "$description": "Indigo-300. Accent hover (lighter), active-line emphasis." },
"legacy-500": { "$value": "#6366f1", "$description": "DEMOTED indigo-500 dup — migrate to accent.default. Do not introduce new." },
"legacy-600": { "$value": "#4f46e5", "$description": "DEMOTED legacy hover for #6366f1 — migrate away." }
},
"premium": {
"$description": "Purple — RESERVED for paid/upgrade UI only. Never in non-premium surfaces.",
"default": { "$value": "#a855f7", "$description": "Purple-500. Upgrade button, highlighted tier, buy badge, primary buy CTA." },
"hover": { "$value": "#c084fc", "$description": "Purple-400." },
"deep": { "$value": "#9333ea", "$description": "Purple-600. Pro-CTA hover." },
"text": { "$value": "#c4b5fd", "$description": "Purple-300. Tier-badge & Pro text." },
"text-soft": { "$value": "#d8b4fe", "$description": "Purple-200." }
},
"text": {
"primary": { "$value": "#e2e8f0", "$description": "Slate-200. Primary body/UI text." },
"strong": { "$value": "#f1f5f9", "$description": "Slate-100. Headings/emphasis near-white. Canonical — fold stray #f5f9ff into this." },
"body": { "$value": "#cbd5e1", "$description": "Slate-300. Running body copy (transcript, descriptions, bullets)." },
"muted": { "$value": "#94a3b8", "$description": "Slate-400. Secondary/muted." },
"label": { "$value": "#64748b", "$description": "Slate-500. Labels & placeholders." },
"faint": { "$value": "#475569", "$description": "Slate-600. Faint meta, timestamps." },
"dim": { "$value": "#334155", "$description": "Slate-700. Dimmest text (doubles as border.hover)." },
"on-accent": { "$value": "#ffffff", "$description": "White — only on filled accent/premium buttons." }
},
"status": {
"$description": "Semantic status ramps. Use the tinted-triplet pattern for chips (see tint.*).",
"success": { "$value": "#22c55e" },
"success-text": { "$value": "#4ade80" },
"success-soft": { "$value": "#86efac" },
"success-deep": { "$value": "#16a34a" },
"error": { "$value": "#ef4444" },
"error-text": { "$value": "#f87171" },
"error-soft": { "$value": "#fca5a5" },
"error-deep": { "$value": "#dc2626" },
"warning": { "$value": "#fbbf24" },
"warning-soft": { "$value": "#fcd34d" },
"warning-faint": { "$value": "#fde68a" },
"warning-deep": { "$value": "#f59e0b" },
"info": { "$value": "#3b82f6", "$description": "Blue-500. Info/status only (+ legacy auth accent). Not a primary interactive color in-app." },
"info-mid": { "$value": "#60a5fa" },
"info-soft": { "$value": "#93c5fd" }
},
"speaker": {
"$description": "8-hue categorical set for diarized speaker chips AH. Rendered as the tinted triplet (bg α≈.18 / this text / border α≈.35).",
"a": { "$value": "#fca5a5", "$description": "red — base rgb(239,68,68)" },
"b": { "$value": "#93c5fd", "$description": "blue — base rgb(59,130,246)" },
"c": { "$value": "#86efac", "$description": "green — base rgb(34,197,94)" },
"d": { "$value": "#fcd34d", "$description": "amber — base rgb(245,158,11)" },
"e": { "$value": "#d8b4fe", "$description": "purple — base rgb(168,85,247)" },
"f": { "$value": "#7dd3fc", "$description": "sky — base rgb(14,165,233)" },
"g": { "$value": "#f9a8d4", "$description": "pink — base rgb(236,72,153)" },
"h": { "$value": "#cbd5e1", "$description": "slate — base rgb(100,116,139)" }
}
},
"tint": {
"$type": "color",
"$description": "Recurring translucent washes. Stored as documented rgba strings — they are tints of the named hues, not standalone primitives.",
"accent-wash": { "$value": "rgba(129,140,248,0.06)", "$description": "Hover/active wash of accent.default. Family: .06 .08 .10 .15 .20." },
"accent-active": { "$value": "rgba(129,140,248,0.08)", "$description": "Active background of accent.default." },
"accent-ring": { "$value": "rgba(129,140,248,0.15)", "$description": "Focus-ring color → box-shadow 0 0 0 3px." },
"chip-bg": { "$value": "rgba(34,197,94,0.10)", "$description": "Representative tinted-triplet background (α≈.10) — swap the hue per status/category." },
"chip-border": { "$value": "rgba(34,197,94,0.20)", "$description": "Representative tinted-triplet border (α≈.20)." },
"scrim": { "$value": "rgba(0,0,0,0.6)", "$description": "Overlay scrim behind modals; family .4 .5 .6 .65." }
},
"font": {
"family": {
"$type": "fontFamily",
"sans": { "$value": ["-apple-system", "BlinkMacSystemFont", "Segoe UI", "Roboto", "sans-serif"], "$description": "System stack — no web fonts." },
"mono": { "$value": ["ui-monospace", "SF Mono", "Menlo", "Consolas", "monospace"], "$description": "Timestamps, license keys, URLs, code. Canonical — snap near-variants to this." }
},
"size": {
"$type": "dimension",
"$description": "Normalized scale (px). Snap strays 15→16, 24→22, 12.5/11.5→12, 10.5/9→10.",
"10": { "$value": "10px", "$description": "micro labels, badges, meta, section labels" },
"11": { "$value": "11px", "$description": "small labels, pills, hints, queue meta" },
"12": { "$value": "12px", "$description": "secondary UI default — buttons, labels, stats" },
"13": { "$value": "13px", "$description": "body default — descriptions, list items" },
"14": { "$value": "14px", "$description": "emphasis / larger body, drawer & settings headers" },
"16": { "$value": "16px", "$description": "input text, modal h2, section heads" },
"18": { "$value": "18px", "$description": "sub-headers, logo wordmark, large numerals" },
"20": { "$value": "20px", "$description": "headings" },
"22": { "$value": "22px", "$description": "large headings, modal/activation titles, tier names" },
"28": { "$value": "28px", "$description": "display — price" }
},
"weight": {
"$type": "fontWeight",
"$description": "Snap 650→600, 680→700.",
"normal": { "$value": 400 },
"medium": { "$value": 500 },
"semibold": { "$value": 600, "$description": "The UI default weight." },
"bold": { "$value": 700, "$description": "Headings, badges." },
"extrabold": { "$value": 800, "$description": "Tier badge, big numerals." }
},
"lineHeight": {
"$type": "number",
"body": { "$value": 1.5 },
"reading": { "$value": 1.55, "$description": "Denser reading blocks." },
"title": { "$value": 1.3 },
"title-large": { "$value": 1.25 },
"single": { "$value": 1, "$description": "Single-line badges/controls." }
},
"letterSpacing": {
"$type": "dimension",
"label": { "$value": "0.05em", "$description": "Uppercase micro-labels." },
"badge": { "$value": "0.04em" },
"tracked": { "$value": "0.06em", "$description": "Tier/section labels (up to 0.08em)." },
"tight": { "$value": "-0.01em", "$description": "Large display titles." }
}
},
"radius": {
"$type": "dimension",
"$description": "Snap strays 3→4, 5→6, 7→6|8, 9→8|10, 11→10|12.",
"4": { "$value": "4px", "$description": "tiny chips, badges, drop-zones, skeleton lines" },
"6": { "$value": "6px", "$description": "default small controls — buttons, chips, menu items" },
"8": { "$value": "8px", "$description": "default medium — inputs, buttons, pills" },
"10": { "$value": "10px", "$description": "larger cards/chunks, status bars" },
"12": { "$value": "12px", "$description": "modals, popovers, menus" },
"14": { "$value": "14px", "$description": "big cards, video embed, activation card" },
"16": { "$value": "16px", "$description": "settings / buy modal" },
"18": { "$value": "18px", "$description": "listen panel" },
"pill": { "$value": "999px", "$description": "fully-rounded badges" },
"circle": { "$value": "50%", "$description": "avatars, dots, play controls, spinner" }
},
"space": {
"$type": "dimension",
"$description": "De-facto dense scale (px). Not a strict 4/8 grid; prefer these steps over new in-between values.",
"4": { "$value": "4px" },
"6": { "$value": "6px" },
"8": { "$value": "8px" },
"10": { "$value": "10px" },
"12": { "$value": "12px" },
"14": { "$value": "14px", "$description": "card margin" },
"16": { "$value": "16px", "$description": "split-screen gap" },
"20": { "$value": "20px" },
"24": { "$value": "24px", "$description": "container horizontal padding" },
"28": { "$value": "28px", "$description": "section padding" },
"32": { "$value": "32px" },
"36": { "$value": "36px", "$description": "landing container vertical padding" }
},
"shadow": {
"$type": "shadow",
"$description": "Flat-by-default — shadows only for floating surfaces. Composite/glow values kept as documented strings.",
"toast": { "$value": "0 8px 24px rgba(0,0,0,0.3)" },
"drawer": { "$value": "0 8px 32px rgba(0,0,0,0.4)", "$description": "Video embed; side drawers use ±8px 0 32px." },
"menu": { "$value": "0 12px 32px rgba(0,0,0,0.5)" },
"panel": { "$value": "0 20px 60px rgba(0,0,0,0.5)", "$description": "Listen panel, activation card." },
"modal": { "$value": "0 24px 64px rgba(0,0,0,0.5)", "$description": "Settings/buy modal (up to ...0.6)." },
"glow-accent": { "$value": "0 4px 24px rgba(129,140,248,0.3)", "$description": "Submit button — signals primary." },
"glow-accent-faint": { "$value": "0 2px 16px rgba(129,140,248,0.06)", "$description": "Expanded chunk." },
"ring-accent": { "$value": "0 0 0 3px rgba(129,140,248,0.15)", "$description": "Input focus ring." },
"glow-premium": { "$value": "0 12px 40px rgba(168,85,247,0.25)", "$description": "Highlighted premium tier." }
},
"motion": {
"$description": "Transitions are quick and functional. Durations as documented strings.",
"duration": {
"fast": { "$value": "0.15s", "$description": "Default control transition (transition: all 0.15s)." },
"base": { "$value": "0.2s", "$description": "Cards, layout shifts." },
"slow": { "$value": "0.4s", "$description": "Chunk-body max-height expand." }
},
"overlay-blur": { "$value": "blur(4px)", "$description": "Modal/overlay backdrop-filter; 46px range." }
}
}
+261
View File
@@ -0,0 +1,261 @@
# Recaps Architecture Simplification — Plan of Record
**Status:** Agreed direction, 2026-05-19. Not yet implemented.
## Why this exists
The current Recaps architecture has the Keysat license doing too many jobs.
For a cloud Pro/Max tenant, the license is acting as: (a) the entitlement
check, (b) the relay credit-pool key, (c) the subscription-expiry source,
AND (d) the take-it-home portability token. Only (d) actually requires a
cryptographically-signed token. The other three are accidental — the
license just happens to carry that data because Keysat was already minting
license tokens during the MVP.
This doc captures the agreed simplification so we can revisit later before
implementation.
---
## The three products, cleanly separated
**Recap Relay** — Backend compute service. AI provider routing
(Gemini / Claude / Whisper / Ollama / etc.), credit ledger, BTCPay-backed
top-ups. Runs on Grant's StartOS. Sold as a service (subscription gives
monthly credit allotment + a la carte top-ups).
**Recaps** — Frontend cloud SaaS for summarizing podcasts/videos. People
sign up, pay a subscription, and use the relay for compute. Hosted at
`recaps.cc`. Also freely available as a `.s9pk` for self-hosting.
**Keysat** — Standalone licensing-as-a-service software, separately
monetized as a B2B product. Recaps happens to use Keysat for the one
specific case of minting a portable license token when a cloud user
clicks "Take Recaps home." Otherwise unrelated to Recaps day-to-day.
These are three products that happen to share an author. Tangling them is
what made the current architecture confusing.
---
## What each layer owns after the simplification
| Concern | Lives in | Why |
|---|---|---|
| Subscription tier + `expires_at` | **Recaps DB** (`users.tier`, `users.subscription_expires_at`) | Billing state belongs with the billing surface. Zaprite/BTCPay webhooks fire at Recaps and update one row. |
| Credit balance (remaining/consumed) | **Relay's ledger** (`credits.json`) | Compute happens at the relay. The relay knows when N credits were actually spent on Gemini tokens. Race conditions get ugly if billing and consumption drift. |
| Purchased credit top-ups (one-shot Lightning) | **Relay's ledger** | BTCPay webhook → relay → bump `purchased_balance` on the relevant pool. Unchanged from today. |
| Monthly tier allotment | **Relay computes** from tier header passed by Recaps | Recaps sends `X-Recap-User-Tier: pro`; relay applies its quota config (`pro.monthly = 50`). |
| Per-user identity at the relay | Keyed by `user:<recaps_user_id>` (cloud) or `lic:<fp>` (self-hosted) | Removes the license-as-credential coupling for cloud requests. License only matters for self-hosted. |
### The header change
```
Cloud Recaps → Relay (today):
Authorization: Bearer <user's LIC1 token>
Cloud Recaps → Relay (after simplification):
X-Recap-User-Id: <recaps_user_id>
X-Recap-User-Tier: pro
Authorization: Bearer <OPERATOR's LIC1> ← proves THIS Recaps server is authorized
```
The operator's bearer token is still needed because the relay needs to
verify that this is Grant's cloud Recaps server (vs. someone else trying
to forge user-id headers). The per-user identity comes from the explicit
headers.
---
## Cloud Recaps is paid-only
**Decided:** cloud Recaps drops the "free signed-in" tier entirely. Self-
hosted IS the free path.
User states on cloud:
- **Anon trial** — cookie-tracked, no account. Gets a small allowance of
credits to taste-test. After the trial runs out, must subscribe to
continue using the cloud service.
- **Subscribed** — account exists, `users.tier ∈ {pro, max}`,
`subscription_expires_at` in the future.
- **Expired subscription** — account preserves the library, can't
summarize new things until renewed. Renewal anytime restores access.
What this means for the codebase: `tenant_credits` table's "free signed-in"
codepath stops applying to cloud users. The table stays in the codebase
because self-hosted multi-tenant operators (Alice running Recaps for her
family) still need per-tenant accounting locally.
---
## Payment provider strategy
**Pro/Max purchase page** has two paths:
```
┌─────────────────────────────────────────────┐
│ Upgrade to Pro — $X/month │
│ │
│ • 50 Recap credits per month │
│ • Channel + podcast subscriptions │
│ • Auto-queue + priority processing │
│ │
│ ┌──────────────────────────────────┐ │
│ │ ⚡ Pay with Bitcoin │ │ ← primary, inline BTCPay
│ └──────────────────────────────────┘ │
│ │
│ Pay with card → │ ← link → Zaprite hosted
│ │
└─────────────────────────────────────────────┘
```
### Bitcoin path: monthly upfront, manual renewal
- User pays one BTCPay invoice for 30 days of Pro
- Inline Lightning QR + BOLT11 + copy button (same UX as credit packs)
- On settle, Recaps sets `subscription_expires_at = now + 30 days`
- No card on file, no autorenewal — Lightning doesn't have a clean
recurring-billing primitive
### Renewal-link emails (NEW, needed for Bitcoin path)
Recaps sends automated emails near expiry:
- **7 days before expiry** — "Your Pro sub renews in 7 days. Tap to renew →"
- **Day of expiry** — "Your Pro sub expired. Tap to renew →"
- **7 days after expiry** — "We've paused your Pro features. Renew anytime →"
- After 30 days post-expiry — stop emailing (avoid being a nag)
Mechanism:
- Email contains a tokenized URL: `https://recaps.cc/renew?token=<base64>`
- Token encodes `{ user_id, action: "renew_pro" }`, short-lived (~14 days)
- Click → Recaps mints fresh BTCPay invoice, renders inline Lightning UI
- On settle, `subscription_expires_at += 30 days` (extends from whichever
is later: existing expiry, or current time — so a user who renews 5
days early doesn't lose those days)
Card subscribers don't need this — Zaprite handles recurring billing
natively via Stripe.
### Card path: Zaprite handles everything
- Click "Pay with card" → redirect to Zaprite hosted checkout
- Zaprite collects card info, charges monthly, handles retries on failure
- Zaprite webhook fires → Recaps updates `subscription_expires_at`
- Cancel button in Settings → Plan calls Zaprite cancel API
- **Card premium pricing handled inside Zaprite admin** (not in Recaps
code) — Grant configures the card-monthly price to be N% higher than
Bitcoin-monthly in Zaprite's product settings
---
## Self-hosted scenarios
**Self-hosted Recaps is free + open source.** Run the `.s9pk` anywhere,
no license check to run the software.
| Scenario | Subscriber to Grant's relay? | How relay access works |
|---|---|---|
| Bob runs Recaps for himself, brings his own Gemini key | No | Bob pays Google directly. Default `.s9pk` has no relay configured — Bob enters his AI provider keys in Settings. |
| Bob runs Recaps for himself + wants Grant's relay | Yes | Bob has a cloud Pro subscription. He clicks "Take Recaps home" in cloud settings → Recaps mints a fresh LIC1 token via Keysat with `expires_at = subscription_expires_at`. Bob pastes it into his self-hosted install. Self-hosted Recaps uses the token as `Authorization: Bearer LIC1-...` to the relay. |
| Alice runs Recaps for family, brings own Gemini key | No | Alice's family is invisible to Grant. |
| Alice runs Recaps for family + uses Grant's relay | Yes | Alice has a cloud subscription. Family-tenants on Alice's install all share Alice's relay-credit pool via Alice's pasted license. Alice manages per-family-member accounting locally via the existing `tenant_credits` + operator-grant flow. If Alice needs more credit headroom for her family, she upgrades her cloud sub to Max or buys credit packs. |
### What the license is for, after the simplification
**One job: the credential that authenticates a self-hosted Recaps install
against Grant's relay.**
Not "are you Pro on cloud" — Recaps DB knows that.
Not "credit pool key" — `user:<recaps_user_id>` keys cloud requests, `lic:<fp>` keys self-hosted.
Not "subscription expiry" — `users.subscription_expires_at` in Recaps DB is the source of truth.
The license is a **deliverable artifact**, minted on demand only when a
cloud Pro user explicitly clicks "Take Recaps home." Most cloud users
never click it; they don't need a license. The license's `expires_at`
mirrors the user's subscription expiry, so when their cloud sub lapses,
the self-hosted install's relay access stops working naturally.
### Grace period for self-hosted licenses
**Decided:** Keysat handles this. When a cloud subscription lapses, Keysat
can keep the license valid for a short grace period (e.g., 7 days) before
revoking. Recaps doesn't manage this — it's a Keysat-internal policy.
---
## Migration order
When ready to implement:
1. **Decide self-hosted = free + open source** ← already decided. Removes
the "is this install licensed" check from the cloud path; keeps it
only as a guard on relay access.
2. **Recaps schema additions:** `users.tier`, `users.subscription_expires_at`,
`users.zaprite_customer_id`, `users.zaprite_subscription_id`,
`users.bitcoin_renewal_token_hash` (single-use). Migrate existing users
by deriving tier + expires_at from their attached license at boot time.
3. **Zaprite webhook handler:** Recaps endpoint accepting Zaprite's order
+ subscription lifecycle events; updates user row accordingly.
4. **Relay header migration:** Cloud Recaps sends `X-Recap-User-Id` +
`X-Recap-User-Tier`. Relay accepts BOTH old (license-keyed) and new
(user-id-keyed) headers for one release as a compat window.
5. **Drop license attachment from cloud signup:** Pro/Max purchase via
Zaprite or BTCPay now updates `users.tier` + `users.subscription_expires_at`
directly. No LIC1 token attached at signup.
6. **Renewal-email pipeline:** Scheduled job in Recaps that scans for
subscriptions approaching expiry, sends the renewal email with a
one-time-use renewal token. Token consumption flow on `/renew?token=…`.
7. **Pro/Max purchase UX redesign:** Bitcoin primary + Card secondary
layout. Bitcoin path uses existing inline BTCPay flow. Card path
redirects to Zaprite hosted checkout.
8. **"Take Recaps home" rework:** Becomes an explicit user-initiated
action that mints a fresh LIC1 token via Keysat at click time, not at
signup. UI shows the token with copy-to-clipboard + install
instructions.
9. **Cancel-subscription button:** Settings → Plan → cancel. Hits Zaprite
cancel API for card subs; for Bitcoin subs there's nothing to cancel
(just let it lapse — they paid one month, they get one month).
10. **Remove "free signed-in" path from cloud:** Anon trial credits stay
as taste-test; signup grant goes away. Self-hosted is the free path
going forward.
Estimated effort: ~1 week of focused work end-to-end. Steps 25 are the
core; the rest are polish on top.
---
## Decided open questions
| Q | Decision |
|---|---|
| Annual or monthly Bitcoin subscription? | **Monthly upfront, manual renewal** with automated renewal-link emails |
| Same price across paths, or premium/discount? | **Card premium configured inside Zaprite admin**, not in Recaps code |
| Grace period on self-hosted licenses when cloud sub lapses? | **Keysat handles this** — Recaps doesn't manage |
| Free tier on cloud? | **No — cloud is paid-only**, self-hosted is the free path |
---
## What stays (don't break)
- Relay's credit ledger + tier-quota math
- BTCPay webhook → relay-pool crediting (for one-shot Lightning credit packs)
- Inline Lightning UX for credit packs (already shipped, working)
- Anon trial mechanic (cookie-tracked taste-test)
- The "Take Recaps home" feature itself (just changes when the token is minted)
- The Keysat license-issuing pipeline (just changes who calls it and when)
## What changes
- Recaps stops attaching a license at every Pro/Max signup
- Recaps gains its own subscription state (`users.tier` + `expires_at`)
- Relay accepts `X-Recap-User-Id` for cloud requests (compat with old license-keyed for self-hosted)
- Pro/Max purchase UX gets the two-option layout (Bitcoin primary, Card link)
- New renewal-email pipeline for Bitcoin subscribers
- Cancel-subscription button wired to Zaprite
- Cloud loses the "free signed-in" tier; self-hosted IS the free path
- "Take Recaps home" reshapes as on-demand mint, surfaced as an explicit
button in cloud settings
## What's deleted
- Nothing — only deprecated. Old license attachment code stays as a fallback for one release window. After migration is fully verified, can be cleaned up in a follow-up.
+175
View File
@@ -0,0 +1,175 @@
# Core Decoupling — Implementation Plan (relay-owns)
**Status:****Implemented + build-ready, 2026-06-04.** Both sides code-
complete, typecheck/syntax clean, unit tests green. Relay bumped to
`0.2.119`, Recaps app to `0.2.143`. Not yet installed/configured on-device —
see "Install + configure runbook" at the bottom.
Scoped slice of `architecture-simplification-plan.md`, with the May plan's
"Recaps-owns billing" reversed per Grant's decision.
## Decisions locked (2026-06-03)
1. **The Recap Relay owns the Pro/Max subscription**, keyed by the Recaps
**user-id** (not a Keysat license). Recaps reads each user's tier from
the relay to gate features.
2. **No credit-pool migration** — no real customers yet; clean cutover.
3. **Keysat leaves the cloud path entirely.** A cloud user has no license.
Keysat/licenses remain ONLY for the (future) self-hosted-operator case.
4. **Server auth = a shared "operator key."** The `recaps.cc` server proves
itself to the relay with a shared secret; it then vouches for its users
via `X-Recap-User-Id`.
5. **Self-serve subscription purchase is DEFERRED.** For this slice, tiers
are **operator-set** (Grant grants Pro/Max). Self-serve BTCPay/card
subscription buying + expiry + renewal = the later "payment" slice.
## Goal (one sentence)
Replace "every cloud relay request carries the user's Keysat license" with
"the `recaps.cc` server authenticates once with an operator key and passes
the user's account-id; the relay tracks that user's tier + credits."
---
## How it works after the change
```
Cloud user's browser
│ (logged-in Recaps session cookie)
recaps.cc server ──reads user's tier from relay, gates features──┐
│ POST /relay/<...> │
│ X-Recap-User-Id: <recaps user id> │
│ X-Recap-Operator-Key: <shared secret> ← proves it's │
│ (NO per-user license bearer) Grant's server│
▼ │
Recap Relay │
• validates operator key → trusts the user-id │
• credit pool keyed by user:<id> │
• stores that user's TIER (+ optional expiry) on the pool ─────┘
• applies the tier's monthly credit quota
```
Self-hosted operators are unchanged: they send a license bearer and no
`X-Recap-User-Id`, so they keep the existing `lic:`/`inst:` path.
---
## Relay changes (recap-relay/)
1. **Identity resolver** (new helper used by every route in place of
`resolveLicense(auth)` + raw installId): if `X-Recap-User-Id` is present
AND `X-Recap-Operator-Key` matches config → identity =
`{ creditKey: "user:<id>", source: "cloud" }`. Else existing license/
install path (`credits.js` `getCreditKey` / `resolveLicense`).
2. **Tier-of-record on the pool** (`credits.js` ledger row): make `tier`
(+ optional `subscription_expires_at`) an authoritative, persisted field
for `user:` pools, instead of reading tier from a license each request.
Quota math (`getTierQuotas`) keys off it as today.
3. **Operator endpoint to set a user's tier**
`POST /admin/users/:userId/tier { tier, expires_at? }` (admin-auth
gated). This is how tiers get set in this slice; the future self-serve
purchase flow writes the same field.
4. **Report tier + balance for a user-id** — extend `/relay/balance` (and
the status surface) to answer for the `user:<id>` identity so Recaps can
read it.
5. **Config:** `relay_cloud_operator_key` (+ a StartOS action to set it,
Phase 1.5).
## Recaps changes (recap/)
6. **Cloud relay identity** (`providers/index.js` `pickRelayIdentity` +
`providers/relay.js` `buildHeaders`): multi-mode cloud user → send
`X-Recap-User-Id` + `X-Recap-Operator-Key` (from server config), DROP the
user-license bearer. Single-mode / self-hosted unchanged.
7. **Entitlement checks read the relay-reported tier**
(`license-middleware.js` multi-mode branch, `tts-routes.js`
`userHasTtsAccess`): derive tier from what the relay reports for this
user (cached on `req.user` / relay-status), not from a parsed license.
Single-mode keeps using the operator `LIC`.
8. **Stop attaching a Keysat license at cloud signup**
(`license-purchase.js`): cloud accounts no longer get a license. (The
existing flow stays available for the self-hosted operator-license case
only.)
9. **Config:** `recap_relay_operator_key` (server-side; never sent to the
browser).
---
## Explicitly deferred (later "payment" slice)
Self-serve Pro/Max subscription purchase (monthly BTCPay/card, expiry,
renewal emails, cancel) · removing the free signed-in tier · "Take Recaps
home" rework. None of these block the decoupling; tiers are operator-set
until then.
## Testing / rollout
1. Relay: identity resolver — cloud (valid key)→`user:` ; cloud (bad/no
key)→reject/fallback ; self-hosted→`lic:`/`inst:`.
2. Relay: operator-set tier → `/relay/balance` reports it → metered call
decrements the `user:` pool at the right quota.
3. Recaps: feature gates (clips, subscriptions, TTS) follow the relay tier.
4. Ship relay first (accepts both old + new), then Recaps cutover. Verify a
self-hosted-style license request still works. Both via `make install` /
sideload — **no registry deploys.**
## Effort
~**23 focused days** (smaller than the migration-laden version): ~1 on the
relay (resolver, tier-on-pool, operator endpoint, config), ~1 on Recaps
(headers, tier-read, gate rewiring), ~0.5 testing.
## Sequencing (resolved 2026-06-03)
Core decoupling ships first with **operator-set tiers**; self-serve
subscription purchase is the immediate next slice. Rationale: land the
structural de-licensing on its own and verify it, then add money-handling
code on a proven foundation rather than entangling a refactor with new
payment flows. No real customers yet, so no cost to this ordering.
---
## What landed (2026-06-04)
**Relay (`recap-relay/`, → 0.2.119):** `identity.js` resolver +
`verifyOperatorKey`; `credits.js` `setUserTier`/`getUserCreditRow` +
`creditKey` threading; `job-credits.js`/`envelope.js` `creditKey`;
`routes/user-tier.js` (`POST`/`GET /relay/user-tier`, operator-key authed);
balance/tts/transcribe/analyze/transcribe-url/summarize-url use
`resolveIdentity`; `config.js` `relay_cloud_operator_key`; admin Settings
expose it as a masked **"Cloud operator key"** field (dashboard +
`PUT /admin/settings`).
**Recaps (`recap/`, → 0.2.143):** `db.js` `users.tier` column +
`migrateUsersTier`; `relay-state.js` `computeCreditKey` keys `user:<id>`;
`providers/index.js` `pickRelayIdentity` emits the cloud identity for paid
users; `providers/relay.js` sends `X-Recap-User-Id`+`X-Recap-Operator-Key`
and adds `setRelayUserTier`/`getRelayUserTier`; `license.js` `viewForTier`;
`license-middleware.js` `/api/license-status` derives the view from
`req.user.tier`; `tts-routes.js` gate reads `req.user.tier`; **3 gates that
keyed off `!keysat_license` now exclude paid-tier users** so they aren't
misrouted to the free-tenant `tenant_credits` path
(`/api/relay/status` display, `/api/process` gate+debit); `config.js` polls
`recap_relay_operator_key` into a live binding; `relay-default.js`
`getRelayOperatorKey` reads env→live-binding; StartOS **"Set Relay Operator
Key"** action + config field; operator **Tenants panel** gets a per-row
tier badge + **Tier** selector (`POST /api/admin/tenants/:id/tier`, which
writes the relay first then caches `users.tier`).
## Install + configure runbook
1. `cd recap-relay && make x86 && make install` (relay 0.2.119). **Never**
`make deploy`/`redeploy` for the relay.
2. Relay dashboard → Settings → Endpoints & credentials → **Cloud operator
key** → paste a fresh secret (`openssl rand -hex 32`). Save.
3. `cd recap && make x86 && make install` (app 0.2.143).
4. Recaps StartOS → Actions → **Set Relay Operator Key** → paste the **same**
secret. (Picked up within one config poll — no restart.)
5. Sign in as operator → **Tenants** → open a user's row → **Tier****Max**
(or Pro). This writes the relay `user:<id>` tier, then caches `users.tier`.
A 502 here means the two operator keys don't match — fix + retry.
6. As that user: confirm the **MAX/PRO badge**, the **Listen** (TTS) button,
that a summarize run is metered against the relay `user:<id>` pool (not
`tenant_credits`), and that `/api/relay/status` shows the relay balance.
7. Regression: a self-hosted-style license request still works (license/
install path untouched — additive).
+145
View File
@@ -0,0 +1,145 @@
# Daily Digest — plan
Status: **proposed** (awaiting go-ahead). Captures the design agreed with Grant on
2026-06-15. Build only after sign-off.
## Goal
An **opt-in** (off by default) daily "wake-up" email to recaps.cc users: the recaps
added to their library in the last ~24 hours, each shown as a **synthesized 12
paragraph overview** generated from that recap's existing per-topic summaries. Turns
passive subscriptions into a daily touchpoint without making the user open the app.
## Decisions (locked 2026-06-15)
- **Content** — "overnight recaps": library additions since the user's last digest.
- **Audience / opt-in** — multi-mode (recaps.cc) first; **off by default**; per-user toggle.
- **Per-episode depth** — a 12 paragraph overview *synthesized from the stored topic
summaries* (`chunks`). NOT raw full text (too long, Gmail clips >~102 KB), NOT a
one-sentence blurb (too thin). This is Grant's call and it's what bounds email size.
- **Volume** — per-episode size is bounded by the 2-paragraph synthesis. Still cap at
~10 episodes per email with an "and N more in your library →" overflow link for
extreme days.
- **Cadence** — once per user per ~24h at a fixed server-time hour (default 08:00).
Timezone-aware send is a v2. **Skip the email entirely when nothing is new.**
- **Dedup** — a per-user `last_digest_at` watermark; each digest covers recaps created
since that instant, so nothing repeats and nothing is missed.
## Data (grounded in code)
- Saved recap record (`server/history.js` `saveToHistory`): `id`, `title`, `type`,
`url`, `createdAt` (ISO), `topicCount`, `chunks` (topics, each with bullet
summaries), `entries` (transcript), `speakers`/`speakerNames`. **No top-level
summary is stored** → the 12 paragraph overview must be synthesized.
- Multi-mode users live in the `users` table (`id`, `email`, …); a user's library
scope is their user id.
## Architecture
Mirror `server/subscription-reminders.js` (the proven daily-scan-plus-email pattern:
self-gating, deduped, never throws).
- **`server/daily-digest.js`** (new)
- `runDigestScan({ force })`: gate on `isSmtpReady()` + public URL set. For each
opted-in user, list sessions with `createdAt > last_digest_at`; if none, skip. For
each new recap, get-or-generate its overview (see below), render the email,
`sendMail`, then advance the watermark. Returns a `{sent, skipped}` summary; never
throws.
- `startDigestScheduler()`: boot delay + interval, fires near the target hour.
Idempotent; safe to start unconditionally in multi mode.
- **Synthesis** — `synthesizeEpisodeOverview(record)`: send the recap's topic titles +
bullet summaries to the relay LLM with a "write a 12 paragraph overview" prompt.
**Cache** the result back onto the session JSON (e.g. `digestOverview`) so it's
generated once and could later power an in-app episode overview. **Sanitize
operator-internal strings at this boundary** (Parakeet/CUDA/LAN IPs etc. must not
reach cloud users — existing repo convention).
- **Email** — `renderDigestEmail({ brandName, episodes, manageUrl, unsubscribeUrl })`
in `server/email-template.js`, matching the existing reminder/magic-link templates.
- **Opt-in storage** — migration in `server/db.js`: add `users.digest_enabled`
(default 0) and `users.last_digest_at` (ms, nullable). Toggle endpoint in
`server/account-routes.js` (requires session). Settings-modal toggle in
`public/index.html`.
- **Unsubscribe** — a one-click tokenized GET link in every email that flips
`digest_enabled = 0` without requiring login (signed token), plus the in-app toggle.
Consent + deliverability hygiene on the young recaps.cc domain.
- **Operator test trigger** — `POST /api/admin/digest/run { test_email }`, mirroring
the reminders test hook, so it can be smoke-tested without waiting a day.
## Cost / credits
The synthesis is one small relay LLM call per new recap per opted-in user, run once and
cached. Bounded by (opted-in users × new recaps/day). **Recommend operator-absorbed**
(it's a retention feature, input is already-short topic summaries) rather than drawing
the user's credits. Confirm.
## Open questions (defaults chosen; confirm or adjust)
1. **Synthesis cost owner**~~operator-absorbed (default) vs user credits?~~
**RESOLVED 2026-06-15: operator-absorbed, zero operator action.** The synthesis
provider is built with `resolveProviderOpts("relay", { req: null })` → the operator's
install identity, the *same* relay credit pool free signed-in users' summaries already
draw from (`providers/index.js` `pickRelayIdentity`). No comped system user-id needed.
Flipping to user-billing later = pass the recipient's cloud identity at the marked line
in `daily-digest.js` `buildSynthesisProvider()`.
2. **Send hour** — 08:00 server time (default)?
3. **Single-mode operator digest** — defer to a follow-on (default: multi-mode only v1)?
4. **Relay contract**~~does an existing relay endpoint (`/relay/analyze`) fit~~
**RESOLVED 2026-06-15: `/relay/analyze` fits as-is, no new relay capability.** The
route (`recap-relay/server/routes/analyze.js`) takes a free-form `{ prompt: string }`
and returns `{ result: { text } }`; the client already wraps it as
`relay.js` `analyzeText({ prompt }) → result.text`. "Topic sections JSON" is only what
today's `chunked-analyze.js` caller asks for in *its* prompt — the endpoint is generic.
Synthesis = build a "summarize these summaries into 12 paragraphs" prompt, read
`result.text`. **No cross-repo change.** (Aside: relay `AGENTS.md:78` still describes
this endpoint as `{ transcript, … } → topic sections JSON` — stale; flag for that repo.)
Billing: each standalone analyze charges 1 credit on the call's credit key unless it
shares an `X-Recap-Job-Id` — that's the Q1 (cost-owner) mechanism, decided at phase 2.
## Build phases
1. **BUILT 2026-06-15.** Schema + opt-in toggle. `db.js`: `users.digest_enabled`
(default 0) + `users.last_digest_at` (ms, nullable) via SCHEMA_SQL +
`migrateUserDigestPrefs`. `account-routes.js`: `GET`/`POST /api/account/digest`
(enabling stamps `last_digest_at = now` so the first send isn't a backlog dump).
`public/index.html`: settings-modal toggle (`renderDigestBlock` + `loadMyDigest` /
`setDigestEnabled`, optimistic with revert).
2. **BUILT 2026-06-15.** Synthesis + cache → `server/daily-digest.js`:
`buildOverviewPrompt` (pure), `scrubOperatorStrings` (conservative backstop — infra
proper nouns + LAN/private hosts; dropped CUDA to avoid mangling legit tech content),
`synthesizeEpisodeOverview` (relay `analyzeText`, operator-absorbed identity, stable
per-episode jobId), `getOrCreateEpisodeOverview` (`digestOverview` cache + best-effort
`patchSession` write-back). NOT wired into a scheduler yet — dormant until phase 3.
Tests: `test/daily-digest.test.js` (12, pass). Note: chunks carry a `summary` text per
topic (not bullets — the Data section's "bullet summaries" wording was loose).
3. **BUILT 2026-06-15.** Email + scan + scheduler + dedup + overflow cap.
`email-template.js` `renderDigestEmail` (minimal inline style, per-episode title→source
link + overview, overflow line, one-click unsubscribe). `daily-digest.js`:
`selectDigestEpisodes` (pure: watermark filter + cap + overflow), `runDigestScan`
(hourly tick, acts at `SEND_HOUR=8`; per-user `MIN_RESEND_MS=20h` + watermark dedup;
skips empty; advances watermark only on successful send; never throws),
`startDigestScheduler`, `setupDigestRoutes` (public `GET /api/digest/unsubscribe?token=`).
`history.js` `listScopeSessions`. `db.js` adds `users.digest_unsub_token` (minted lazily
on first send). Wired in `index.js` (multi-mode) + `tenant-auth.js` public path.
4. **BUILT 2026-06-15.** `POST /api/admin/digest/run``{test_email}` sends a sample
render; bare body forces a real scan now (bypasses the hour gate, not the resend gate).
Mirrors `/api/admin/reminders/run`.
5. **DONE.** `test/daily-digest.test.js` — 19 tests (prompt, scrub, synth/cache,
`selectDigestEpisodes` watermark/cap/overflow/empty, `scopeForUser`, email render).
Full suite **138 pass**. Verified on a real multi-mode boot: migrations apply, scheduler
starts, and the unsubscribe route (400/404/200 + flips `digest_enabled`) works end-to-end.
## Status: feature-complete, awaiting on-box smoke test
Built end-to-end but **not yet installed** (no version bump). The relay synthesis call and
SMTP send can only be exercised on the operator's box. Operator smoke test:
`POST /api/admin/digest/run {test_email}` to eyeball the render; then opt in, add a recap,
and force a scan (or wait for 08:00) to see a real synthesized digest.
**Fresh-eyes review applied (2026-06-15).** Three correctness fixes after a reviewer pass:
(1) the watermark now advances to the newest *sent* recap but never past a failed/deferred
one (`nextDigestWatermark`) — the old `now` stamp silently dropped both synthesis-failures
and over-cap overflow recaps forever; (2) `force` no longer bypasses the in-progress lock,
so an operator force-run during the scheduled tick can't double-send; (3) `idx_users_unsub_token`
is created in the migration, not `SCHEMA_SQL` (the latter runs before the column exists on
upgraded DBs → would crash boot). Existing-DB upgrade verified on a realistic pre-digest
schema. Also added an index on the unauthenticated token lookup + a null-scope guard.
+19
View File
@@ -0,0 +1,19 @@
---
paths:
- server/providers/relay.js
- server/relay-capabilities.js
- server/relay-default.js
- server/billing-routes.js
- server/credits-purchase.js
- server/subscription-reminders.js
- server/config.js
---
# Client-side contract with the relay
Endpoint shapes + auth model are documented canonically in `../recap-relay/AGENTS.md`. The client side is:
- **Env vars** — `RECAP_RELAY_BASE_URL` (default `https://relay.recaps.cc`) + `RECAP_RELAY_OPERATOR_KEY` (matches the relay's `relay_cloud_operator_key`). Both gitignored; reference names, never values. Resolved in `server/relay-default.js` and `server/config.js`.
- **Auth direction (what the client SENDS)** — cloud calls send `X-Recap-Operator-Key` + `X-Recap-User-Id`; self-hosted calls send `X-Recap-Install-Id` (+ optional `Authorization: <license>`). Set the same `X-Recap-Job-Id` on transcribe + analyze in one summary so the relay bills one credit, not two.
- **Endpoints called** — `/relay/{transcribe, transcribe-url, jobs/:id, summarize-url, summarize-url/:jobId/events, analyze, tts, balance, capabilities, policy, credits/packages, credits/buy, credits/invoice/:id, user-tier, user-tier/:id, tier-invoice, tier-zaprite-order, tier-plans, expiring-subscriptions}`. Settle webhooks land on the relay side, never here.
- **Files** — `server/providers/relay.js` (transcribe/summarize/analyze/tts + balance + tier reads/writes), `server/relay-capabilities.js` (capabilities poll), `server/billing-routes.js` (tier purchase orchestration), `server/credits-purchase.js` (credit-pack purchase proxy), `server/subscription-reminders.js` (polls `/relay/expiring-subscriptions`). (`/relay/policy` is a small inline proxy in `index.js`.)
+282
View File
@@ -0,0 +1,282 @@
# Path 2B + Path 1 Interweave Plan
Companion doc to `architecture-simplification-plan.md` (Path 1) and the
chat thread that proposed Path 2A (relay-only upload, ship first).
This doc covers:
1. **Path 2B** — bringing internal-meeting analysis into the Recaps
cloud frontend as a first-class feature alongside YouTube/podcast
summaries.
2. **How Path 2B depends on Path 1** — what Path 1 unlocks vs. what
could be partially built without it.
3. **Migration path** — how Path 2A's relay-only upload data flows
forward into Path 2B's cloud-side library.
---
## Context
Recap Relay (the operator-side backend) is now generic enough to
analyze ANY audio — not just YouTube/podcasts. The download step is
the only YouTube-specific code; everything downstream (transcribe →
diarize → cluster → analyze → polish) applies cleanly to arbitrary
audio. Path 2A exposes this via a relay-admin-only upload UI so
operator Grant can run internal meeting analysis on his own hardware
TODAY without waiting on Recaps multi-tenant work.
Path 2B is the longer arc: same capability surfaced in the cloud
Recaps app, so signed-in users can submit private meeting audio,
manage it alongside their other content, and (optionally) share with
colleagues.
---
## Path 1 (recap) state
`architecture-simplification-plan.md` defines:
- One Recaps binary, two modes via `RECAP_MODE=single|multi` env var
- Magic-link + optional password auth via StartOS SMTP
- Per-user library at `/data/history/<userId>/<sessionId>.json`
- Per-user keysat license (mintable via Keysat admin API)
- BTCPay subscription + one-time credit-purchase flows
- Lite-settings UI for non-operator cloud users
- Self-hosted operator stays single-tenant by default; the .s9pk
ships free + open
Status: written but not built. The relay-side work has continued in
parallel (FIFO queue, clustering suppression, polish pass) which
strengthens the case for Path 1 — the cloud user experience benefits
from all of it, and the keysat-license layer is increasingly
friction-without-value for cloud users who already have email-verified
accounts.
---
## Path 2B — internal meetings in cloud Recaps
### What it looks like
A signed-in Recaps user gets a second submission affordance alongside
the existing "Paste a YouTube/podcast link" input:
```
┌─────────────────────────────────────────────────────────┐
│ Submit content │
│ ○ Paste a YouTube/podcast link │
│ ○ Upload audio file (private) [Choose file…] │
│ │
│ Title: [_____________________________] │
│ Participants: [_____________________________] (opt) │
│ Meeting type: [▼ default / 1:1 / all-hands / …] │
│ │
│ [ Summarize ] │
└─────────────────────────────────────────────────────────┘
```
The submission flows the same way YouTube submissions do:
Recaps-app → Relay (`/relay/v1/summarize-upload` for files,
existing `/relay/v1/summarize-url` for URLs). The relay handles both
via the same pipeline — only the input step differs (download vs.
multipart receive).
### Library + rendering
Each saved session in the cloud user's library has a `type` field:
- `"youtube"` — existing rendering (video player + topics + transcript chips)
- `"podcast"` — existing podcast rendering (audio player + topics + transcript)
- `"meeting"` — new rendering: no media player; topics + transcript chips
expandable below each topic card; PLUS a "Meeting analysis" block at
the top with Decisions / Action Items / Open Questions / Key Quotes
(the structured-extras pass from Phase 2 of Path 2A)
Library list view shows a small icon distinguishing meeting items so
the user can filter at a glance.
### Privacy + sharing
Meetings are PRIVATE by default — visible only to the submitting user.
Two later features:
- **Share with team** — a meeting can be optionally shared with N
named cloud-Recaps users (each looks up by email). Other users see
it in a "Shared with me" library section.
- **Export to markdown / PDF** — already exists for YouTube content;
add the meeting-extras blocks to the export.
### Audio handling
- Uploaded audio goes from browser → Recaps-app (Node Express,
multipart middleware) → Recap Relay (forward as multipart to
`/relay/v1/summarize-upload`).
- Recaps-app's tmp file is deleted immediately after the relay
acknowledges receipt.
- Relay's tmp file is deleted after the pipeline completes.
- NEITHER side keeps the audio. The TRANSCRIPT + analysis are saved.
If the user wants to re-process (different prompt set), they'd
re-upload the audio.
- The transcript stays in the user's per-user library at
`/data/history/<userId>/<sessionId>.json`, scoped + isolated.
---
## Path 2B's hard prerequisite: Path 1
Path 2B fundamentally requires multi-tenant auth in Recaps. Without
Path 1:
- No per-user library separation
- No way to know whether the meeting audio is private to a user vs.
visible to everyone running this Recaps instance
- No way to share with named other users
- No way to bill upload-heavy users differently from URL-only users
(uploads might warrant a different price tier given the storage
cost)
You COULD build a stripped-down Path 2B on single-tenant Recaps —
operator uploads audio, it's saved to the operator's library, no
sharing. But that's roughly equivalent to Path 2A with a fancier UI,
just on the wrong side of the codebase split (Recaps-app vs. relay).
Not worth the duplication.
So: **ship Path 2A as the immediate beachhead, do Path 1 next, then
Path 2B on top.**
### What Path 1 unlocks (relevant to 2B)
The Path 1 doc already covers most of what 2B needs:
- Per-user library at `/data/history/<userId>/*.json`
- Auth-aware request scoping (`req.userId`)
- Per-user keysat license OR the simplified user-id + tier headers
- BTCPay subscription tracking (for billing upload-heavy use)
Path 1's "Lite-settings panel for cloud users" already imagines a
post-auth Recaps UI without operator config noise. The submission
input would extend in 2B to add the upload option.
The relay-side header migration Path 1 proposes (`X-Recap-User-Id` +
`X-Recap-User-Tier` replacing the bearer license token) is also
beneficial for 2B — uploads from a single user with N concurrent
browsers all carry the same user-id, so credit accounting is per-user
not per-install.
---
## How Path 2A's data flows into Path 2B
When Path 2A ships (relay-only upload, results saved to
`/data/internal-meetings/<id>.json` on the relay), those summaries
are tied to the OPERATOR (Grant) — there's no cloud-user concept yet.
When Path 2B lands:
1. **Migration script** — walks `/data/internal-meetings/*.json` and
re-homes each entry under the operator's cloud user account
(`/data/history/owner/*.json` initially, then `/data/history/
<operator-user-id>/*.json` after Path 1's owner→admin rename).
2. **Same JSON shape** — Path 2A should save in a shape compatible
with Path 2B's expected library shape (chunks + entries + speakers
+ meeting-extras). One way to guarantee this: design the Phase 1
save shape now to match what Recaps' `saveToHistory` produces for
`type=meeting`, even though no Recaps UI consumes it yet.
3. **No re-processing needed** — transcripts and analysis transfer
verbatim. The user just sees them appear in their cloud library.
The relay-side upload endpoint (`/relay/v1/summarize-upload`) is the
same in both worlds. Path 2A calls it via the operator dashboard's
admin auth; Path 2B calls it via Recaps-app server proxying a
signed-in cloud user's POST.
So the relay code path is built once and serves both.
---
## Operator-editable prompt sets (Phase 3 in 2A; carries to 2B)
The "meeting type" dropdown is operator-editable. Each set has:
- Name (e.g. "1:1 with direct report")
- Topic-analysis prompt template (replaces the YouTube/podcast version)
- Meeting-extras prompt template (Decisions / Action Items / ...)
- Optional metadata schema overrides (e.g. "this meeting type always
expects 2 participants")
Stored in `relay_meeting_prompt_sets_json` config field. Operator
edits via the dashboard (similar to existing prompts panel). Cloud
Recaps users pick a set at submission time; the relay applies it
during analyze + polish.
Default sets ship built-in:
- `default` — neutral meeting prompt, all sections enabled
- `1on1` — emphasizes Action Items + Open Questions; light on Decisions
- `all-hands` — emphasizes Decisions + Key Quotes; less actionable
- `customer-interview` — emphasizes Key Quotes + Open Questions; light
on Decisions
- `standup` — short-form; Action Items + Open Questions only
---
## Suggested order of operations
1. **Path 2A Phase 1** — relay-only upload, no extras, basic
topic+transcript rendering. ~2-3 days.
2. **Path 2A Phase 2** — meeting-extras analysis pass (Decisions /
Action Items / Open Questions / Key Quotes). ~1-2 days.
3. **Path 2A Phase 3** — prompt sets dropdown. ~1 day.
4. (Use Path 2A in production for some weeks; gather feedback on
prompts, output quality, UX.)
5. **Path 1** — multi-tenant Recaps. ~3-4 weeks per the existing
architecture-simplification doc, modulo amendments.
6. **Path 2B** — surface internal meetings in cloud Recaps. ~1.5-2
weeks given Path 1 has shipped. Migrates the Path 2A artifacts
into the new per-user library.
Total wall time: ~6-8 weeks for the full arc. Path 2A capability
available to operator after step 1 (~2-3 days). Cloud users get
meetings after step 6.
---
## Open questions
These should be settled BEFORE Path 2B build starts:
1. **Pricing for uploads.** Does an upload count the same as a URL
submission against a user's monthly credit cap? Or is it priced
differently to reflect the upload bandwidth + storage cost? My
default: same price (1 credit per submission) — bandwidth cost is
trivial, storage is just JSON.
2. **Audio retention.** Default: never retain. Optional per-user
setting "keep audio for 7/30/90 days so I can re-process with a
different prompt set"? Adds operator storage cost; only worth it
if users actually want it.
3. **Sharing model.** Path 2B Phase 1 = no sharing, private only.
Phase 2 = shared with N named users. Phase 3 = optional public
share URL. Each phase adds auth/permission complexity. Worth
designing the data model now (`{ ownerId, sharedWith: [userId] }`)
even if only ownerId is populated in Phase 1.
4. **Speaker name persistence.** If a user identifies "Speaker_A" as
"Matt Hill" in one meeting, should that name auto-suggest in the
next meeting if a fingerprint match is found? Requires storing
fingerprints per-user across meetings. Big privacy + product
decision. My instinct: opt-in toggle in user settings, default
off.
5. **Meeting type defaults.** Should Recaps' submission flow have a
default meeting type, or force the user to pick? My instinct:
default to "default" set; let users pick if they want.
---
## Decision points for Grant
- Confirm the phasing above (2A → 1 → 2B) is the right order
- Pre-commit to the "uploaded audio is never retained" default — it's
the privacy-safer choice and aligns with how YouTube downloads
already work
- Pick a side on the open questions above before Path 2B starts, OR
defer them as Phase 2/3 of Path 2B
+131
View File
@@ -0,0 +1,131 @@
# Per-Tenant Subscriptions — Implementation Plan
**Status:****ALL STEPS DONE (app 0.2.149, 2026-06-04).** Per-tenant
subscriptions are live: every Pro/Max user gets their own subscriptions +
auto-queue, processed under their own account. The gate is flipped to
tier-based. Offline-verified (99 server tests incl. the ephemeral-session
mechanism + both-mode boot smokes); the actual processing-as-owner run is
the on-device test (see checklist below).
**Step 4 (the hard part), as built:** the background processor now finds
approved items across every scope (`listAutoQueueScopes`) and processes each
AS its owner — `processItemInternally(item, scope)` mints a short-lived real
session (`mintInternalSession` in auth-routes.js) for the owning user, sends
it as the `recap_session` cookie on the loopback `/api/process` call, and
deletes it on every exit path. No auth bypass — a bad/expired token just
401s and the item is marked failed. Single mode sends no cookie (resolves to
"owner"). `userIdForScope`: single→null; multi "owner"→admin; tenant→the
user id. Bonus: this also fixes the operator's OWN multi-mode auto-processing,
which previously ran the loopback with no identity.
**Gate flip:** `PRO_FEATURE_GATES` subscriptions gate is now tier-based in
multi mode (Pro/Max/admin pass, free → 402); frontend `canUseSubscriptions()
= hasEntitlement("subscriptions")` (operator-only clauses reverted).
## Landed in 0.2.147
- `server/subscriptions.js` — scope-keyed storage for subscriptions / skip /
seen / auto-queue + file-locked `mutateAutoQueue` (atomic read-modify-write,
replacing the global in-memory `autoQueue`), `listSubscriptionScopes()`,
`migrateGlobalSubscriptionsToOwner()`, plus the dedup (`getProcessedVideoIds`,
`isKnownVideo`).
- `index.js` — the check loop fans out over `listSubscriptionScopes()` into a
per-scope `checkScopeSubscriptions(scope)`; every endpoint resolves
`scope = subScope(req)` (= scopeForRequest, "owner" for the operator); the
processor + boot recovery use `mutateAutoQueue`; boot runs the migration +
per-scope library reconcile. Behind the gate every scope resolves to
"owner", so behaviour is unchanged for the operator — the plumbing is just
per-scope now.
- `history.js``addToSkipList(scope, videoId)` is scope-keyed.
---
## Goal
Each signed-in Pro/Max tenant manages their **own** channel/podcast
subscriptions; discovered episodes land in **their** auto-queue; approving
one summarizes it under **their** account (their credits, their library,
their relay identity). The operator (admin) keeps theirs. No tenant sees or
affects another's.
## What already exists (foundation)
- `server/subscriptions.js` — extracted, unit-tested. `getProcessedVideoIds(scope)`
(scope-aware library scan — the dedup fix) + `isKnownVideo()` (pure
predicate). Already takes a `scope`, so per-tenant dedup is free.
- `history.js``scopeForRequest(req)` (→ "owner" for admin/single, else
`safeComponent(user.id)`), `getScopeHistoryDir(scope)`, `scopeDir`,
`renameScopeDir`. The whole per-scope filesystem layer is in place.
- The interim isolation gate: `PRO_FEATURE_GATES[subscriptions].adminOnlyInMulti`
(server) + `canUseSubscriptions()` (frontend). Both get relaxed here.
## The work
### 1. Scope the storage (mechanical, testable)
Move the four global files into `subscriptions.js`, each keyed by scope and
rooted at `scopeDir(scope)` instead of the history root:
`subscriptions.json`, `auto-queue.json`, `skip-list.json`, `seen-list.json`.
Add `loadSubscriptions(scope)` / `saveSubscriptions(scope, …)` + the skip /
seen / auto-queue equivalents. **Drop the global in-memory `autoQueue`**
load/save per scope per request (the in-memory cache is what makes the
current code single-tenant). Unit-test the round-trips per scope.
### 2. Rescope the endpoints (mechanical)
All ~15 `/api/subscriptions*` + `/api/auto-queue*` handlers derive
`scope = scopeForRequest(req)` and read/write that scope's files. Relax the
gate: drop `adminOnlyInMulti`; gate on the `subscriptions` entitlement
(tier) instead, so paid tenants get in. Update `canUseSubscriptions()` to
`hasEntitlement("subscriptions")` (drop the `!isMulti||isAdmin` clause) and
revert the operator-only frontend branches.
### 3. Rescope the check loop (moderate)
`_checkSubscriptionsInner()` becomes per-scope. Enumerate scopes that have a
subscriptions file (readdir `history/`, keep subdirs whose `subscriptions.json`
is non-empty; plus "owner"). For each: load that scope's subs, dedup via
`getProcessedVideoIds(scope)` + the scope's skip/seen/queue, append to that
scope's auto-queue. The boot + hourly timers iterate all scopes.
### 4. Fix the processor identity (the risky bit — needs on-device test)
`backgroundProcessor()` + `processItemInternally(item)` must process each
item **as its owner**. The item already can carry `scope`/`ownerUserId`.
**Recommended: ephemeral session.** Before the loopback request, mint a
short-lived row in the existing `sessions` table for the owner, send its
token as the `Cookie`, then delete the row in a `finally`. tenant-auth then
resolves `req.user` = owner normally → correct scope, credits, relay
identity. Reuses real auth (no new trust path). **Failure mode is safe:** a
bad/missing session just makes the internal request 401 → the item is
marked `failed`, never an auth hole. Single mode unchanged (no auth, scope
already "owner").
*Alternative:* extract the core of the `/api/process` handler into a
function callable in-process with an explicit `{scope, identity}` (no HTTP,
no cookie). Cleaner long-term but a large refactor of a big handler — more
risk of behavioral drift. Prefer the ephemeral session first.
The single global processor loop stays fine — it just pulls approved items
across all scopes (each item knows its owner) and processes sequentially
with the existing inter-item delay.
### 5. Migration (one-time, on boot)
Move the existing history-root files
(`subscriptions.json`/`auto-queue.json`/`skip-list.json`/`seen-list.json`)
into `history/owner/` (the admin's scope) once, if present and the target
doesn't exist. Preserves the operator's current subscriptions under their
own scope. Idempotent.
## On-device test checklist (gates "done")
1. Tenant A subscribes to a channel; tenant B sees **nothing** of A's.
2. A's discovered episode appears only in A's queue; approving it
summarizes under **A** (A's credits decremented, saved to A's library,
A's relay `user:<id>` pool billed).
3. Operator's own subscriptions still discover + auto-process.
4. Per-scope dedup: a video already in A's library is not re-queued for A;
the same video can still be independently queued for B.
5. Crash-recovery: items stuck `processing` resume under the right owner.
6. Single mode unaffected.
## Effort / risk
Steps 13 + 5: ~half a day, low risk, unit-testable offline. Step 4: the
real work — small code, but auth/billing-adjacent, so it carries the
testing burden. Land 13 behind the existing operator-only gate first
(no behavior change for tenants), verify, then flip the gate + ship 4.
+94
View File
@@ -0,0 +1,94 @@
# Self-Serve Pro/Max Purchase — Implementation Plan
**Status:** Phases 14 BUILT + INSTALLED + LIVE on immense-voyage.local
(relay 0.2.121, app 0.2.153) as of 2026-06-08. Bitcoin rail end-to-end test
pending; card rail awaits operator Zaprite config. Phase 5 (SMTP expiry
reminders) remains. Follows the core decoupling (the relay owns tiers,
keyed by user-id) — this lets users buy their own Pro/Max instead of the
operator granting them by hand.
**Phase 4 (cards via Zaprite) — DONE.** Symmetric with the Bitcoin rail
(one-time prepaid checkout, NOT Zaprite recurring): relay `zaprite-client.js`
(POST /v1/orders + GET /v1/orders/:id), `POST /relay/tier-zaprite-order`
(operator-keyed), `POST /relay/zaprite/webhook` (re-fetch-to-verify — no
signature needed; both rails land at extendUserTier). Config:
`relay_zaprite_{api_key,base_url,currency}` + `relay_tier_prices_fiat_cents_json`
(default $21/$42), set via the new "Set Zaprite Connection" StartOS action.
`/api/billing/buy` gained `method:"card"`; tier-plans reports `card_available`
+ fiat prices; UI shows "Pay by card · $21" only when Zaprite is configured.
Operator TODO: run "Set Zaprite Connection" (paste API key) + register the
webhook at `https://<relay-host>/relay/zaprite/webhook` in Zaprite.
## Decisions (from Grant)
- **Prepaid periods, NOT auto-recurring.** A user pays for a fixed period
(default **30 days**) of Pro/Max. Near expiry they get an email reminder
and pay again to extend. At expiry, the tier drops to Core. No stored
payment method, no card-vault / dunning.
- **Two payment rails:**
- **Bitcoin / Lightning via BTCPay** — the *preferred* path. Relay already
has `btcpay-client.js` (createInvoice + webhook HMAC) used for credit
purchases; extend it for tier purchases.
- **Cards via Zaprite** — secondary. Grant has a Zaprite org + API +
webhooks. New integration.
- **UI:** the main pill button is **"Pay with Bitcoin"** (opens a BTCPay
invoice). Directly below it, a smaller **"Pay by card"** link (opens a
Zaprite checkout).
- **Expiry reminders via email.** Set up SMTP (Amazon SES per the Start9
recommendation) and send an automated "your Pro/Max expires in N days"
email. Recaps already has an SMTP transport (magic-link emails).
- **Prices:** from the relay's `relay_tier_prices_usd_json` (today Pro $5 /
Max $15 per period), USD-denominated, paid in sats for BTCPay.
## Phases (each shippable on its own)
### Phase 1 — Relay: prepaid tier model + expiry enforcement (foundation)
Rail-agnostic. Everything else depends on it.
- `setUserTier({userId, tier, periodDays})` → set tier, set
`subscription_expires_at`. **Extend from the current expiry** if the user
is already active (so paying early adds time, doesn't reset it).
- **Enforce expiry:** when the relay resolves a user's tier (identityTier /
the metered-route gate), treat `subscription_expires_at < now` as Core.
Add a lazy check + a periodic sweep so expired users actually drop.
- Keep the operator-grant path (`/relay/user-tier`) working — it's the comp
tool. A manual grant can set no-expiry (operator comp) vs a purchase sets
a dated period.
### Phase 2 — Bitcoin/Lightning purchase (BTCPay)
- Relay: `POST /relay/tier-invoice` (operator-key authed) — body
`{user_id, tier, period_days}``createInvoice` with metadata
`{kind:"tier_subscription", userId, tier, periodDays}` → returns the
checkout URL + invoice id.
- Relay webhook: on a settled `tier_subscription` invoice → `setUserTier`
(extend by periodDays). (Mirrors the existing credit-purchase webhook
branch.)
- Recaps: a `pending_purchases`-style record + the settle→poll→cache-tier
loop (reuse the credit-purchase machinery). On settle, refresh
`/api/license-status` so the badge flips to Pro/Max.
### Phase 3 — Purchase UI
- Tier picker (Pro / Max, price, "30 days") with the **"Pay with Bitcoin"**
pill + **"Pay by card"** link. Reuse / replace the existing buy modal.
- Bitcoin → opens the BTCPay checkout (Phase 2). Card → opens Zaprite
(Phase 4).
### Phase 4 — Cards via Zaprite
- Relay (or Recaps): create a Zaprite checkout for the tier (Grant's org +
API), metadata carrying `{userId, tier, periodDays}`.
- Zaprite webhook → verify signature → `setUserTier` (extend). Same landing
point as the BTCPay webhook.
- Wire the "Pay by card" link to it.
### Phase 5 — Expiry reminder emails
- SMTP via SES (Grant sets up SES + StartOS System SMTP; Recaps' transport
already exists).
- Periodic job: find users with `subscription_expires_at` in ~N days, email
a "renew" notice with a link back to the purchase UI. Idempotent (don't
double-send).
## Notes / open defaults (sensible unless Grant says otherwise)
- Period = 30 days. Grace = none beyond the advance email (downgrade on
expiry). Extend-from-current-expiry on early renewal.
- Relay stays **`make install` only** (private — never registry-deploy).
- The operator-key path authenticates Recaps→relay for invoice creation, the
same as the tier-grant flow.
+412
View File
@@ -0,0 +1,412 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
<title>Sign in to Recaps</title>
<!-- Match index.html's PWA setup so a user installed via the
Add-to-Home-Screen flow can land on /auth.html (signed-out)
without losing the standalone display + theme colors. -->
<link rel="manifest" href="/manifest.json">
<meta name="theme-color" content="#0a0e1a">
<link rel="apple-touch-icon" href="/assets/icon.png">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<meta name="apple-mobile-web-app-title" content="Recaps">
<meta name="mobile-web-app-capable" content="yes">
<!-- Social preview tags — same as index.html so any shared
/auth.html link previews cleanly too. -->
<meta property="og:type" content="website">
<meta property="og:site_name" content="Recaps">
<meta property="og:title" content="Sign in to Recaps">
<meta property="og:description" content="Summarize any YouTube video or podcast episode into topic-level summaries with timestamps.">
<meta property="og:url" content="https://recaps.cc/auth.html">
<meta property="og:image" content="https://recaps.cc/assets/icon.png">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Sign in to Recaps">
<meta name="twitter:image" content="https://recaps.cc/assets/icon.png">
<link rel="icon" type="image/png" href="/assets/icon.png">
<style>
/* Design tokens (subset used by this page) — mirror of design/tokens.tokens.json.
This is a standalone document, so it carries its own copy; keep it in sync with
the canonical :root in index.html. Accent is indigo, not the old blue. */
:root {
--bg: #0a0e1a;
--surface: #111827;
--border: #1e293b;
--accent: #818cf8;
--accent-hover: #a5b4fc;
--text: #e2e8f0;
--text-strong: #f1f5f9;
--text-body: #cbd5e1;
--text-muted: #94a3b8;
--text-label: #64748b;
--text-faint: #475569;
--error-soft: #fca5a5;
--success-text: #4ade80;
}
* { box-sizing: border-box; margin: 0; padding: 0; }
html, body { height: 100%; }
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
background: var(--bg);
color: var(--text);
display: flex;
align-items: center;
justify-content: center;
padding: 24px;
}
.card {
width: 100%;
max-width: 420px;
background: var(--surface);
border: 1px solid var(--border);
border-radius: 12px;
padding: 32px 28px;
}
.logo {
display: flex;
align-items: center;
gap: 12px;
margin-bottom: 28px;
}
.logo img { width: 32px; height: 32px; border-radius: 6px; }
.logo span { font-size: 18px; font-weight: 600; color: var(--text-strong); }
h1 {
font-size: 22px;
font-weight: 600;
color: var(--text-strong);
margin-bottom: 8px;
}
p.lede {
font-size: 14px;
line-height: 1.55;
color: var(--text-muted);
margin-bottom: 24px;
}
label {
display: block;
font-size: 13px;
font-weight: 500;
color: var(--text-body);
margin-bottom: 8px;
}
input[type=email],
input[type=password] {
width: 100%;
background: var(--bg);
border: 1px solid var(--border);
border-radius: 8px;
padding: 12px 14px;
font-size: 16px;
color: var(--text-strong);
font-family: inherit;
outline: none;
transition: border-color 0.15s ease;
/* Browsers auto-fill password fields with their own bright
background; -webkit-text-fill-color + a long inset shadow
override that so the field stays on-brand. */
-webkit-text-fill-color: var(--text-strong);
-webkit-box-shadow: 0 0 0 1000px var(--bg) inset;
caret-color: var(--text-strong);
}
input[type=email]:focus,
input[type=password]:focus { border-color: var(--accent); }
input[type=email]::placeholder,
input[type=password]::placeholder { color: var(--text-faint); }
button {
width: 100%;
margin-top: 16px;
background: var(--accent);
color: white;
border: none;
border-radius: 8px;
padding: 12px 14px;
font-size: 16px;
font-weight: 600;
cursor: pointer;
transition: background 0.15s ease;
font-family: inherit;
}
button:hover:not(:disabled) { background: var(--accent-hover); }
button:disabled { background: var(--border); cursor: not-allowed; opacity: 0.6; }
.feedback {
margin-top: 20px;
padding: 14px 16px;
border-radius: 8px;
font-size: 14px;
line-height: 1.5;
display: none;
}
.feedback.success {
background: rgba(34, 197, 94, 0.1);
border: 1px solid rgba(34, 197, 94, 0.3);
color: var(--success-text);
display: block;
}
.feedback.error {
background: rgba(239, 68, 68, 0.1);
border: 1px solid rgba(239, 68, 68, 0.3);
color: var(--error-soft);
display: block;
}
.footer {
margin-top: 28px;
font-size: 12px;
color: var(--text-label);
text-align: center;
line-height: 1.5;
}
.footer a { color: var(--text-muted); text-decoration: none; }
.footer a:hover { color: var(--text-body); }
/* Password group hidden by default — most users want the magic
link and the optional-password field cluttered the form. The
"Use password instead" link below the submit button reveals
this when needed. */
.password-group[hidden] { display: none; }
.toggle-pwd-row {
text-align: center;
margin-top: 14px;
font-size: 12px;
color: var(--text-label);
}
.toggle-pwd-row a {
color: var(--text-muted);
text-decoration: underline;
cursor: pointer;
}
.toggle-pwd-row a:hover { color: var(--text-body); }
</style>
</head>
<body>
<div class="card">
<div class="logo">
<img src="/assets/icon.png" alt="Recaps" onerror="this.style.display='none'">
<span>Recaps</span>
</div>
<h1 id="auth-heading">Sign in</h1>
<p class="lede" id="auth-lede">
Enter your email — we'll send a sign-in link.
</p>
<form id="signin-form" autocomplete="on">
<label for="email">Email</label>
<input
type="email"
id="email"
name="email"
required
autocomplete="email"
/>
<div class="password-group" id="password-group" hidden>
<label for="password" style="margin-top:12px;">Password</label>
<input
type="password"
id="password"
name="password"
autocomplete="current-password"
minlength="8"
/>
</div>
<button type="submit" id="submit-btn">Send sign-in link</button>
</form>
<div class="toggle-pwd-row" id="toggle-pwd-row">
<a id="toggle-pwd" role="button" tabindex="0">Use password instead</a>
</div>
<div class="feedback" id="feedback"></div>
<div class="footer">
First time here? We'll create your account when you click the sign-in link.
</div>
</div>
<script>
const form = document.getElementById('signin-form');
const btn = document.getElementById('submit-btn');
const feedback = document.getElementById('feedback');
const emailInput = document.getElementById('email');
const passwordInput = document.getElementById('password');
const passwordGroup = document.getElementById('password-group');
const togglePwd = document.getElementById('toggle-pwd');
const togglePwdRow = document.getElementById('toggle-pwd-row');
const heading = document.getElementById('auth-heading');
const lede = document.getElementById('auth-lede');
// Reveal the password field on demand. Most users sign in via
// magic link so the password field is clutter by default; this
// toggle surfaces it when needed. Once revealed it stays open
// for the rest of the session — no "hide again" affordance
// because re-hiding after typing would lose state and confuse.
function revealPasswordField() {
passwordGroup.hidden = false;
togglePwdRow.style.display = 'none';
// Defer focus to give the browser a tick to lay out the field.
setTimeout(() => passwordInput.focus(), 0);
updateBtnLabel();
}
togglePwd.addEventListener('click', (e) => {
e.preventDefault();
revealPasswordField();
});
togglePwd.addEventListener('keydown', (e) => {
if (e.key === 'Enter' || e.key === ' ') {
e.preventDefault();
revealPasswordField();
}
});
// ?intent=signup vs ?intent=signin — same form, slightly different
// copy. Signup leans "we'll create your account when you click
// the link"; signin leans "welcome back". If neither param is
// present we default to the generic Sign in copy (current
// behavior).
(function tailorIntent() {
try {
const intent = new URLSearchParams(location.search).get('intent');
if (intent === 'signup') {
document.title = 'Create your Recaps account';
heading.textContent = 'Create your account';
lede.textContent = "Enter your email — we'll send a sign-in link and create your account when you click it. No password to set up.";
} else if (intent === 'signin') {
document.title = 'Sign in to Recaps';
heading.textContent = 'Sign in';
// lede stays as the default
}
} catch {}
})();
function setFeedback(msg, kind) {
feedback.textContent = msg;
feedback.className = 'feedback ' + kind;
}
function clearFeedback() {
feedback.textContent = '';
feedback.className = 'feedback';
}
// Submit button label tracks whether the password field is in
// play. Hidden + empty = "Send sign-in link" (the dominant case);
// revealed + filled = "Sign in"; revealed + empty falls back to
// "Send sign-in link" so the user can change their mind without
// re-toggling the field.
function updateBtnLabel() {
if (btn.disabled) return;
const usingPwd = !passwordGroup.hidden && passwordInput.value.length > 0;
btn.textContent = usingPwd ? 'Sign in' : 'Send sign-in link';
}
passwordInput.addEventListener('input', updateBtnLabel);
updateBtnLabel();
async function signInWithPassword(email, password) {
const res = await fetch('/auth/signin-password', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ email, password }),
});
if (res.ok) {
// Cookie set server-side. Land on the main app.
window.location.href = '/';
return;
}
if (res.status === 429) {
const body = await res.json().catch(() => ({}));
setFeedback(body.message || 'Too many attempts. Try again later.', 'error');
} else if (res.status === 401) {
setFeedback('Email or password is wrong. Leave the password blank to receive a sign-in link instead.', 'error');
} else {
setFeedback('Something went wrong. Please try again.', 'error');
}
}
form.addEventListener('submit', async (e) => {
e.preventDefault();
clearFeedback();
const email = emailInput.value.trim();
// Only honor the password value if the user explicitly opened
// the password section. Otherwise a browser autofill into the
// hidden field would force the form into password-sign-in mode
// against the user's intent.
const password = passwordGroup.hidden ? '' : passwordInput.value;
if (!email) return;
btn.disabled = true;
const usingPassword = password.length > 0;
btn.textContent = usingPassword ? 'Signing in...' : 'Sending...';
if (usingPassword) {
try {
await signInWithPassword(email, password);
} catch (err) {
setFeedback('Network error. Check your connection and try again.', 'error');
} finally {
btn.disabled = false;
updateBtnLabel();
}
return;
}
// Silent retry on transport failures — iOS Safari can dispatch a
// POST onto a pooled keep-alive socket the server (or a proxy in
// front of it) has already closed. Unlike a GET, a non-idempotent
// POST isn't transparently re-sent on a fresh connection; it
// surfaces a "Load failed" TypeError instead. A single quick retry
// tends to reuse the same dead socket and fail again (the reported
// "first tap errors, second works"), so retry a few times with
// growing backoff to outlast Safari evicting the socket. Server
// errors (4xx/5xx) are returned as-is and never retried — they're
// deliberate responses, not transport flakes.
const reqInit = {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ email }),
};
async function postWithRetry() {
const backoffsMs = [400, 1200];
for (let attempt = 0; ; attempt++) {
try {
return await fetch('/auth/request-link', reqInit);
} catch (e) {
if (attempt >= backoffsMs.length) throw e;
await new Promise((r) => setTimeout(r, backoffsMs[attempt]));
}
}
}
try {
const res = await postWithRetry();
if (res.status === 429) {
const body = await res.json().catch(() => ({}));
setFeedback(body.message || 'Too many requests. Try again later.', 'error');
} else if (res.status === 503) {
const body = await res.json().catch(() => ({}));
setFeedback(
body.message || 'Sign-in is temporarily unavailable.',
'error',
);
} else if (!res.ok) {
setFeedback('Something went wrong. Please try again.', 'error');
} else {
setFeedback(
'Check your email — we sent a sign-in link to ' + email + '. It expires in 15 minutes.',
'success',
);
form.reset();
}
} catch (err) {
setFeedback('Network error. Check your connection and try again.', 'error');
} finally {
btn.disabled = false;
updateBtnLabel();
}
});
</script>
</body>
</html>
+9106 -532
View File
File diff suppressed because it is too large Load Diff
+32
View File
@@ -0,0 +1,32 @@
{
"name": "Recap",
"short_name": "Recap",
"description": "Summarize YouTube videos and podcasts. Paste a link, get topic-level summaries with timestamps.",
"start_url": "/",
"scope": "/",
"display": "standalone",
"orientation": "any",
"background_color": "#0a0e1a",
"theme_color": "#0a0e1a",
"icons": [
{
"src": "/assets/icon.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "any"
},
{
"src": "/assets/icon.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "any"
},
{
"src": "/assets/icon.png",
"sizes": "1024x1024",
"type": "image/png",
"purpose": "any maskable"
}
],
"categories": ["productivity", "utilities"]
}
+338
View File
@@ -0,0 +1,338 @@
// Per-user "my account" endpoints for multi-tenant Recap. Distinct
// from admin-routes.js — these are scoped to the currently signed-in
// user (req.user), not to whichever tenant id you pass in the URL.
// Used by the lite settings panel to render "Active sessions" and
// "Sign out everywhere" actions for a tenant managing their own
// account.
//
// Endpoints:
// GET /api/account/sessions — my active sessions
// DELETE /api/account/sessions/:sessionId — revoke a specific session of mine
// POST /api/account/sessions/revoke-others — revoke everything BUT the current session
//
// Multi-mode only. Single mode never mounts these — the synthetic
// "owner" user has no session table to manage.
import { getDb } from "./db.js";
import { requireUser } from "./tenant-auth.js";
import { hashPassword, validatePasswordPolicy } from "./auth-routes.js";
import fs from "fs/promises";
import path from "path";
import { getHistoryDir } from "./history.js";
export function setupAccountRoutes(app) {
// ── My license key (for the "Take Recap home" flow) ────────────────
// The Pro/Max license is a bearer credential — anyone with the LIC1-
// string can present it to the relay. We only return the CALLING
// user's own key (gated by req.user.id), never anyone else's. The key
// is stored on users.keysat_license; the parsed entitlement state
// already comes via /api/license-status. This endpoint is the one
// place the raw string is exposed — used by the lite settings panel
// to show a copy-to-clipboard "Take Recap home" block for paid
// tenants who want to also run Recap on their own StartOS server.
app.get("/api/account/license-key", requireUser, (req, res) => {
if (!req.user || !req.user.id) {
return res.status(403).json({ error: "no_user" });
}
try {
const row = getDb()
.prepare("SELECT keysat_license FROM users WHERE id = ?")
.get(req.user.id);
const key = (row?.keysat_license || "").trim();
if (!key) {
return res.status(404).json({ error: "no_license" });
}
res.json({ license_key: key });
} catch (err) {
console.error("[account] license-key lookup failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
// List MY active sessions (the device list).
app.get("/api/account/sessions", requireUser, (req, res) => {
// Trial users (req.userId starts with "anon:") don't have a row in
// the sessions table — they're tracked via anon_trials. Bail with
// an empty list so the UI doesn't error.
if (!req.user || !req.user.id) {
return res.json({ sessions: [], current_session_id: null });
}
try {
const rows = getDb()
.prepare(
`SELECT id, created_at, expires_at, last_used_at, user_agent, ip_address
FROM sessions
WHERE user_id = ? AND expires_at > ?
ORDER BY last_used_at DESC, created_at DESC`,
)
.all(req.user.id, Date.now());
res.json({
sessions: rows,
// Tell the UI which row is the CURRENT session so it can
// disable the "Revoke" button for that one (sign out instead).
current_session_id: req.session?.id || null,
});
} catch (err) {
console.error("[account] sessions list failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
// Revoke a single session of mine. The session id MUST belong to me
// (we WHERE clause both id AND user_id). Otherwise a tenant could
// delete someone else's session by guessing the id.
app.delete(
"/api/account/sessions/:sessionId",
requireUser,
(req, res) => {
if (!req.user || !req.user.id) {
return res.status(400).json({ error: "trial_has_no_sessions" });
}
const sessionId = req.params.sessionId;
try {
const result = getDb()
.prepare("DELETE FROM sessions WHERE id = ? AND user_id = ?")
.run(sessionId, req.user.id);
if (result.changes === 0) {
return res.status(404).json({ error: "session_not_found" });
}
res.json({ ok: true });
} catch (err) {
console.error("[account] revoke session failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── Set / clear my password ────────────────────────────────────────
// Magic-link is the primary auth. Setting a password is optional —
// makes returning sign-in faster. Set and "change" use the same
// endpoint (overwrite); the reset flow is "sign in via magic link,
// then call this endpoint with the new password."
//
// POST /api/account/password { password } — set or overwrite
// DELETE /api/account/password — clear / revert to
// magic-link-only
//
// Lives in /api/account/* (not /auth/*) because both require an
// active session and need the tenant-auth middleware to attach
// req.user — /auth/* is the public-path namespace bypassed by the
// middleware so unauthenticated visitors can request links / verify.
app.post("/api/account/password", requireUser, (req, res) => {
if (!req.user || !req.user.id) {
return res.status(401).json({ error: "auth_required" });
}
const password = req.body?.password;
const policyErr = validatePasswordPolicy(password);
if (policyErr) {
return res.status(400).json({
error: policyErr,
message:
policyErr === "password_too_short"
? "Use at least 8 characters."
: policyErr === "password_too_long"
? "256 characters max."
: "Pick a password.",
});
}
try {
const hash = hashPassword(password);
getDb()
.prepare("UPDATE users SET password_hash = ? WHERE id = ?")
.run(hash, req.user.id);
console.log(`[account] password set for user ${req.user.id}`);
res.json({ ok: true });
} catch (err) {
console.error("[account] set-password failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
app.delete("/api/account/password", requireUser, (req, res) => {
if (!req.user || !req.user.id) {
return res.status(401).json({ error: "auth_required" });
}
try {
getDb()
.prepare("UPDATE users SET password_hash = NULL WHERE id = ?")
.run(req.user.id);
console.log(`[account] password cleared for user ${req.user.id}`);
res.json({ ok: true });
} catch (err) {
console.error("[account] clear-password failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
// ── Delete my account ──────────────────────────────────────────────
// GDPR-style hard delete of the calling user. Confirms via a body
// sentinel ({confirm: "DELETE"}) so a stray DELETE request can't
// wipe an account by accident. After deletion the session cookie is
// cleared; the user lands back on the anonymous landing page.
//
// Cascades (via the schema's ON DELETE CASCADE) handle:
// - sessions (drop everywhere)
// - tenant_credits (drop balance row)
// - library_meta (drop the index entries)
// - subscriptions (drop billing history)
// - magic_link_tokens stay (they have no FK to users — they're
// keyed by email, harmless to leave)
// - anon_trials.converted_to_user_id SET NULL (analytics row stays)
//
// The filesystem-side history folder /data/history/<user_id>/ is
// removed separately — SQLite cascades don't reach the FS. We do
// this AFTER the DB delete so partial failures don't leave dangling
// metadata.
//
// Admin users CANNOT delete themselves via this endpoint — that
// would leave the multi-tenant Recap orphaned with no admin. They
// can downgrade themselves first via SQL if they really mean it.
app.delete("/api/account", requireUser, async (req, res) => {
if (!req.user || !req.user.id) {
return res.status(403).json({ error: "no_user" });
}
if (req.user.is_admin) {
return res.status(400).json({
error: "cannot_self_delete_admin",
message:
"Operator account can't be self-deleted (no admin would be left). Demote yourself first.",
});
}
if (req.body?.confirm !== "DELETE") {
return res.status(400).json({
error: "confirmation_required",
message: "Send {confirm: \"DELETE\"} in the body to confirm.",
});
}
const userId = req.user.id;
try {
// FK cascade handles sessions, tenant_credits, library_meta,
// subscriptions. Run inside a transaction so a crash mid-delete
// doesn't leave the user partially intact.
const db = getDb();
const result = db
.transaction(() => {
return db
.prepare("DELETE FROM users WHERE id = ?")
.run(userId);
})();
if (result.changes === 0) {
return res.status(404).json({ error: "user_not_found" });
}
} catch (err) {
console.error("[account] delete user DB op failed:", err);
return res.status(500).json({ error: "internal_error" });
}
// FS cleanup. Best-effort — failures here don't roll the user
// back into existence. Worst case is an orphan folder that the
// operator can sweep manually.
try {
const userDir = path.join(getHistoryDir(), userId);
await fs.rm(userDir, { recursive: true, force: true });
} catch (err) {
console.warn(
`[account] history dir cleanup failed for ${userId} (DB delete succeeded):`,
err,
);
}
// Clear the session cookie so the next request from this browser
// is anonymous, not "stale-session-401-prompted-to-sign-in".
res.setHeader(
"Set-Cookie",
"recap_session=; Max-Age=0; Path=/; HttpOnly; SameSite=Lax; Secure",
);
console.log(`[account] user ${userId} deleted their account`);
res.json({ ok: true });
});
// "Sign out everywhere except this device" — useful after a
// suspicious-login email or just for hygiene. Deletes every session
// for this user EXCEPT the one carrying the request.
app.post(
"/api/account/sessions/revoke-others",
requireUser,
(req, res) => {
if (!req.user || !req.user.id) {
return res.status(400).json({ error: "trial_has_no_sessions" });
}
const currentSessionId = req.session?.id;
try {
let result;
if (currentSessionId) {
result = getDb()
.prepare(
"DELETE FROM sessions WHERE user_id = ? AND id != ?",
)
.run(req.user.id, currentSessionId);
} else {
// No current session detected (shouldn't happen if requireUser
// passed, but defensively): treat as "revoke all of mine."
result = getDb()
.prepare("DELETE FROM sessions WHERE user_id = ?")
.run(req.user.id);
}
res.json({ ok: true, revoked: result.changes });
} catch (err) {
console.error("[account] revoke-others failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── Daily Digest opt-in ────────────────────────────────────────────
// Opt-in (off by default) daily email of the last ~24h of library
// recaps. The relay-owned subscription tier is unrelated — any
// signed-in user may toggle this. GET reads current state; POST
// {enabled:bool} flips it. Enabling stamps last_digest_at to "now"
// so the first digest covers only recaps added AFTER opt-in, never
// the user's whole backlog (the scan picks createdAt > watermark).
app.get("/api/account/digest", requireUser, (req, res) => {
if (!req.user || !req.user.id) {
return res.status(401).json({ error: "auth_required" });
}
try {
const row = getDb()
.prepare("SELECT digest_enabled, last_digest_at FROM users WHERE id = ?")
.get(req.user.id);
res.json({
enabled: !!row?.digest_enabled,
last_digest_at: row?.last_digest_at ?? null,
});
} catch (err) {
console.error("[account] digest read failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
app.post("/api/account/digest", requireUser, (req, res) => {
if (!req.user || !req.user.id) {
return res.status(401).json({ error: "auth_required" });
}
const enabled = req.body?.enabled;
if (typeof enabled !== "boolean") {
return res.status(400).json({ error: "enabled_must_be_boolean" });
}
try {
if (enabled) {
// Start the watermark at opt-in so the first send isn't a backlog dump.
getDb()
.prepare(
"UPDATE users SET digest_enabled = 1, last_digest_at = ? WHERE id = ?",
)
.run(Date.now(), req.user.id);
} else {
getDb()
.prepare("UPDATE users SET digest_enabled = 0 WHERE id = ?")
.run(req.user.id);
}
console.log(
`[account] digest ${enabled ? "enabled" : "disabled"} for user ${req.user.id}`,
);
res.json({ ok: true, enabled });
} catch (err) {
console.error("[account] digest toggle failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
}
+300
View File
@@ -0,0 +1,300 @@
// Admin login gate.
//
// Reads username + scrypt password hash + session secret out of
// /data/config/startos-config.json (set by the "Set Admin Password"
// StartOS action). When a hash is set, gates /api/* behind a signed
// HttpOnly cookie. The static frontend stays open so the login screen
// can paint, but every API endpoint except the gate's own + /api/health
// returns 401 admin_login_required until the cookie validates.
//
// Cookie format: <base64url(payload)>.<base64url(hmac)>
// payload: { u: username, iat: epoch_ms, fp: fingerprint(passwordHash) }
// hmac: HMAC-SHA256(payload, sessionSecret)
//
// Changing the password rotates fp, which invalidates all existing
// sessions on the next request. Changing/clearing the session secret
// has the same effect.
//
// ADMIN is exported as a `let` binding so importers see the live value
// after each config-poll refresh.
import fs from "fs/promises";
import path from "path";
import {
randomBytes,
scryptSync,
createHmac,
timingSafeEqual,
} from "crypto";
const COOKIE_NAME = "recap_admin_session";
const COOKIE_MAX_AGE_MS = 30 * 24 * 60 * 60 * 1000; // 30 days
const SCRYPT_KEYLEN = 64;
// Endpoints reachable WITHOUT an admin session, even when the gate is
// enabled. The login flow itself + the bare-minimum status endpoints.
const ADMIN_OPEN_PATHS = new Set([
"/api/admin/status",
"/api/admin/login",
"/api/admin/logout",
"/api/health",
]);
// ── Module state ────────────────────────────────────────────────────────────
// Live snapshot of the admin-auth config. Refreshed from
// /data/config/startos-config.json every CONFIG_POLL_MS. `enabled` is
// derived (true iff a hash is set).
export let ADMIN = {
enabled: false,
username: "",
passwordHash: "",
passwordSalt: "",
sessionSecret: "",
};
let startosConfigPath = null;
// ── Init ────────────────────────────────────────────────────────────────────
export async function initAdminAuth({ dataDir }) {
startosConfigPath = path.join(dataDir, "config", "startos-config.json");
await refreshAdminConfig("startup");
const pollMs = parseInt(
process.env.RECAP_CONFIG_POLL_MS || "3000",
10
);
setInterval(() => {
refreshAdminConfig("config poll").catch(() => {});
}, pollMs);
}
async function refreshAdminConfig(reason) {
let next = {
enabled: false,
username: "",
passwordHash: "",
passwordSalt: "",
sessionSecret: "",
};
try {
const content = await fs.readFile(startosConfigPath, "utf-8");
const cfg = JSON.parse(content);
next = {
enabled: !!(cfg.recap_admin_password_hash && cfg.recap_admin_password_salt),
username: cfg.recap_admin_username || "",
passwordHash: cfg.recap_admin_password_hash || "",
passwordSalt: cfg.recap_admin_password_salt || "",
sessionSecret: cfg.recap_admin_session_secret || "",
};
} catch {
// File missing / unreadable — leave gate disabled.
}
if (
next.enabled !== ADMIN.enabled ||
next.username !== ADMIN.username ||
next.passwordHash !== ADMIN.passwordHash ||
next.sessionSecret !== ADMIN.sessionSecret
) {
ADMIN = next;
if (reason !== "config poll" || ADMIN.enabled !== false) {
console.log(
`[admin-auth] refresh (${reason}): enabled=${ADMIN.enabled} user=${ADMIN.username || "(unset)"}`
);
}
} else {
ADMIN = next;
}
}
// ── Cookie helpers ──────────────────────────────────────────────────────────
function b64url(buf) {
return Buffer.from(buf)
.toString("base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, "");
}
function b64urlDecode(s) {
const pad = s.length % 4 === 0 ? 0 : 4 - (s.length % 4);
const padded = s + "=".repeat(pad);
return Buffer.from(padded.replace(/-/g, "+").replace(/_/g, "/"), "base64");
}
function hashFingerprint(hash) {
// First 16 hex chars of the password hash. Stored in the cookie so
// changing the password invalidates all existing sessions.
return (hash || "").slice(0, 16);
}
function signSession({ username, hash, secret }) {
const payload = JSON.stringify({
u: username,
iat: Date.now(),
fp: hashFingerprint(hash),
});
const payloadB64 = b64url(payload);
const sig = createHmac("sha256", secret).update(payloadB64).digest();
return `${payloadB64}.${b64url(sig)}`;
}
function verifySession(token, { username, hash, secret }) {
if (!token || typeof token !== "string") return false;
const dot = token.indexOf(".");
if (dot < 0) return false;
const payloadB64 = token.slice(0, dot);
const sigB64 = token.slice(dot + 1);
if (!payloadB64 || !sigB64) return false;
let expected;
try {
expected = createHmac("sha256", secret).update(payloadB64).digest();
} catch {
return false;
}
let provided;
try {
provided = b64urlDecode(sigB64);
} catch {
return false;
}
if (provided.length !== expected.length) return false;
if (!timingSafeEqual(provided, expected)) return false;
let payload;
try {
payload = JSON.parse(b64urlDecode(payloadB64).toString("utf-8"));
} catch {
return false;
}
if (!payload || payload.u !== username) return false;
if (payload.fp !== hashFingerprint(hash)) return false;
if (typeof payload.iat !== "number") return false;
if (Date.now() - payload.iat > COOKIE_MAX_AGE_MS) return false;
return true;
}
function parseCookies(header) {
const out = {};
if (!header || typeof header !== "string") return out;
for (const part of header.split(";")) {
const eq = part.indexOf("=");
if (eq < 0) continue;
const k = part.slice(0, eq).trim();
const v = part.slice(eq + 1).trim();
if (k) out[k] = decodeURIComponent(v);
}
return out;
}
function buildSetCookie(value, { req, maxAgeMs }) {
const parts = [
`${COOKIE_NAME}=${value}`,
"HttpOnly",
"SameSite=Lax",
"Path=/",
];
if (maxAgeMs > 0) {
parts.push(`Max-Age=${Math.floor(maxAgeMs / 1000)}`);
} else {
parts.push("Max-Age=0");
}
// Mark Secure only when the request itself is HTTPS, so the cookie
// still works on plain-HTTP LAN access (the common StartOS dev setup).
const proto = (req.headers["x-forwarded-proto"] || "").toString().toLowerCase();
const isHttps = req.secure || proto.includes("https");
if (isHttps) parts.push("Secure");
return parts.join("; ");
}
function isSessionAuthed(req) {
if (!ADMIN.enabled) return true;
const cookies = parseCookies(req.headers.cookie);
return verifySession(cookies[COOKIE_NAME], {
username: ADMIN.username,
hash: ADMIN.passwordHash,
secret: ADMIN.sessionSecret,
});
}
// ── Middleware ──────────────────────────────────────────────────────────────
// Register BEFORE setupLicenseMiddleware. When the admin gate is
// disabled, this is a no-op pass-through.
export function setupAdminAuthMiddleware(app) {
app.use((req, res, next) => {
if (!ADMIN.enabled) return next();
if (!req.path.startsWith("/api/")) return next();
if (ADMIN_OPEN_PATHS.has(req.path)) return next();
if (isSessionAuthed(req)) return next();
return res.status(401).json({
error: "admin_login_required",
message: "Admin login required.",
});
});
}
// ── Routes ──────────────────────────────────────────────────────────────────
export function setupAdminAuthRoutes(app) {
app.get("/api/admin/status", (req, res) => {
res.json({
enabled: ADMIN.enabled,
authed: isSessionAuthed(req),
username: ADMIN.enabled ? ADMIN.username : null,
});
});
app.post("/api/admin/login", (req, res) => {
if (!ADMIN.enabled) {
// No password set; treat as success so the frontend doesn't get
// stuck on the login screen if the admin clears the password.
return res.json({ ok: true, enabled: false });
}
const username = (req.body && req.body.username) || "";
const password = (req.body && req.body.password) || "";
if (!username || !password) {
return res
.status(400)
.json({ error: "missing_credentials", message: "Username and password required." });
}
if (username !== ADMIN.username) {
return res
.status(401)
.json({ error: "invalid_credentials", message: "Invalid username or password." });
}
let computed;
try {
computed = scryptSync(password, ADMIN.passwordSalt, SCRYPT_KEYLEN);
} catch {
return res
.status(500)
.json({ error: "hash_failed", message: "Could not verify password." });
}
let stored;
try {
stored = Buffer.from(ADMIN.passwordHash, "hex");
} catch {
return res
.status(500)
.json({ error: "stored_hash_invalid", message: "Stored password hash is unreadable." });
}
if (computed.length !== stored.length || !timingSafeEqual(computed, stored)) {
return res
.status(401)
.json({ error: "invalid_credentials", message: "Invalid username or password." });
}
const token = signSession({
username: ADMIN.username,
hash: ADMIN.passwordHash,
secret: ADMIN.sessionSecret,
});
res.setHeader(
"Set-Cookie",
buildSetCookie(token, { req, maxAgeMs: COOKIE_MAX_AGE_MS })
);
res.json({ ok: true, enabled: true, username: ADMIN.username });
});
app.post("/api/admin/logout", (req, res) => {
res.setHeader("Set-Cookie", buildSetCookie("", { req, maxAgeMs: 0 }));
res.json({ ok: true });
});
}
+548
View File
@@ -0,0 +1,548 @@
// Operator (is_admin = 1) endpoints for multi-tenant Recap. Every
// route here is gated by requireOperator from tenant-auth.js — non-
// admin users get 403. Single-mode never mounts these.
//
// What's exposed:
// GET /api/admin/tenants — list all users + credits + license status
// POST /api/admin/tenants/:id/grant — add credits to a user's local balance
// POST /api/admin/tenants/:id/tier — set a user's subscription tier (relay-owned)
// GET /api/admin/tenants/:id/sessions — list a user's active sessions
// DELETE /api/admin/tenants/:id/sessions — revoke ALL of a user's sessions
// DELETE /api/admin/sessions/:sessionId — revoke a specific session
// GET /api/admin/recent-signups — signups grouped by IP/UA in the last N hours
//
// All responses are JSON. Tenant rows shape:
// { id, email, display_name, is_admin, tier, has_license, balance,
// lifetime_granted, lifetime_consumed, created_at, last_signin_at,
// signup_ip, signup_user_agent, session_count }
//
// Volume notes: SQLite, ~100s of tenants tops for an alpha cohort.
// No pagination yet — caps at LIMIT 500 to be defensive.
import { getDb } from "./db.js";
import { requireOperator } from "./tenant-auth.js";
import fs from "fs/promises";
import path from "path";
import { getHistoryDir } from "./history.js";
import { getRelayOperatorKey } from "./relay-default.js";
const MAX_TENANT_ROWS = 500;
const RECENT_SIGNUPS_MAX_HOURS = 24 * 14; // 2 weeks
export function setupAdminRoutes(app) {
// ── Run / test the subscription expiry-reminder scan ───────────────────
// Operator-only. With { test_email } it sends a SAMPLE reminder to that
// address — verifies the recaps.cc SMTP + the email rendering without
// needing a real near-expiry subscription. Without it, forces an
// immediate scan of real expiring subscriptions and returns the summary
// ({ sent, skipped } or a { skipped: <reason> } if a precondition fails).
app.post("/api/admin/reminders/run", requireOperator, async (req, res) => {
try {
const testEmail =
typeof req.body?.test_email === "string" ? req.body.test_email.trim() : "";
if (testEmail) {
const { isSmtpReady, sendMail } = await import("./smtp.js");
if (!isSmtpReady()) {
return res.status(503).json({
error: "smtp_not_ready",
message: "Configure StartOS System SMTP first.",
});
}
const { renderSubscriptionReminderEmail } = await import(
"./email-template.js"
);
const { getConfigSnapshot } = await import("./config.js");
const snap = await getConfigSnapshot();
const publicUrl = (snap.recap_public_url || "")
.trim()
.replace(/\/$/, "");
const msg = renderSubscriptionReminderEmail({
brandName: "Recaps",
tier: "pro",
expiresAt: new Date(Date.now() + 7 * 86_400_000).toISOString(),
daysLeft: 7,
kind: "upcoming_7d",
manageUrl: `${publicUrl}/?renew=1`,
});
await sendMail({
to: testEmail,
subject: msg.subject,
text: msg.text,
html: msg.html,
});
return res.json({ ok: true, test_email_sent_to: testEmail });
}
const { runReminderScan } = await import("./subscription-reminders.js");
const result = await runReminderScan({ force: true });
res.json({ ok: true, ...result });
} catch (err) {
console.error("[admin] reminder run failed:", err?.message || err);
res.status(500).json({
error: "reminder_run_failed",
message: err?.message || String(err),
});
}
});
// Daily-digest test trigger. With {test_email}, sends a sample digest
// to that address so the operator can eyeball the rendering without
// opted-in users or waiting for the send hour. Without it, forces a
// real scan now (bypassing the 08:00 gate, NOT the per-user resend gate).
app.post("/api/admin/digest/run", requireOperator, async (req, res) => {
try {
const testEmail =
typeof req.body?.test_email === "string" ? req.body.test_email.trim() : "";
if (testEmail) {
const { isSmtpReady, sendMail } = await import("./smtp.js");
if (!isSmtpReady()) {
return res.status(503).json({
error: "smtp_not_ready",
message: "Configure StartOS System SMTP first.",
});
}
const { renderDigestEmail } = await import("./email-template.js");
const { getConfigSnapshot } = await import("./config.js");
const snap = await getConfigSnapshot();
const publicUrl = (snap.recap_public_url || "")
.trim()
.replace(/\/$/, "");
const msg = renderDigestEmail({
brandName: "Recaps",
episodes: [
{
title: "Sample podcast episode",
type: "podcast",
url: "https://example.com/episode",
overview:
"This is a sample overview paragraph so you can see how a digest entry renders. The real thing is synthesized from each recap's stored topic summaries.",
},
{
title: "Sample YouTube video",
type: "youtube",
url: "https://youtube.com/watch?v=example",
overview:
"A second sample entry, showing how multiple recaps stack in one email.",
},
],
overflowCount: 0,
manageUrl: `${publicUrl}/`,
unsubscribeUrl: `${publicUrl}/api/digest/unsubscribe?token=sample`,
});
await sendMail({
to: testEmail,
subject: msg.subject,
text: msg.text,
html: msg.html,
});
return res.json({ ok: true, test_email_sent_to: testEmail });
}
const { runDigestScan } = await import("./daily-digest.js");
const result = await runDigestScan({ force: true });
res.json({ ok: true, ...result });
} catch (err) {
console.error("[admin] digest run failed:", err?.message || err);
res.status(500).json({
error: "digest_run_failed",
message: err?.message || String(err),
});
}
});
// ── List all tenants ───────────────────────────────────────────────────
app.get("/api/admin/tenants", requireOperator, (req, res) => {
try {
const rows = getDb()
.prepare(
`SELECT
u.id, u.email, u.display_name, u.is_admin, u.tier,
u.created_at, u.last_signin_at,
u.signup_ip, u.signup_user_agent,
CASE WHEN u.keysat_license IS NOT NULL AND length(u.keysat_license) > 0 THEN 1 ELSE 0 END AS has_license,
COALESCE(tc.purchased_balance, 0) AS purchased_balance,
COALESCE(tc.replenish_balance, 0) AS replenish_balance,
COALESCE(tc.purchased_balance, 0) + COALESCE(tc.replenish_balance, 0) AS balance,
COALESCE(tc.lifetime_granted, 0) AS lifetime_granted,
COALESCE(tc.lifetime_consumed, 0) AS lifetime_consumed,
(SELECT COUNT(*) FROM sessions s
WHERE s.user_id = u.id AND s.expires_at > ?) AS session_count
FROM users u
LEFT JOIN tenant_credits tc ON tc.user_id = u.id
ORDER BY u.created_at DESC
LIMIT ?`,
)
.all(Date.now(), MAX_TENANT_ROWS);
// The relay-owned tier can only be set when this server holds the
// matching operator key (otherwise the grant just 502s). The UI hides
// the per-row "Tier" control when this is false, so a self-hosted
// operator — who has no matching key for the canonical relay — doesn't
// see a button that can't work.
res.json({
tenants: rows,
relay_operator_key_configured: !!getRelayOperatorKey(),
});
} catch (err) {
console.error("[admin] /tenants failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
// ── Grant credits to a tenant ──────────────────────────────────────────
// Body: { amount }. Positive integer.
//
// Upserts tenant_credits and increments balance + lifetime_granted.
// Records the operator's id + a timestamp so we have an audit trail
// when /api/admin/audit lands later (currently the grant is implicit
// via lifetime_granted; the trail can be reconstructed by diffing).
app.post(
"/api/admin/tenants/:id/grant",
requireOperator,
async (req, res) => {
const userId = req.params.id;
const amount = parseInt(req.body?.amount, 10);
if (!Number.isFinite(amount) || amount <= 0 || amount > 100000) {
return res
.status(400)
.json({ error: "bad_amount", message: "amount must be 1..100000" });
}
try {
const db = getDb();
// Ensure the user exists before we upsert credits.
const user = db
.prepare("SELECT id, email FROM users WHERE id = ?")
.get(userId);
if (!user) {
return res.status(404).json({ error: "user_not_found" });
}
// Admin grants go into the PURCHASED bucket — they're explicit
// operator-initiated credits and shouldn't get wiped by the
// next replenishment cycle. Helper handles upsert + lifetime_granted.
const { addPurchased } = await import("./tenant-credits.js");
const row = addPurchased(userId, amount);
const total =
(row?.purchased_balance || 0) + (row?.replenish_balance || 0);
console.log(
`[admin] granted ${amount} credits to ${user.email} (by ${req.user.email}) — total balance ${total}`,
);
res.json({
ok: true,
user_id: userId,
balance: total,
purchased_balance: row?.purchased_balance || 0,
replenish_balance: row?.replenish_balance || 0,
granted: amount,
});
} catch (err) {
console.error("[admin] grant failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── Set a tenant's subscription tier ───────────────────────────────────
// Body: { tier }. One of "core" | "pro" | "max".
//
// Core-decoupling: the RELAY owns the subscription (keyed by Recaps
// user-id), NOT a per-user Keysat license. This route writes the
// relay-side tier FIRST (the authoritative owner), and only on success
// caches it on the local users row. That ordering keeps the two from
// drifting — we never show a user as Pro in the UI while the relay still
// rejects their cloud calls. The cached users.tier is what the per-user
// feature gates (tts-routes, license-status) and the providers'
// cloud-identity selection read on each request.
app.post(
"/api/admin/tenants/:id/tier",
requireOperator,
async (req, res) => {
const userId = req.params.id;
const tier = String(req.body?.tier || "").trim().toLowerCase();
if (!["core", "pro", "max"].includes(tier)) {
return res.status(400).json({
error: "bad_tier",
message: 'tier must be "core", "pro", or "max"',
});
}
try {
const db = getDb();
const user = db
.prepare("SELECT id, email FROM users WHERE id = ?")
.get(userId);
if (!user) {
return res.status(404).json({ error: "user_not_found" });
}
// 1) Authoritative write to the relay (the subscription owner).
// Throws if the operator key / relay base URL isn't configured,
// or if the relay rejects the call — in which case we DON'T
// touch the local cache.
const { setRelayUserTier } = await import("./providers/relay.js");
try {
await setRelayUserTier({ userId, tier });
} catch (err) {
console.error(
"[admin] relay tier push failed:",
err?.message || err,
);
return res.status(502).json({
error: "relay_tier_failed",
message:
"Couldn't set the tier on the relay (the subscription owner). " +
"Check the relay operator key + base URL, then retry. " +
(err?.message || ""),
});
}
// 2) Cache locally so feature gates + the badge see it immediately.
db.prepare("UPDATE users SET tier = ? WHERE id = ?").run(tier, userId);
console.log(
`[admin] set tier=${tier} for ${user.email} (by ${req.user.email})`,
);
res.json({ ok: true, user_id: userId, tier });
} catch (err) {
console.error("[admin] set tier failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── List a tenant's active sessions ────────────────────────────────────
app.get(
"/api/admin/tenants/:id/sessions",
requireOperator,
(req, res) => {
const userId = req.params.id;
try {
const rows = getDb()
.prepare(
`SELECT id, created_at, expires_at, last_used_at, user_agent, ip_address
FROM sessions
WHERE user_id = ? AND expires_at > ?
ORDER BY last_used_at DESC, created_at DESC`,
)
.all(userId, Date.now());
res.json({ sessions: rows });
} catch (err) {
console.error("[admin] sessions list failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── Revoke ALL sessions for a tenant ───────────────────────────────────
// Operator-side "sign this user out everywhere" button. Useful when
// a tenant flags account theft, or for ban-hammer scenarios. The
// user can sign back in via magic link unless the operator also
// disables their email (future feature).
app.delete(
"/api/admin/tenants/:id/sessions",
requireOperator,
(req, res) => {
const userId = req.params.id;
try {
const result = getDb()
.prepare("DELETE FROM sessions WHERE user_id = ?")
.run(userId);
console.log(
`[admin] revoked ${result.changes} session(s) for user ${userId} (by ${req.user.email})`,
);
res.json({ ok: true, revoked: result.changes });
} catch (err) {
console.error("[admin] revoke all sessions failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── Revoke a single session by id ──────────────────────────────────────
app.delete(
"/api/admin/sessions/:sessionId",
requireOperator,
(req, res) => {
const sessionId = req.params.sessionId;
try {
const result = getDb()
.prepare("DELETE FROM sessions WHERE id = ?")
.run(sessionId);
res.json({ ok: true, revoked: result.changes });
} catch (err) {
console.error("[admin] revoke session failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── Delete a tenant (admin ban-hammer) ─────────────────────────────────
// Operator removes a tenant — abuse response, account cleanup, etc.
// Mirrors the user-side /api/account DELETE but operator-scoped:
// - Refuses to delete another admin (admins protect each other)
// - Refuses to delete the calling operator (use the user-side
// endpoint or SQL if you really need that)
// - Cascades sessions, tenant_credits, library_meta via FK ON DELETE
// - FS cleanup of /data/history/<user_id>/ is best-effort
app.delete(
"/api/admin/tenants/:id",
requireOperator,
async (req, res) => {
const userId = req.params.id;
if (userId === req.user.id) {
return res.status(400).json({
error: "cannot_delete_self",
message:
"Use the user-side endpoint to delete your own account, or demote first.",
});
}
try {
const db = getDb();
const target = db
.prepare("SELECT id, email, is_admin FROM users WHERE id = ?")
.get(userId);
if (!target) {
return res.status(404).json({ error: "user_not_found" });
}
if (target.is_admin) {
return res.status(400).json({
error: "cannot_delete_admin",
message:
"Demote this user from admin first if you really want to delete them.",
});
}
db.prepare("DELETE FROM users WHERE id = ?").run(userId);
console.log(
`[admin] deleted tenant ${target.email} (id ${userId}) (by ${req.user.email})`,
);
// FS cleanup, best-effort.
try {
const userDir = path.join(getHistoryDir(), userId);
await fs.rm(userDir, { recursive: true, force: true });
} catch (err) {
console.warn(
`[admin] history-dir cleanup failed for ${userId}:`,
err?.message || err,
);
}
res.json({ ok: true });
} catch (err) {
console.error("[admin] delete tenant failed:", err);
res.status(500).json({ error: "internal_error" });
}
},
);
// ── Recent signups (forensic / abuse detection) ────────────────────────
// Query: ?hours=N (default 24, clamped to RECENT_SIGNUPS_MAX_HOURS).
// Returns:
// {
// window_hours: 24,
// by_ip: [{ ip, count, emails[], first_seen, last_seen }],
// by_ua: [{ ua, count, ... }],
// by_hour: [{ hour, count }],
// totals: { signups, magic_link_requests }
// }
app.get("/api/admin/recent-signups", requireOperator, (req, res) => {
const hoursReq = parseInt(req.query?.hours, 10) || 24;
const hours = Math.min(
Math.max(1, hoursReq),
RECENT_SIGNUPS_MAX_HOURS,
);
const cutoff = Date.now() - hours * 60 * 60 * 1000;
try {
const db = getDb();
// Tracked sources:
// - users.signup_* — confirmed signups (someone clicked magic link)
// - magic_link_tokens.request_* — link requests, includes unconverted
//
// Top-line metrics show both so the operator can spot patterns
// like "10000 link requests, 2 actual signups" → scripted abuse.
// Signups by IP
const byIp = db
.prepare(
`SELECT signup_ip AS ip, COUNT(*) AS count,
MIN(created_at) AS first_seen,
MAX(created_at) AS last_seen,
GROUP_CONCAT(email, '|') AS emails_joined
FROM users
WHERE created_at > ? AND signup_ip IS NOT NULL AND signup_ip != ''
GROUP BY signup_ip
ORDER BY count DESC, last_seen DESC
LIMIT 50`,
)
.all(cutoff)
.map((r) => ({
ip: r.ip,
count: r.count,
first_seen: r.first_seen,
last_seen: r.last_seen,
emails: (r.emails_joined || "").split("|").filter(Boolean),
}));
// Signups by UA (truncated for readability — group on the first
// 80 chars so "Mozilla/5.0 ... Chrome/120" rows collapse into
// one even if patch versions differ slightly).
const byUa = db
.prepare(
`SELECT substr(signup_user_agent, 1, 80) AS ua, COUNT(*) AS count,
MIN(created_at) AS first_seen,
MAX(created_at) AS last_seen
FROM users
WHERE created_at > ? AND signup_user_agent IS NOT NULL AND signup_user_agent != ''
GROUP BY substr(signup_user_agent, 1, 80)
ORDER BY count DESC, last_seen DESC
LIMIT 50`,
)
.all(cutoff);
// Signups per hour (for a sparkline; small array).
const byHour = db
.prepare(
`SELECT (created_at / 3600000) * 3600000 AS hour, COUNT(*) AS count
FROM users
WHERE created_at > ?
GROUP BY hour
ORDER BY hour ASC`,
)
.all(cutoff);
const signupCount = db
.prepare("SELECT COUNT(*) AS n FROM users WHERE created_at > ?")
.get(cutoff).n;
const magicLinkCount = db
.prepare(
"SELECT COUNT(*) AS n FROM magic_link_tokens WHERE created_at > ?",
)
.get(cutoff).n;
// Magic-link request distribution by IP (catches "lots of
// requests, no signups" abuse — abusers who keep requesting
// links but never click them, or who can't because the email
// isn't real). Truncate emails-joined to first 5 to keep
// payload reasonable.
const linkByIp = db
.prepare(
`SELECT request_ip AS ip, COUNT(*) AS count,
MIN(created_at) AS first_seen,
MAX(created_at) AS last_seen,
COUNT(DISTINCT email) AS distinct_emails
FROM magic_link_tokens
WHERE created_at > ? AND request_ip IS NOT NULL AND request_ip != ''
GROUP BY request_ip
ORDER BY count DESC, last_seen DESC
LIMIT 50`,
)
.all(cutoff);
res.json({
window_hours: hours,
totals: {
signups: signupCount,
magic_link_requests: magicLinkCount,
},
signups_by_ip: byIp,
signups_by_ua: byUa,
signups_by_hour: byHour,
magic_links_by_ip: linkByIp,
});
} catch (err) {
console.error("[admin] recent-signups failed:", err);
res.status(500).json({ error: "internal_error" });
}
});
}
+387
View File
@@ -0,0 +1,387 @@
// Cookie-gated "taste before sign-up" trial for unauthenticated
// visitors on a multi-tenant Recap. First time a visitor POSTs to
// /api/process without a session cookie, we issue a recap_anon_trial
// cookie (32-byte random), insert an anon_trials row with N credits,
// and let them summarize without signing up. After credits_used >=
// credits_total, the UI nudges them to create an account.
//
// Trial summaries forward the OPERATOR's install_id + license to the
// relay — they're paid for out of the operator's credit pool, gated
// solely by the anon_trials.credits_total field. tenant_credits is
// irrelevant for trials (no user row exists yet).
//
// Multi-mode only. Single-mode never imports this module.
import { randomBytes } from "crypto";
import { getDb } from "./db.js";
import { getConfigSnapshot } from "./config.js";
export const TRIAL_COOKIE = "recap_anon_trial";
const TRIAL_COOKIE_MAX_AGE_MS = 7 * 24 * 60 * 60 * 1000; // 7 days
// Returns the trial config from the StartOS config snapshot. Cached
// per-call by getConfigSnapshot (already polled by config.js), so
// reading per-request is cheap.
async function getTrialConfig() {
const snap = await getConfigSnapshot();
// Prefer the new `trials_per_ip_lifetime` field; fall back to the
// legacy `trials_per_ip_per_day` so a config that hasn't been
// re-saved under the new name keeps working with its existing value.
// (Semantics is now lifetime in BOTH cases — the rename is mostly
// cosmetic for the operator-facing knob.)
const ipCap =
snap.trials_per_ip_lifetime ??
snap.trials_per_ip_per_day ??
5;
return {
creditsPerVisitor: Math.max(
0,
parseInt(snap.trial_credits_per_visitor ?? 1, 10) || 0,
),
perIpLifetime: Math.max(1, parseInt(ipCap, 10) || 5),
};
}
// Resolve the real client IP. We rely on Express's `trust proxy` setting
// (configured in index.js to the number of trusted proxies in front of the
// app) so req.ip is the address the trusted proxy observed — NOT a value the
// client can spoof by sending their own X-Forwarded-For. This previously took
// the first XFF entry verbatim, which a client could forge to mint unlimited
// trials. Falls back to the raw socket address if req.ip isn't populated.
export function getClientIp(req) {
const ip = req.ip || req.socket?.remoteAddress || "";
return ip.replace(/^::ffff:/, "");
}
// Expand an IPv6 string to its full 8-group :-separated form with
// each group lowercased + zero-padded to 4 hex chars. Returns null
// if the input doesn't look like a valid IPv6 string. Used by
// ipCapKey() below — we need a stable canonical form before we can
// extract the /64 prefix for cap counting.
function expandIpv6(addr) {
if (!addr || typeof addr !== "string") return null;
// Strip any IPv4-mapped suffix (e.g. ::ffff:1.2.3.4) — those are
// really IPv4 addresses tunneled through an IPv6 envelope; the
// caller has already stripped ::ffff: in getClientIp, but
// defensively handle the bare form.
if (/\d+\.\d+\.\d+\.\d+$/.test(addr)) return null;
if (!addr.includes(":")) return null;
const dblIdx = addr.indexOf("::");
let groups;
if (dblIdx === -1) {
// No ::, must be 8 groups.
groups = addr.split(":");
if (groups.length !== 8) return null;
} else {
// :: shorthand — split into left and right halves, fill with 0s.
if (addr.indexOf("::", dblIdx + 2) !== -1) return null; // two :: → invalid
const left = addr.slice(0, dblIdx);
const right = addr.slice(dblIdx + 2);
const leftGroups = left ? left.split(":") : [];
const rightGroups = right ? right.split(":") : [];
const missing = 8 - leftGroups.length - rightGroups.length;
if (missing < 0) return null;
groups = [...leftGroups, ...Array(missing).fill("0"), ...rightGroups];
}
// Validate + normalize each group.
const out = [];
for (const g of groups) {
if (!/^[0-9a-fA-F]{1,4}$/.test(g)) return null;
out.push(g.toLowerCase().padStart(4, "0"));
}
return out.join(":");
}
// Cap-counting key for an IP. IPv4 returns the address unchanged
// (whole-address caps work fine — only one device can claim a
// public-routable IPv4). IPv6 returns the /64 prefix because RFC
// 4941 privacy extensions rotate the lower 64 bits of the address
// per-device, per-session, often hourly — counting by full IPv6
// address means a single laptop can mint a fresh trial cookie
// every time its OS picks a new temporary address. /64 is the
// smallest network unit an ISP delegates to a subscriber, so it's
// the correct boundary for "this household / this network" caps.
// Returns null when the input isn't usable; callers treat null as
// "no cap" (same fallback behavior as before).
//
// Trade-off: a /64 might be shared by multiple unrelated subscribers
// in some carrier configurations, so legitimate visitors could
// (rarely) get capped together. Operator can tune
// trials_per_ip_lifetime higher if they're seeing real complaints.
export function ipCapKey(ip) {
if (!ip) return null;
// IPv4: whole address.
if (/^\d+\.\d+\.\d+\.\d+$/.test(ip)) return ip;
// IPv6: /64 prefix. Expand to full form, take first 4 groups,
// append `:` so the SQL LIKE match doesn't accidentally match
// unrelated prefixes that happen to share a textual leading
// substring (e.g. "2600:17" prefix-matching "2600:171…").
const expanded = expandIpv6(ip);
if (!expanded) return null;
const groups = expanded.split(":");
if (groups.length < 4) return null;
return groups.slice(0, 4).join(":") + ":";
}
// Truncate UA so a pathological 8KB header doesn't bloat the DB. 256
// chars covers every legit browser UA string with room to spare.
function clipUA(ua) {
if (!ua) return "";
return String(ua).slice(0, 256);
}
// Count ALL trial cookies ever issued from this IP. Switched from
// rolling-24h to lifetime in 0.2.84 — a user who clears cookies can no
// longer just wait a day and replay the trial. The trade-off: a
// shared-NAT household whose router IP got 5 different family
// members' trials over a year will eventually be capped. The
// operator can tune `trials_per_ip_lifetime` higher if that's a
// real concern, or grant credits manually from the admin panel.
export function ipTrialsLifetime(ip) {
const key = ipCapKey(ip);
if (!key) return 0;
// IPv4 → exact match. IPv6 → prefix LIKE match (key ends with ":",
// so the SQL LIKE 'key%' walks every full IPv6 in that /64 prefix).
// Both are index-friendly when ip_address is indexed.
if (key.includes(":")) {
const row = getDb()
.prepare("SELECT COUNT(*) AS n FROM anon_trials WHERE ip_address LIKE ?")
.get(key + "%");
return row?.n || 0;
}
const row = getDb()
.prepare("SELECT COUNT(*) AS n FROM anon_trials WHERE ip_address = ?")
.get(key);
return row?.n || 0;
}
// Insert a new trial row and return it. Caller is responsible for
// setting the Set-Cookie header on the response.
function createTrial({ ip, ua, creditsTotal }) {
const cookieId = randomBytes(32).toString("base64url");
const now = Date.now();
getDb()
.prepare(
`INSERT INTO anon_trials
(cookie_id, ip_address, user_agent, created_at, credits_total, credits_used)
VALUES (?, ?, ?, ?, ?, 0)`,
)
.run(cookieId, ip || null, clipUA(ua), now, creditsTotal);
return {
cookie_id: cookieId,
ip_address: ip || null,
user_agent: clipUA(ua),
created_at: now,
credits_total: creditsTotal,
credits_used: 0,
last_used_at: null,
converted_to_user_id: null,
};
}
// lookupTrial(cookieId) — returns the row or null. Doesn't check
// credit balance; that's the caller's job.
export function lookupTrial(cookieId) {
if (!cookieId) return null;
return (
getDb()
.prepare("SELECT * FROM anon_trials WHERE cookie_id = ?")
.get(cookieId) || null
);
}
// hasTrialBudget(trial) — true iff the trial row exists and has
// unused credits. Centralized so the policy is one place.
export function hasTrialBudget(trial) {
if (!trial) return false;
return trial.credits_used < trial.credits_total;
}
// issueIfEligible({ req, res, forceMint }) — atomic "do we issue a
// trial cookie to this request?" decision. Called by the auth
// middleware when no session cookie is present AND the request is
// hitting a path that counts as "actually using the product" (e.g.
// POST /api/process).
//
// Returns the trial row on success, or null if:
// - trial_credits_per_visitor is 0 (trials disabled) AND
// forceMint is false
// - the IP has already exhausted its lifetime mint quota AND
// forceMint is false
// - a DB error occurred (logged, fail-closed to "no trial")
//
// forceMint: caller asserts this isn't a free-trial-abuse scenario
// (e.g., the visitor is paying for credits — they need a tracking
// cookie regardless of trial policy). When set:
// - IP lifetime cap is bypassed
// - Trials-disabled config is bypassed, but the minted cookie
// gets credits_total = 0 (no free bonus on a trials-off install)
// Normal /api/process traffic should NEVER set this; only paid-flow
// callers (/api/credits/buy) where "no cookie → can't credit the
// settle" is a worse failure than enforcing the free-trial cap.
//
// On success, also writes the Set-Cookie header so the browser
// carries the trial id on subsequent requests.
export async function issueIfEligible({ req, res, forceMint = false }) {
let cfg;
try {
cfg = await getTrialConfig();
} catch (err) {
console.warn("[anon-trial] config read failed:", err);
return null;
}
// Decide the credits_total this trial row starts with. Paying
// buyers on a trials-off install still get a cookie — credits_total
// is just 0 — so /api/credits/buy has somewhere to land the purchase.
const creditsTotal = forceMint
? Math.max(0, cfg.creditsPerVisitor)
: cfg.creditsPerVisitor;
if (!forceMint && creditsTotal <= 0) return null; // trials disabled
const ip = getClientIp(req);
if (!forceMint && ip && ipTrialsLifetime(ip) >= cfg.perIpLifetime) {
// Over the lifetime IP cap. Don't issue; visitor sees the same
// "sign up" nudge a returning trial-exhausted user sees. The
// operator can grep magic_link_tokens + anon_trials by ip_address
// if a pattern emerges, or manually grant credits to specific
// tenants from the admin panel.
return null;
}
let trial;
try {
trial = createTrial({
ip,
ua: req.headers?.["user-agent"] || "",
creditsTotal,
});
} catch (err) {
console.warn("[anon-trial] createTrial failed:", err);
return null;
}
// Diagnostic: log the resolved IP + the cap-counting key so an
// operator can grep mint events to verify the IP detector is
// working (a flood of mints with ip=null or ip=127.0.0.1 means
// the reverse-proxy isn't passing X-Forwarded-For). Cap key
// shows the bucket this mint counted under — for IPv6 this is
// the /64 prefix the mint will share with future addresses on
// the same network.
console.log(
`[anon-trial] minted cookie for ip=${ip || "(unknown)"} cap_key=${ipCapKey(ip) || "(none)"} credits=${creditsTotal}`,
);
// Set-Cookie. HttpOnly so browser-side JS can't read or replay it;
// SameSite=Lax is enough since we never need cross-site trial
// posts. Secure is on in production (StartOS terminates HTTPS at
// the tunnel) but harmless on localhost dev.
const maxAgeSeconds = Math.floor(TRIAL_COOKIE_MAX_AGE_MS / 1000);
res.setHeader?.(
"Set-Cookie",
[
`${TRIAL_COOKIE}=${trial.cookie_id}`,
`Max-Age=${maxAgeSeconds}`,
"Path=/",
"HttpOnly",
"SameSite=Lax",
"Secure",
].join("; "),
);
return trial;
}
// debitOne(cookieId) — atomic +1 to credits_used. Returns the new
// row. Caller (the /api/process handler in multi-mode) calls this
// AFTER the relay accepts the request, so a failed relay call
// doesn't burn a trial credit.
export function debitOne(cookieId) {
const now = Date.now();
getDb()
.prepare(
"UPDATE anon_trials SET credits_used = credits_used + 1, last_used_at = ? WHERE cookie_id = ?",
)
.run(now, cookieId);
return lookupTrial(cookieId);
}
// linkToUser(cookieId, userId) — called by /auth/verify when a trial
// holder completes signup. Records the conversion for analytics AND
// transfers any unused credits on the trial to the user's
// tenant_credits balance. This is the user-facing promise: "your free
// trial credits + any credits you bought during the trial transfer
// to your account when you sign up."
//
// "Unused" = credits_total - credits_used. The default trial allowance
// (e.g. 1 free) plus any credits the visitor purchased anonymously
// minus whatever they've already spent.
//
// Returns the number of credits transferred (0 if none).
//
// Async because we opportunistically sweep any settled-but-unapplied
// pending purchases for this anon cookie FIRST, so a la carte credits
// the visitor bought right before signup (and where the BTCPay
// redirect killed the frontend poller) get rolled into credits_total
// before we compute the transfer. Without this sweep, a buy-then-
// immediately-sign-up flow drops the just-purchased credits on the
// floor.
export async function linkToUser(cookieId, userId) {
if (!cookieId || !userId) return 0;
try {
const { sweepUnappliedPurchases } = await import("./credits-purchase.js");
await sweepUnappliedPurchases({ buyerType: "anon", buyerId: cookieId });
} catch (err) {
console.warn(
`[anon-trial] pre-link sweep failed for ${cookieId}: ${err?.message || err}`,
);
}
const db = getDb();
const trial = db
.prepare(
"SELECT credits_total, credits_used FROM anon_trials WHERE cookie_id = ?",
)
.get(cookieId);
const remaining = trial
? Math.max(
0,
(trial.credits_total || 0) - (trial.credits_used || 0),
)
: 0;
// Carry-over credits go into the PURCHASED bucket — they're a mix
// of "leftover free trial allowance" + "credits the anon bought a
// la carte". Treating all of them as purchased (permanent) is the
// safe interpretation; the user paid for some of these and the
// others were earned by clicking through a signup. We don't want
// the next replenishment cycle wiping them.
const tx = db.transaction(() => {
db.prepare(
"UPDATE anon_trials SET converted_to_user_id = ? WHERE cookie_id = ?",
).run(userId, cookieId);
if (remaining > 0) {
const existing = db
.prepare("SELECT user_id FROM tenant_credits WHERE user_id = ?")
.get(userId);
if (existing) {
db.prepare(
`UPDATE tenant_credits
SET purchased_balance = purchased_balance + ?,
lifetime_granted = lifetime_granted + ?
WHERE user_id = ?`,
).run(remaining, remaining, userId);
} else {
db.prepare(
`INSERT INTO tenant_credits
(user_id, purchased_balance, replenish_balance, last_replenish_at,
lifetime_granted, lifetime_consumed)
VALUES (?, ?, 0, ?, ?, 0)`,
).run(userId, remaining, Date.now(), remaining);
}
}
});
tx();
if (remaining > 0) {
console.log(
`[anon-trial] transferred ${remaining} unused credits from ${cookieId} → user ${userId}`,
);
}
return remaining;
}
+121 -10
View File
@@ -7,6 +7,8 @@ import { promisify } from "util";
import path from "path";
import http from "http";
import https from "https";
import dns from "dns";
import net from "net";
import { createWriteStream } from "fs";
const execFileAsync = promisify(execFile);
@@ -53,34 +55,143 @@ export async function splitAudioFile(inputPath, outputDir, chunkSeconds = 2700)
"-acodec", "copy",
chunkPath,
], { timeout: 120000 });
chunks.push({ path: chunkPath, startOffset: startSec, index: i });
chunks.push({
path: chunkPath,
startOffset: startSec,
// Actual seconds in THIS chunk (the last chunk is usually
// shorter than chunkSeconds). Carried downstream so the
// transcribe-stitching code can sanity-cap timestamps each
// chunk's model emits — some models hallucinate offsets
// way past the chunk's audio (observed: gemini-3.1-flash-lite
// emitting [10:12:44] on a 45-min chunk).
durationSec: segLen,
index: i,
});
startSec += chunkSeconds;
i++;
}
return chunks;
}
// ── SSRF guard for outbound podcast fetches ─────────────────────────────────
// downloadPodcastAudio fetches a fully user-controlled URL, so without a
// guard a caller could point it at internal services (cloud metadata at
// 169.254.169.254, LAN hosts, localhost) and read the response back through
// the transcript. isBlockedAddress rejects loopback / private / link-local /
// reserved / multicast targets for IPv4, IPv6, and IPv4-mapped IPv6.
export function isBlockedAddress(ip) {
if (!ip || typeof ip !== "string") return true;
// IPv4-mapped IPv6 in dotted form (::ffff:1.2.3.4) — judge by the embedded
// IPv4. (The hex-encoded forms are caught in the IPv6 branch below.)
const mapped = ip.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/i);
const addr = mapped ? mapped[1] : ip;
if (net.isIPv4(addr)) {
const [a, b] = addr.split(".").map(Number);
if (a === 0) return true; // 0.0.0.0/8 "this network"
if (a === 10) return true; // private
if (a === 127) return true; // loopback
if (a === 169 && b === 254) return true; // link-local (cloud metadata)
if (a === 172 && b >= 16 && b <= 31) return true; // private
if (a === 192 && b === 168) return true; // private
if (a === 100 && b >= 64 && b <= 127) return true; // CGNAT (100.64.0.0/10)
if (a >= 224) return true; // multicast + reserved (224.0.0.0+)
return false;
}
if (net.isIPv6(addr)) {
const a = addr.toLowerCase();
if (a === "::1" || a === "::") return true; // loopback / unspecified
if (a.startsWith("fc") || a.startsWith("fd")) return true; // fc00::/7 ULA
if (/^fe[89ab]/.test(a)) return true; // fe80::/10 link-local
if (a.startsWith("ff")) return true; // ff00::/8 multicast
// Translation / embedded-IPv4 prefixes can smuggle a private IPv4 past the
// rules above (the dotted ::ffff:1.2.3.4 form is normalized to IPv4 at the
// top; these catch the hex-encoded forms: IPv4-mapped/-compatible, SIIT,
// NAT64, 6to4). None is ever a real podcast host, so block the whole
// prefix rather than decode the embedded address.
if (/^::[0-9a-f]/.test(a)) return true; // ::/96 mapped / compat / SIIT (hex)
if (a.startsWith("64:ff9b:")) return true; // NAT64 well-known (RFC 6052)
if (a.startsWith("2002:")) return true; // 6to4
return false;
}
return true; // unrecognized → block
}
// dns.lookup wrapper that fails the connection if the host resolves to a
// blocked address. Passed as the `lookup` option to http(s).get, so the
// check runs at connect time on every attempt — including each redirect
// hop — which also closes the DNS-rebinding window (the address we validate
// is the address the socket connects to).
function guardedLookup(hostname, options, callback) {
if (typeof options === "function") {
callback = options;
options = {};
}
dns.lookup(hostname, options, (err, address, family) => {
if (err) return callback(err);
const addrs = Array.isArray(address) ? address : [{ address, family }];
for (const a of addrs) {
if (isBlockedAddress(a.address)) {
return callback(
new Error(`refusing to fetch podcast audio from disallowed address ${a.address}`),
);
}
}
callback(null, address, family);
});
}
// ── Download a podcast episode by URL ───────────────────────────────────────
// Streams the HTTP response straight to disk. Follows redirects. Rejects
// on any non-200 final status. Used by /api/process when the input URL is
// a podcast episode rather than a YouTube video.
// Streams the HTTP response straight to disk. Follows up to MAX_PODCAST_REDIRECTS
// redirects (resolving relative Location headers), rejects on any non-200 final
// status, and refuses non-HTTP(S) schemes and internal addresses (see the SSRF
// guard above). Used by /api/process when the input is a podcast episode.
const MAX_PODCAST_REDIRECTS = 5;
export function downloadPodcastAudio(audioUrl, destPath) {
return new Promise((resolve, reject) => {
const doFetch = (url) => {
const getter = url.startsWith("https") ? https : http;
getter.get(url, (res) => {
const doFetch = (rawUrl, redirectsLeft) => {
let url;
try {
url = new URL(rawUrl);
} catch {
return reject(new Error("invalid podcast audio URL"));
}
if (url.protocol !== "http:" && url.protocol !== "https:") {
return reject(new Error(`refusing non-HTTP podcast URL (${url.protocol})`));
}
// IP-literal hosts (e.g. http://169.254.169.254) never hit the DNS
// `lookup` hook — the socket connects to the literal directly — so they
// must be checked here. guardedLookup below covers hostnames that
// *resolve* to a blocked address (and the DNS-rebinding case).
const host = url.hostname.replace(/^\[|\]$/g, ""); // strip IPv6 brackets
if (net.isIP(host) && isBlockedAddress(host)) {
return reject(
new Error(`refusing to fetch podcast audio from disallowed address ${host}`),
);
}
const getter = url.protocol === "https:" ? https : http;
getter
.get(url, { lookup: guardedLookup }, (res) => {
if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
return doFetch(res.headers.location);
res.resume(); // drain so the socket is freed
if (redirectsLeft <= 0) {
return reject(new Error("too many redirects downloading podcast audio"));
}
const next = new URL(res.headers.location, url).toString();
return doFetch(next, redirectsLeft - 1);
}
if (res.statusCode !== 200) {
res.resume();
return reject(new Error(`HTTP ${res.statusCode} downloading podcast audio`));
}
const fileStream = createWriteStream(destPath);
res.pipe(fileStream);
fileStream.on("finish", () => fileStream.close(resolve));
fileStream.on("error", reject);
}).on("error", reject);
})
.on("error", reject);
};
doFetch(audioUrl);
doFetch(audioUrl, MAX_PODCAST_REDIRECTS);
});
}
+814
View File
@@ -0,0 +1,814 @@
// Magic-link auth endpoints for multi-tenant cloud mode.
//
// Flow:
// 1. POST /auth/request-link { email }
// - Normalize, rate-limit, generate token, hash, store hash
// - Send email with verifyUrl containing the plaintext token
// - Always returns { ok: true } — never leaks whether email exists
//
// 2. GET /auth/verify?token=<plaintext>
// - Hash, look up, validate (unused + unexpired)
// - Mark used; upsert user; issue session cookie
// - If first user ever, mark is_admin = 1 (operator bootstrap)
// - If request carries a recap_anon_trial cookie, link the trial
// row to the new user_id so their trial summary lands in their
// library and the conversion gets recorded
// - 302 redirect to /
//
// 3. POST /auth/signout
// - Delete session row, clear cookie, redirect to / (or 204 if API)
//
// Multi-mode only. The route registration helper itself returns early
// in single mode so single-mode boot doesn't initialize SMTP / DB
// codepaths.
import {
randomBytes,
createHash,
scryptSync,
timingSafeEqual,
} from "crypto";
import * as cookie from "cookie";
import { getDb } from "./db.js";
import { sendMail, isSmtpReady } from "./smtp.js";
import { renderMagicLinkEmail } from "./email-template.js";
import { getConfigSnapshot } from "./config.js";
import { getClientIp, TRIAL_COOKIE, linkToUser } from "./anon-trial.js";
import { renameScopeDir } from "./history.js";
import { requireUser } from "./tenant-auth.js";
export const SESSION_COOKIE = "recap_session";
const SESSION_MAX_AGE_MS = 30 * 24 * 60 * 60 * 1000; // 30 days
const MAGIC_LINK_TTL_MS = 15 * 60 * 1000; // 15 minutes
// ── Password hashing ─────────────────────────────────────────────────
// scrypt with a per-password 16-byte salt, stored as
// "scrypt$<saltHex>$<hashHex>" in users.password_hash. Same primitive
// admin-auth.js uses for the single-mode operator password, so the
// crypto surface area stays consistent across the codebase. N=2^15
// (KDF cost) is what the Node docs suggest for interactive logins —
// ~50ms on commodity hardware, slow enough to deter brute force,
// fast enough to not feel laggy.
const SCRYPT_KEYLEN = 64;
const SCRYPT_OPTS = { N: 1 << 15, r: 8, p: 1, maxmem: 64 * 1024 * 1024 };
const SCRYPT_PREFIX = "scrypt$";
export function hashPassword(plain) {
const salt = randomBytes(16);
const derived = scryptSync(plain, salt, SCRYPT_KEYLEN, SCRYPT_OPTS);
return `${SCRYPT_PREFIX}${salt.toString("hex")}$${derived.toString("hex")}`;
}
function verifyPassword(plain, stored) {
if (!stored || !stored.startsWith(SCRYPT_PREFIX)) return false;
const [, saltHex, hashHex] = stored.split("$");
if (!saltHex || !hashHex) return false;
let salt, expected;
try {
salt = Buffer.from(saltHex, "hex");
expected = Buffer.from(hashHex, "hex");
} catch {
return false;
}
if (expected.length !== SCRYPT_KEYLEN) return false;
let actual;
try {
actual = scryptSync(plain, salt, SCRYPT_KEYLEN, SCRYPT_OPTS);
} catch {
return false;
}
// timingSafeEqual requires equal length, which we just enforced above.
return timingSafeEqual(actual, expected);
}
// Mild policy — 8 char minimum, 256 max. We deliberately don't enforce
// "one uppercase + one digit + ..." style rules; the consensus modern
// view (NIST 800-63B) is that length matters far more than composition,
// and rules that demand specific shapes just push users toward
// predictable substitutions ("Password1!"). Min 8 catches the worst
// cases without alienating people using passphrases.
export function validatePasswordPolicy(plain) {
if (typeof plain !== "string") return "password_required";
if (plain.length < 8) return "password_too_short";
if (plain.length > 256) return "password_too_long";
return null;
}
// ── Rate limits ────────────────────────────────────────────────────────
// Both buckets are in-memory: not durable across restarts, but the worst
// case is one extra link request per email/IP per restart, which is fine.
// For abuse on the scale where in-memory limits are insufficient, the
// operator's IP/UA logs + manual deny-list are the next layer.
const MAX_LINKS_PER_EMAIL_PER_HOUR = 5;
const MAX_LINKS_PER_IP_PER_HOUR = 10;
const ONE_HOUR_MS = 60 * 60 * 1000;
const emailBuckets = new Map(); // email → [timestamp, ...]
const ipBuckets = new Map(); // ip → [timestamp, ...]
function pushBucket(map, key, now) {
const arr = map.get(key) || [];
const fresh = arr.filter((t) => now - t < ONE_HOUR_MS);
fresh.push(now);
map.set(key, fresh);
return fresh.length;
}
function bucketCount(map, key, now) {
const arr = map.get(key) || [];
const fresh = arr.filter((t) => now - t < ONE_HOUR_MS);
if (fresh.length !== arr.length) map.set(key, fresh);
return fresh.length;
}
// Periodically prune old bucket entries so the maps don't grow
// unbounded under heavy traffic. Fire-and-forget; the filter above
// also self-prunes per access.
const BUCKET_GC_MS = 5 * 60 * 1000;
setInterval(() => {
const cutoff = Date.now() - ONE_HOUR_MS;
for (const map of [emailBuckets, ipBuckets]) {
for (const [key, arr] of map.entries()) {
const fresh = arr.filter((t) => t > cutoff);
if (fresh.length === 0) map.delete(key);
else if (fresh.length !== arr.length) map.set(key, fresh);
}
}
}, BUCKET_GC_MS).unref?.();
function normalizeEmail(raw) {
if (typeof raw !== "string") return "";
return raw.trim().toLowerCase();
}
function isPlausibleEmail(s) {
// Deliberately permissive — the user's mail server is the source of
// truth for whether an address works (they either receive the link or
// they don't). Just sanity-check that we have something with @ and a
// dot in the domain.
return /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(s) && s.length <= 254;
}
function sha256(s) {
return createHash("sha256").update(s).digest("hex");
}
function uuid() {
// 16 random bytes formatted as a UUIDv4-ish hex with hyphens. Doesn't
// need to be RFC-compliant — uniqueness is the only requirement.
return randomBytes(16).toString("hex");
}
function clipUA(ua) {
return String(ua || "").slice(0, 256);
}
// ── Public URL plumbing ────────────────────────────────────────────────
// The verify URL in the email needs the operator's ClearNet URL set via
// the "Set Recap Public URL" StartOS action. We refuse to send magic
// links if it's empty — otherwise the email would link to localhost or
// an internal hostname.
async function getPublicUrl() {
const snap = await getConfigSnapshot();
const url = (snap.recap_public_url || "").trim();
return url.replace(/\/$/, "");
}
// ── Route handlers ─────────────────────────────────────────────────────
async function handleRequestLink(req, res) {
const email = normalizeEmail(req.body?.email);
if (!email || !isPlausibleEmail(email)) {
// Deliberately vague — don't help enumerate valid emails. Just say
// "ok" even on bad input so the response shape is the same.
return res.json({ ok: true });
}
const publicUrl = await getPublicUrl();
if (!publicUrl) {
console.error(
"[auth] /auth/request-link blocked: recap_public_url not set. Run the 'Set Recap Public URL' StartOS action first.",
);
return res.status(503).json({
error: "public_url_not_set",
message:
"This Recap instance hasn't been fully configured yet. Ask the operator to set the public URL.",
});
}
if (!isSmtpReady()) {
console.error(
"[auth] /auth/request-link blocked: SMTP not ready. Configure StartOS System SMTP.",
);
return res.status(503).json({
error: "smtp_not_ready",
message:
"This Recap instance can't send email yet. Ask the operator to configure SMTP.",
});
}
const ip = getClientIp(req);
const now = Date.now();
// Rate limits. Two buckets — email and IP. Either trips → 429 with a
// generic message that doesn't leak which limit was hit.
if (bucketCount(emailBuckets, email, now) >= MAX_LINKS_PER_EMAIL_PER_HOUR) {
return res.status(429).json({
error: "rate_limited",
message: "Too many sign-in requests for this email. Try again in an hour.",
});
}
if (ip && bucketCount(ipBuckets, ip, now) >= MAX_LINKS_PER_IP_PER_HOUR) {
return res.status(429).json({
error: "rate_limited",
message: "Too many sign-in requests from this network. Try again in an hour.",
});
}
pushBucket(emailBuckets, email, now);
if (ip) pushBucket(ipBuckets, ip, now);
// Determine intent: signin (existing user) vs signup (new email).
// Both flows are identical to the server — the field is just for
// analytics. We don't leak intent in the response.
const existing = getDb()
.prepare("SELECT id FROM users WHERE email = ?")
.get(email);
const intent = existing ? "signin" : "signup";
// Capture the trial cookie at request-link time and store it
// server-side alongside the token. At /auth/verify we'll use it
// to link the trial → user even when the magic-link click lands
// in a different browser / cookie jar than the one that requested
// the link (the iOS Private mode + in-app email webview case).
// We read it directly from req.headers.cookie because the trial
// cookie middleware may or may not have populated req.trial
// depending on path matching — being explicit here is safer.
let trialCookieId = null;
try {
const parsed = cookie.parse(req.headers?.cookie || "");
if (parsed[TRIAL_COOKIE]) trialCookieId = parsed[TRIAL_COOKIE];
} catch {
// cookie parse failures are non-fatal; fall through with null
}
// Issue + send via the shared helper. We deliberately don't
// surface send failures back to the user — the standard advice for
// magic-link auth is "always pretend we sent it" so attackers can't
// probe which emails are configured. Operator sees the error in
// logs and can investigate (usually SMTP creds wrong or Gmail
// rate-limited).
await sendSignInLink({
email,
intent,
ip,
userAgent: req.headers?.["user-agent"],
trialCookieId,
});
res.json({ ok: true });
}
async function handleVerify(req, res) {
const plaintext = String(req.query?.token || "").trim();
if (!plaintext) {
return res.status(400).send(renderErrorPage("Missing token."));
}
const tokenHash = sha256(plaintext);
const now = Date.now();
const tx = getDb().transaction(() => {
const row = getDb()
.prepare(
"SELECT * FROM magic_link_tokens WHERE token_hash = ? AND used_at IS NULL AND expires_at > ?",
)
.get(tokenHash, now);
if (!row) return { error: "invalid_or_expired" };
getDb()
.prepare("UPDATE magic_link_tokens SET used_at = ? WHERE token_hash = ?")
.run(now, tokenHash);
return { row };
});
let result;
try {
result = tx();
} catch (err) {
console.error("[auth] verify tx failed:", err);
return res.status(500).send(renderErrorPage("Internal error."));
}
if (result.error) {
return res
.status(400)
.send(
renderErrorPage(
"This sign-in link has expired or already been used. Request a fresh one.",
),
);
}
const email = result.row.email;
const ip = getClientIp(req);
const ua = clipUA(req.headers?.["user-agent"]);
// Upsert user row.
let user = getDb()
.prepare("SELECT * FROM users WHERE email = ?")
.get(email);
if (!user) {
const id = uuid();
const syntheticInstallId = uuid();
// First user ever = operator bootstrap. We could also gate this on
// "is the install fresh (no users at all)" — which is what this
// check does. Once at least one user exists, subsequent signups
// are regular tenants with is_admin = 0.
const userCountRow = getDb()
.prepare("SELECT COUNT(*) AS n FROM users")
.get();
const isAdmin = (userCountRow?.n || 0) === 0 ? 1 : 0;
getDb()
.prepare(
`INSERT INTO users
(id, email, created_at, last_signin_at, synthetic_install_id, is_admin, signup_ip, signup_user_agent)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run(id, email, now, now, syntheticInstallId, isAdmin, ip || null, ua);
user = getDb().prepare("SELECT * FROM users WHERE id = ?").get(id);
// Seed tenant_credits for new NON-ADMIN users with the operator's
// configured default. Goes into the REPLENISHABLE bucket so
// setReplenishPeriod=daily/weekly/monthly refills it on schedule.
// Admin users skip this entirely — their relay calls bill the
// operator pool (via /data/license.txt), not a local balance.
//
// Pro/Max users (license attached at creation, e.g. via the
// anon-signup-with-purchase flow) ALSO skip this — they spend
// from their license-keyed relay pool, so a tenant_credits row
// would just sit unused and confuse the admin Tenants view.
if (!isAdmin && !user.keysat_license) {
try {
const { seedSignup } = await import("./tenant-credits.js");
await seedSignup(id);
} catch (err) {
console.warn("[auth] tenant_credits seed failed:", err);
}
}
if (isAdmin) {
console.log(
`[auth] First user signed up — ${email} bootstrapped as operator (is_admin=1)`,
);
// Legacy library at /data/history/owner/ stays where it is —
// admin's scopeForRequest() returns "owner" regardless of mode,
// so single-mode and multi-mode admin both read the same path.
// No rename. Switching back to single mode keeps the operator's
// library accessible at /data/history/owner/.
}
} else {
getDb()
.prepare("UPDATE users SET last_signin_at = ? WHERE id = ?")
.run(now, user.id);
}
// Resolve which anon trial cookie to link. Two sources:
// 1. result.row.trial_cookie_id — captured server-side at
// /auth/request-link time. Survives cross-browser / in-app-
// webview magic-link clicks because it doesn't depend on the
// verify request carrying the cookie.
// 2. req.cookies[TRIAL_COOKIE] — the legacy path, used when the
// magic-link click lands in the SAME browser that did the
// anon activity (typical desktop / non-private-mode flow).
//
// Priority: token-row wins. It represents the explicit intent
// captured at signup-request time and is the more reliable
// signal. The req cookie is a fallback for old token rows from
// before the trial_cookie_id column existed, and for any
// edge-case where the request-link path didn't capture it.
//
// We also rename the trial's history folder (anon/<cookie_id>/) to
// the user's id so the summary they ran before signing up shows
// up in their library. Skipped for returning users (existing
// account) — they already have a library and we can't safely
// merge filesystem-side. The trial row still gets linked; the
// anon/<cookie_id>/ folder just stays as an orphan.
try {
let trialCookieId = result.row.trial_cookie_id || null;
if (!trialCookieId) {
const cookies = cookie.parse(req.headers?.cookie || "");
trialCookieId = cookies[TRIAL_COOKIE] || null;
}
if (trialCookieId) {
await linkToUser(trialCookieId, user.id);
if (user.created_at === user.last_signin_at) {
try {
await renameScopeDir(`anon/${trialCookieId}`, user.id);
} catch (err) {
console.warn(
"[auth] trial→user scope rename failed:",
err?.message || err,
);
}
}
}
} catch {
// best-effort; trial linking isn't on the critical signin path
}
// Issue session — shared with /auth/signin-password so cookie shape
// + sessions-row format stay identical regardless of which auth
// method got the user here.
issueSession({ userId: user.id, req, res });
res.redirect(302, "/");
}
// Shared session-issuance helper. Used by both /auth/verify (magic-link
// success path) and /auth/signin-password (password success path) so
// the cookie shape + sessions-row format stay identical.
function issueSession({ userId, req, res }) {
const sessionId = randomBytes(32).toString("base64url");
const now = Date.now();
const expiresAt = now + SESSION_MAX_AGE_MS;
const ua = clipUA(req.headers?.["user-agent"]);
const ip = getClientIp(req);
getDb()
.prepare(
`INSERT INTO sessions
(id, user_id, created_at, expires_at, last_used_at, user_agent, ip_address)
VALUES (?, ?, ?, ?, ?, ?, ?)`,
)
.run(sessionId, userId, now, expiresAt, now, ua, ip || null);
const maxAgeSeconds = Math.floor(SESSION_MAX_AGE_MS / 1000);
res.setHeader(
"Set-Cookie",
[
`${SESSION_COOKIE}=${sessionId}`,
`Max-Age=${maxAgeSeconds}`,
"Path=/",
"HttpOnly",
"SameSite=Lax",
"Secure",
].join("; "),
);
return sessionId;
}
// ── Internal sessions for the subscription background processor ──────────────
// Per-tenant subscriptions: the processor summarizes an approved auto-queue
// item by calling /api/process over loopback, which in multi mode is
// authenticated. So it mints a SHORT-LIVED real session for the item's
// owning user, sends it as the recap_session cookie, and deletes it right
// after. This reuses the real auth path — NOT a bypass: a bad/expired token
// just 401s and the item is marked failed, never an auth hole.
export function mintInternalSession(userId) {
const sessionId = randomBytes(32).toString("base64url");
const now = Date.now();
getDb()
.prepare(
`INSERT INTO sessions
(id, user_id, created_at, expires_at, last_used_at, user_agent, ip_address)
VALUES (?, ?, ?, ?, ?, ?, ?)`,
)
.run(
sessionId,
userId,
now,
now + 30 * 60 * 1000, // 30 min is plenty for one summarize run
now,
"subscription-processor",
"127.0.0.1",
);
return sessionId;
}
export function deleteInternalSession(sessionId) {
if (!sessionId) return;
try {
getDb().prepare("DELETE FROM sessions WHERE id = ?").run(sessionId);
} catch {}
}
// The operator/admin owns the "owner" scope in multi mode — resolve their
// user id so the processor can run owner-scoped items as the operator.
export function adminUserId() {
try {
const row = getDb()
.prepare("SELECT id FROM users WHERE is_admin = 1 ORDER BY created_at LIMIT 1")
.get();
return row?.id || null;
} catch {
return null;
}
}
// /auth/signin-password — accept { email, password }, verify, issue a
// session cookie. Used by the auth.html form when the user opts to
// type a password (faster than waiting on an email).
//
// We DELIBERATELY don't differentiate "no such email" vs "wrong
// password" in the error response — both return 401 with a generic
// message. This stops credential-stuffing tools from using us as an
// email-existence oracle.
//
// Rate limits: same email + IP buckets as /auth/request-link, so the
// password endpoint can't be brute-forced any faster than someone
// could spam magic-link emails.
async function handleSignInPassword(req, res) {
const email = normalizeEmail(req.body?.email);
const password = req.body?.password;
if (!email || !isPlausibleEmail(email) || typeof password !== "string") {
return res.status(401).json({
error: "bad_credentials",
message: "Email or password is wrong.",
});
}
const ip = getClientIp(req);
const now = Date.now();
if (bucketCount(emailBuckets, email, now) >= MAX_LINKS_PER_EMAIL_PER_HOUR) {
return res
.status(429)
.json({ error: "rate_limited", message: "Too many sign-in attempts." });
}
if (ip && bucketCount(ipBuckets, ip, now) >= MAX_LINKS_PER_IP_PER_HOUR) {
return res
.status(429)
.json({ error: "rate_limited", message: "Too many sign-in attempts." });
}
pushBucket(emailBuckets, email, now);
if (ip) pushBucket(ipBuckets, ip, now);
let user;
try {
user = getDb()
.prepare("SELECT * FROM users WHERE email = ?")
.get(email);
} catch (err) {
console.error("[auth] signin-password lookup failed:", err);
return res.status(500).json({ error: "internal_error" });
}
// Run scrypt even when the user doesn't exist so timing doesn't
// betray which emails are registered. The dummy verify spends the
// same ~50ms scrypt-or-so as the real path.
const stored = user?.password_hash || `${SCRYPT_PREFIX}0000$0000`;
const ok = verifyPassword(password, stored);
if (!user || !user.password_hash || !ok) {
return res
.status(401)
.json({ error: "bad_credentials", message: "Email or password is wrong." });
}
// Authenticated. Update last_signin_at and issue a session.
getDb()
.prepare("UPDATE users SET last_signin_at = ? WHERE id = ?")
.run(now, user.id);
issueSession({ userId: user.id, req, res });
res.json({ ok: true });
}
async function handleSignout(req, res) {
try {
const cookies = cookie.parse(req.headers?.cookie || "");
const sessionId = cookies[SESSION_COOKIE];
if (sessionId) {
getDb()
.prepare("DELETE FROM sessions WHERE id = ?")
.run(sessionId);
}
} catch {}
// Expire the cookie immediately.
res.setHeader(
"Set-Cookie",
`${SESSION_COOKIE}=; Max-Age=0; Path=/; HttpOnly; SameSite=Lax; Secure`,
);
// Honor JSON callers (UI fetch) vs link callers (form submit).
if (req.accepts?.("json") && !req.accepts?.("html")) {
return res.status(204).end();
}
res.redirect(302, "/");
}
// Self-contained branded error page used when /auth/verify fails.
// Doesn't pull in the full app shell so it works even if the static
// bundle is broken or partially served. Visual style matches /auth.html
// (same dark-glass card, same primary-button accent) so the user feels
// they're still in the same product flow.
function renderErrorPage(message) {
const safe = String(message)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;");
return `<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
<title>Sign-in failed · Recaps</title>
<link rel="icon" type="image/png" href="/assets/icon.png">
<meta name="theme-color" content="#0a0e1a">
<style>
*{box-sizing:border-box;margin:0;padding:0}
html,body{height:100%}
body{
font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;
background:#0a0e1a;color:#e2e8f0;
display:flex;align-items:center;justify-content:center;padding:24px;
}
.card{
width:100%;max-width:420px;background:#121828;
border:1px solid #1f2942;border-radius:12px;padding:32px 28px;
}
.logo{display:flex;align-items:center;gap:12px;margin-bottom:24px}
.logo img{width:32px;height:32px;border-radius:6px}
.logo span{font-size:18px;font-weight:600;color:#f5f9ff}
h1{font-size:20px;font-weight:600;color:#f5f9ff;margin-bottom:10px}
p.msg{font-size:14px;line-height:1.55;color:#cbd5e1;margin-bottom:22px}
.btn-row{display:flex;gap:8px;flex-wrap:wrap}
a.btn-primary{
display:inline-block;background:#3b82f6;color:#fff;text-decoration:none;
font-size:14px;font-weight:600;padding:10px 18px;border-radius:8px;
}
a.btn-primary:hover{background:#2563eb}
a.btn-secondary{
display:inline-block;background:transparent;color:#94a3b8;text-decoration:none;
font-size:14px;font-weight:500;padding:10px 14px;border-radius:8px;border:1px solid #334155;
}
a.btn-secondary:hover{color:#cbd5e1;border-color:#475569}
</style>
</head>
<body>
<div class="card">
<div class="logo">
<img src="/assets/icon.png" alt="Recaps" onerror="this.style.display='none'">
<span>Recaps</span>
</div>
<h1>Sign-in didn't work</h1>
<p class="msg">${safe}</p>
<div class="btn-row">
<a href="/auth.html" class="btn-primary">Get a new sign-in link</a>
<a href="/" class="btn-secondary">Back to Recaps</a>
</div>
</div>
</body>
</html>`;
}
// sendSignInLink({ email, intent, ip?, userAgent?, emailBody? }) —
// reusable magic-link issuance + email send. Used by:
// • /auth/request-link — visitor-initiated sign-in
// • license-purchase poll-settle — system-initiated post-purchase
// "your account is ready" send
//
// Generates a 32-byte token, hashes it, stores the hash in
// magic_link_tokens, builds a verifyUrl, sends the email with either
// the default magic-link body OR a caller-supplied (subject, text,
// html) tuple for custom flows. Returns { ok: true, expires_at } on
// success; { ok: false, error, message? } on failure.
//
// Doesn't enforce rate limits — that's the caller's job. /auth/request-link
// has the per-email + per-IP buckets; the post-purchase path is
// inherently rate-limited by the actual payment, so no extra bucket
// needed.
export async function sendSignInLink({
email,
intent = "signin",
ip = null,
userAgent = "",
emailBody = null,
trialCookieId = null,
}) {
if (!email || !isPlausibleEmail(email)) {
return { ok: false, error: "bad_email" };
}
const publicUrl = await getPublicUrl();
if (!publicUrl) {
return { ok: false, error: "public_url_not_set" };
}
if (!isSmtpReady()) {
return { ok: false, error: "smtp_not_ready" };
}
const now = Date.now();
const plaintext = randomBytes(32).toString("base64url");
const tokenHash = sha256(plaintext);
const expiresAt = now + MAGIC_LINK_TTL_MS;
try {
getDb()
.prepare(
`INSERT INTO magic_link_tokens
(token_hash, email, created_at, expires_at, intent, request_ip, request_ua, trial_cookie_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
)
.run(
tokenHash,
email,
now,
expiresAt,
intent,
ip || null,
clipUA(userAgent),
trialCookieId || null,
);
} catch (err) {
console.error("[auth] sendSignInLink insert failed:", err);
return { ok: false, error: "internal_error" };
}
const verifyUrl = `${publicUrl}/auth/verify?token=${encodeURIComponent(plaintext)}`;
// Default to the standard sign-in email body; callers can override
// either with a pre-built {subject,text,html} object OR a function
// that receives the verifyUrl and returns that shape. The function
// form is what license-purchase uses to inject the celebratory
// "your Pro account is ready" copy with the verifyUrl pre-rendered
// into the body.
let message;
if (typeof emailBody === "function") {
message = emailBody(verifyUrl);
} else if (emailBody && typeof emailBody === "object") {
message = emailBody;
} else {
message = renderMagicLinkEmail({
verifyUrl,
brandName: "Recaps",
expiresInMinutes: 15,
});
}
try {
await sendMail({
to: email,
subject: message.subject,
text: message.text,
html: message.html,
});
} catch (err) {
console.error(
"[auth] sendSignInLink sendMail failed for",
email,
":",
err?.message || err,
);
// The token is already inserted; if the operator's SMTP is flaky
// the user can re-request a link. Return error so the caller can
// decide whether to surface it or swallow.
return { ok: false, error: "send_failed" };
}
return { ok: true, expires_at: expiresAt };
}
// setupAuthRoutes(app) — registers /auth/* endpoints. Multi-mode only;
// wired in server/index.js behind the RECAP_MODE === 'multi' branch.
//
// Magic-link is the primary auth surface:
// POST /auth/request-link — issue a magic link by email
// GET /auth/verify?token=... — consume the link, issue session
// POST /auth/signout — drop the session
// GET /auth/signout — same (link-click convenience)
//
// Password endpoints are the optional faster-signin add-on:
// POST /auth/set-password — set OR overwrite my password
// POST /auth/clear-password — remove my password (magic-link only)
// POST /auth/signin-password — sign in with email + password
//
// Note there is no /auth/reset-password endpoint by design — reset is
// implemented as "request a magic link, sign in, then call
// /auth/set-password with the new one." Adding a dedicated reset
// endpoint would duplicate the magic-link flow without adding any
// security or UX.
export function setupAuthRoutes(app) {
app.post("/auth/request-link", (req, res) => {
handleRequestLink(req, res).catch((err) => {
console.error("[auth] /auth/request-link unhandled:", err);
res.status(500).json({ error: "internal_error" });
});
});
app.get("/auth/verify", (req, res) => {
handleVerify(req, res).catch((err) => {
console.error("[auth] /auth/verify unhandled:", err);
res.status(500).send(renderErrorPage("Internal error."));
});
});
app.post("/auth/signin-password", (req, res) => {
handleSignInPassword(req, res).catch((err) => {
console.error("[auth] /auth/signin-password unhandled:", err);
res.status(500).json({ error: "internal_error" });
});
});
// Note: /api/account/password (set + clear) is registered by
// account-routes.js, not here — those endpoints REQUIRE an existing
// session, so they live outside the /auth/* public-path namespace
// (which is allowed through the tenant-auth middleware unauthenticated).
app.post("/auth/signout", (req, res) => {
handleSignout(req, res).catch((err) => {
console.error("[auth] /auth/signout unhandled:", err);
res.status(500).json({ error: "internal_error" });
});
});
// Convenience GET version for plain link clicks ("Sign out") from
// the UI without needing a form POST.
app.get("/auth/signout", (req, res) => {
handleSignout(req, res).catch((err) => {
console.error("[auth] /auth/signout unhandled:", err);
res.status(500).send(renderErrorPage("Internal error."));
});
});
}
+252
View File
@@ -0,0 +1,252 @@
// Self-serve subscription purchase (multi-mode / cloud only).
//
// Lets a signed-in cloud user buy their OWN prepaid Pro/Max period
// instead of waiting for the operator to grant it by hand. The relay
// owns the subscription (keyed by Recaps user-id, per the core-
// decoupling); Recaps just brokers the purchase:
//
// POST /api/billing/buy → ask the relay to mint a BTCPay invoice
// for {tier}; return the checkout URL the
// frontend opens. On settlement the relay's
// webhook extends the user's tier.
// GET /api/billing/status → pull the user's current (expiry-enforced)
// tier from the relay and refresh the local
// users.tier cache so the badge flips the
// moment payment lands. The frontend polls
// this after opening checkout.
//
// Auth: both routes require a real signed-in user (req.user.id). Anon /
// trial visitors (req.userId = "anon:<cookie>") are refused — a tier is
// keyed to a durable user-id, which a trial cookie isn't.
//
// These live under /api/billing (NOT /api/subscriptions — that prefix is
// the channel-subscriptions feature, which is itself Pro-gated; a free
// user must be able to reach the buy flow). The prefix is added to the
// license middleware's open list so the activation gate lets Core users
// through to purchase.
import { getDb } from "./db.js";
import { requireUser } from "./tenant-auth.js";
import {
createRelayTierInvoice,
createRelayZapriteOrder,
getRelayUserTier,
getRelayTierPlans,
} from "./providers/relay.js";
const BUYABLE_TIERS = new Set(["pro", "max"]);
const PAYMENT_METHODS = new Set(["bitcoin", "card"]);
// Fallback prices (sats / 30-day period) used only when the relay is
// unreachable while rendering the picker — matches the relay config
// defaults so the UI never shows a blank price. The actual charge is
// always computed relay-side at invoice time.
const FALLBACK_PLANS = {
period_days: 30,
plans: [
{ tier: "pro", sats: 21000 },
{ tier: "max", sats: 42000 },
],
};
// Pull the user's effective (expiry-enforced) tier from the relay — the
// authoritative subscription owner — and update the cached users.tier if
// it drifted. Returns { tier, expires_at, synced } or { synced:false }
// when the relay is unreachable / unconfigured (caller falls back to the
// cached value rather than erroring the request).
export async function syncUserTierFromRelay(userId) {
if (!userId) return { synced: false };
let report;
try {
report = await getRelayUserTier({ userId });
} catch (err) {
console.warn(
`[billing] relay tier read failed for ${userId}: ${err?.message || err}`,
);
return { synced: false };
}
// getRelayUserTier swallows errors and returns null when the relay
// base URL / operator key isn't configured. Treat that as "can't
// sync" rather than "downgrade to core".
if (!report || typeof report.tier !== "string") {
return { synced: false };
}
const tier = report.tier; // already expiry-enforced by the relay
const expiresAt = report.subscription_expires_at || null;
try {
const db = getDb();
const row = db.prepare("SELECT tier FROM users WHERE id = ?").get(userId);
if (row && row.tier !== tier) {
db.prepare("UPDATE users SET tier = ? WHERE id = ?").run(tier, userId);
console.log(
`[billing] synced ${userId} tier ${row.tier || "core"}${tier} from relay`,
);
}
} catch (err) {
console.warn(
`[billing] tier cache update failed for ${userId}: ${err?.message || err}`,
);
}
return {
tier,
expires_at: expiresAt,
tier_snapshot: report.tier_snapshot || tier,
subscription_expired: !!report.subscription_expired,
synced: true,
};
}
// Build the buyer-facing origin so the BTCPay checkout can redirect back
// to the app after settlement. Honors the reverse-proxy forwarding
// headers StartOS sets in front of the service.
function originFor(req) {
const proto =
(req.headers["x-forwarded-proto"] || "").split(",")[0].trim() ||
req.protocol ||
"https";
const host = req.headers["x-forwarded-host"] || req.headers.host || "";
return host ? `${proto}://${host}` : "";
}
export function setupBillingRoutes(app) {
// GET /api/billing/plans → { period_days, plans: [{tier, sats}] }
// Powers the purchase picker. Prices come from the relay (the pricing
// source of truth); falls back to the config defaults if the relay is
// briefly unreachable so the modal still renders.
app.get("/api/billing/plans", requireUser, async (req, res) => {
if (!req.user || !req.user.id) {
return res.status(403).json({ error: "must_be_signed_in" });
}
try {
const data = await getRelayTierPlans();
if (data && Array.isArray(data.plans) && data.plans.length) {
return res.json({
period_days: data.period_days || FALLBACK_PLANS.period_days,
plans: data.plans,
// Whether the card (Zaprite) rail is configured — the UI hides
// the "Pay by card" link when false so it never offers a rail
// that 503s. Bitcoin is always available (the BTCPay rail).
card_available: !!data.card_available,
source: "relay",
});
}
} catch (err) {
console.warn(`[billing] plans read failed: ${err?.message || err}`);
}
return res.json({ ...FALLBACK_PLANS, card_available: false, source: "fallback" });
});
// POST /api/billing/buy body: { tier: "pro" | "max", method?: "bitcoin" | "card" }
// Bitcoin (default) → BTCPay invoice; card → Zaprite hosted checkout.
// Returns { ok, method, checkout_url, tier, period_days, ... }.
app.post("/api/billing/buy", requireUser, async (req, res) => {
// Must be a real signed-in user — a tier is keyed to a durable
// user-id, not an anon trial cookie.
if (!req.user || !req.user.id) {
return res.status(403).json({
error: "must_be_signed_in",
message: "Sign in to buy a subscription.",
});
}
const tier = String(req.body?.tier || "").trim().toLowerCase();
if (!BUYABLE_TIERS.has(tier)) {
return res.status(400).json({
error: "bad_tier",
message: 'tier must be "pro" or "max"',
});
}
const method = String(req.body?.method || "bitcoin").trim().toLowerCase();
if (!PAYMENT_METHODS.has(method)) {
return res.status(400).json({
error: "bad_method",
message: 'method must be "bitcoin" or "card"',
});
}
const origin = originFor(req);
// Land back on the app with a marker the frontend uses to kick an
// immediate status sync (the modal also polls, so this is a courtesy
// for buyers who follow the checkout redirect).
const returnUrl = origin ? `${origin}/?billing=success` : null;
try {
if (method === "card") {
const order = await createRelayZapriteOrder({
userId: req.user.id,
tier,
returnUrl,
});
return res.json({
ok: true,
method: "card",
checkout_url: order.checkout_url || null,
order_id: order.order_id || null,
amount: order.amount ?? null,
currency: order.currency || null,
tier: order.tier || tier,
period_days: order.period_days ?? null,
});
}
// Bitcoin (default) — BTCPay invoice.
const invoice = await createRelayTierInvoice({
userId: req.user.id,
tier,
returnUrl,
});
res.json({
ok: true,
method: "bitcoin",
checkout_url: invoice.checkout_url || null,
invoice_id: invoice.invoice_id || null,
sats: invoice.sats ?? null,
tier: invoice.tier || tier,
period_days: invoice.period_days ?? null,
// Lightning BOLT11 for the inline QR (no redirect). Null → the app
// falls back to opening the hosted checkout_url.
bolt11: invoice.bolt11 || null,
lightning_payment_link: invoice.lightning_payment_link || null,
lightning_expires_at: invoice.lightning_expires_at || null,
});
} catch (err) {
const status = err?.status || 502;
console.error(
`[billing] buy failed for ${req.user.id} (${tier}/${method}): ${err?.message || err}`,
);
// 503 from the relay = that rail isn't configured; surface a hint.
const notConfigured =
status === 503 || /not[_ ]configured/i.test(err?.message || "");
const rail = method === "card" ? "Card" : "Bitcoin";
const tool = method === "card" ? "Zaprite" : "BTCPay";
res.status(notConfigured ? 503 : 502).json({
error: notConfigured ? "payments_unavailable" : "checkout_failed",
message: notConfigured
? `${rail} payments aren't set up on this server yet. Ask the operator to configure ${tool}.`
: "Couldn't start the payment. Please try again in a moment.",
});
}
});
// GET /api/billing/status
// Returns { tier, expires_at, synced } — the user's current relay-owned
// tier, with the local cache refreshed as a side effect.
app.get("/api/billing/status", requireUser, async (req, res) => {
if (!req.user || !req.user.id) {
return res.status(403).json({ error: "must_be_signed_in" });
}
const synced = await syncUserTierFromRelay(req.user.id);
if (synced.synced) {
return res.json({
tier: synced.tier,
expires_at: synced.expires_at,
tier_snapshot: synced.tier_snapshot,
subscription_expired: synced.subscription_expired,
synced: true,
});
}
// Relay unreachable / unconfigured — fall back to the cached tier so
// the UI still renders a sane badge instead of erroring.
return res.json({
tier: req.user.tier || "core",
expires_at: null,
synced: false,
});
});
}
+487
View File
@@ -0,0 +1,487 @@
// Chunked topic-analysis: split a long transcript into overlapping
// time-windowed slices, analyze each slice in parallel, stitch the
// returned sections back into one coherent list.
//
// Why: a single-shot analyze call against a 2-hour transcript spends
// most of its wall-time on prefill (typically 25K+ tokens). Splitting
// into 18-min slices gives the model a much smaller prompt per call,
// and firing the slices concurrently lets the backend (relay/vLLM or
// Gemini) batch them. End-to-end wall-time drops from minutes to
// tens of seconds for long content, with no quality regression as
// long as the slice boundaries are chosen with overlap and the
// stitcher trusts the second slice for the overlap region.
//
// Public entry point: runChunkedAnalysis().
import { buildAnalysisPrompt } from "./gemini-helpers.js";
// ── Tunables ────────────────────────────────────────────────────────────────
// Window body: the part of a chunk that "owns" its topic boundaries.
// Overlap: a tail appended to each window so a topic spanning a
// boundary still gets seen in full by at least one window.
// Stride = body. Windows advance by `body` seconds; each window
// covers `body + overlap` seconds of audio.
const WINDOW_BODY_SECONDS = 18 * 60; // 18 min
const WINDOW_OVERLAP_SECONDS = 2 * 60; // 2 min
// Don't chunk below this duration. A single analyze call against
// <25 min is fast on its own and avoids the stitching complexity
// for the common short-content case.
// Exported so the orchestrator can mirror the decision when picking
// whether to coalesce: above this duration the chunker handles
// granularity per-window, so the pre-chunk coalesce is unnecessary
// and would hurt section-boundary precision.
export const CHUNKING_CUTOFF_SECONDS = 25 * 60; // 25 min
// Max concurrent analyze calls in flight. Gemini paid Tier 1 allows
// ~1000 RPM for flash and ~150 RPM for pro — 12 in-flight is well
// under either ceiling and saturates most operator workloads
// without queueing. Operator hardware (vLLM on a single Spark) caps
// out around 8-12 concurrent for our prompt size, so 12 is a
// reasonable cross-backend default.
const DEFAULT_CONCURRENCY = 12;
// ── Window planning ─────────────────────────────────────────────────────────
// Plans a set of overlapping windows over the entries array. Each
// window has:
// - startIdx, endIdx: inclusive bounds into the entries array
// - bodyStartIdx: index where this window's "body" begins
// (i.e., everything before this index is the
// overlap with the previous window's tail)
// The first window has bodyStartIdx === startIdx. Windows after the
// first have bodyStartIdx > startIdx by ~overlap seconds.
//
// The stitcher uses bodyStartIdx of window N+1 to decide whether a
// section from window N falls in the contested overlap region.
export function planAnalysisWindows(entries, opts = {}) {
const bodySec = opts.bodySeconds ?? WINDOW_BODY_SECONDS;
const overlapSec = opts.overlapSeconds ?? WINDOW_OVERLAP_SECONDS;
const totalSec = (entries[entries.length - 1].offset || 0) +
(entries[entries.length - 1].duration || 0);
const cutoffSec = opts.cutoffSeconds ?? CHUNKING_CUTOFF_SECONDS;
if (totalSec <= cutoffSec) {
return [{ startIdx: 0, endIdx: entries.length - 1, bodyStartIdx: 0 }];
}
const windows = [];
let bodyStartSec = 0;
while (bodyStartSec < totalSec) {
// The window's covered span (body + tail overlap):
const windowEndSec = bodyStartSec + bodySec + overlapSec;
// Body start in entry-index space: first entry with offset >= bodyStartSec.
const bodyStartIdx = firstEntryAtOrAfter(entries, bodyStartSec);
// If there are NO entries at or after bodyStartSec, we've consumed
// all entries. Stop the loop.
if (bodyStartIdx >= entries.length) break;
// GAP HANDLING: if the next entry after bodyStartSec is far in
// the future (past this window's body + overlap), there's a gap
// in the transcript timeline. This commonly happens when the
// transcribe step truncated a middle chunk — the timeline has
// valid entries at, e.g., 0-31 min and 90-94 min but nothing in
// between. Without this fix, the old loop would BREAK at the gap
// (because endIdx < bodyStartIdx triggered the "sparse trailing
// window" exit), silently dropping the entries past the gap from
// analysis entirely. Now we jump bodyStartSec forward to the
// next entry's offset (rounded down to a body-stride boundary
// so subsequent window alignment stays sensible) and continue.
const nextEntryOffset = entries[bodyStartIdx].offset || 0;
if (nextEntryOffset >= windowEndSec) {
bodyStartSec = Math.max(
bodyStartSec + bodySec,
Math.floor(nextEntryOffset / bodySec) * bodySec
);
continue;
}
// Window's entry range: from the start of overlap-with-prior
// (i.e., bodyStartSec - overlapSec, clamped at 0) through windowEndSec.
const overlapWithPriorSec = Math.max(0, bodyStartSec - overlapSec);
const startIdx = firstEntryAtOrAfter(entries, overlapWithPriorSec);
const endIdx = lastEntryBefore(entries, windowEndSec);
if (endIdx < bodyStartIdx) {
// Defensive: shouldn't happen with the gap-handling above, but
// if it does, advance the body cursor rather than break so we
// don't get stuck.
bodyStartSec += bodySec;
continue;
}
windows.push({ startIdx, endIdx, bodyStartIdx });
// Stop if this window already covers the last entry.
if (endIdx >= entries.length - 1) break;
bodyStartSec += bodySec;
}
return windows;
}
function firstEntryAtOrAfter(entries, sec) {
// Linear scan; entries are sorted by offset.
for (let i = 0; i < entries.length; i++) {
if ((entries[i].offset || 0) >= sec) return i;
}
return entries.length;
}
function lastEntryBefore(entries, sec) {
// Largest i s.t. entries[i].offset < sec.
let ans = -1;
for (let i = 0; i < entries.length; i++) {
if ((entries[i].offset || 0) < sec) ans = i;
else break;
}
// If no entry has offset < sec, return -1 → caller treats as empty.
// If the whole array fits, return entries.length - 1.
return ans === -1 ? -1 : ans;
}
// ── Parallel analyzer ───────────────────────────────────────────────────────
// Fires N analyze calls concurrently with a bounded in-flight count.
// Each call gets its own slice of entries plus a freshly-built prompt.
// Returns array of { window, ok, sections | error, cost, model }.
//
// Errors are isolated per window — a single-window failure doesn't
// fail the whole batch. The stitcher gets to decide what to do
// about gaps.
async function analyzeWindowsInParallel({
entries,
windows,
analyzer,
fallbackModels,
concurrency,
onProgress,
onWindowComplete,
signal,
jobId,
// Total audio duration in seconds — passed through to
// buildAnalysisPrompt so the section-count target scales with the
// full video length (not just per-window). Recap-relay does the
// same; matching here keeps segmentation density consistent
// across both pipelines. When omitted, buildAnalysisPrompt falls
// back to deriving from the entries themselves.
totalAudioSec = 0,
}) {
const results = new Array(windows.length);
let next = 0;
let completed = 0;
async function worker() {
while (true) {
if (signal?.aborted) return;
const my = next++;
if (my >= windows.length) return;
const w = windows[my];
const windowEntries = entries.slice(w.startIdx, w.endIdx + 1);
const prompt = buildAnalysisPrompt(windowEntries, { totalAudioSec });
// Try the configured model first, then walk fallbacks.
let lastErr = null;
let result = null;
let usedModel = null;
for (const tryModel of fallbackModels) {
try {
result = await analyzer.analyzeText({
prompt,
model: tryModel,
onProgress: () => {}, // suppress per-chunk progress noise
signal,
jobId,
});
usedModel = tryModel;
break;
} catch (err) {
if (signal?.aborted) return;
lastErr = err;
}
}
if (!result) {
results[my] = { window: w, ok: false, error: lastErr };
completed++;
onProgress?.(`Window ${my + 1}/${windows.length} failed: ${lastErr?.message?.slice(0, 100) || "unknown"}`);
continue;
}
const parsed = safeParseSections(result.text);
if (!parsed) {
results[my] = { window: w, ok: false, error: new Error("invalid JSON") };
completed++;
onProgress?.(`Window ${my + 1}/${windows.length} returned invalid JSON`);
continue;
}
results[my] = {
window: w,
ok: true,
sections: parsed.sections,
model: usedModel,
cost: result.cost,
};
completed++;
onProgress?.(`Window ${my + 1}/${windows.length} done (${parsed.sections.length} topics)`);
// Fire the streaming callback with this window's BODY-OWNED
// sections — the ones the final stitcher will keep from this
// window. Computed deterministically per-window so the UI can
// render incrementally as windows arrive (even out of order),
// without later having to "undo" any displayed sections.
//
// Rule: window N owns sections whose globalStart falls before
// window(N+1).bodyStartIdx. Sections starting at or after the
// next window's body are deferred — window N+1 will produce an
// authoritative version of them with more downstream context.
if (onWindowComplete) {
const nextBody = my + 1 < windows.length
? windows[my + 1].bodyStartIdx
: Infinity;
const offset = w.startIdx;
const owned = [];
for (const s of parsed.sections) {
const globalStart = offset + (s.startIndex ?? 0);
const globalEnd = offset + (s.endIndex ?? 0);
if (globalStart >= nextBody) continue;
owned.push({
startIndex: globalStart,
endIndex: globalEnd,
title: s.title,
summary: s.summary,
});
}
try {
await onWindowComplete({
windowIdx: my,
totalWindows: windows.length,
ownedSections: owned,
});
} catch (cbErr) {
// Callback errors must not derail the analyze loop —
// streaming is best-effort and the canonical result still
// ships at the end.
console.warn(
`[chunked-analyze] onWindowComplete callback failed: ${cbErr?.message || cbErr}`
);
}
}
}
}
const workers = Array.from({ length: Math.min(concurrency, windows.length) }, worker);
await Promise.all(workers);
return results;
}
function safeParseSections(text) {
if (!text) return null;
let jsonStr = text.trim();
const cb = jsonStr.match(/```(?:json)?\s*([\s\S]*?)```/);
if (cb) jsonStr = cb[1].trim();
try {
const parsed = JSON.parse(jsonStr);
if (!parsed || !Array.isArray(parsed.sections)) return null;
return parsed;
} catch {
return null;
}
}
// ── Stitcher ────────────────────────────────────────────────────────────────
// Merges per-window section lists into a single ordered list of
// non-overlapping sections referencing entries by their position in
// the FULL (un-chunked) entries array.
//
// The rule: each window N owns sections whose globalStart falls in
// its body (i.e., globalStart < window(N+1).bodyStartIdx). Any
// section starting at or after the next window's body boundary is
// dropped because the next window will have produced a better
// version of that same section with more downstream context. The
// last window has no successor, so all its sections are kept.
//
// After collection, sections are sorted and any residual overlap
// (which shouldn't happen if windows are well-formed but might
// arise from model index errors) is repaired by clamping endIndex
// to the next section's startIndex - 1.
export function stitchAnalysisResults(results) {
const out = [];
for (let i = 0; i < results.length; i++) {
const r = results[i];
if (!r || !r.ok) continue;
const next = results[i + 1];
const nextBody = next && next.window
? next.window.bodyStartIdx
: Infinity;
const offset = r.window.startIdx;
for (const s of r.sections) {
const globalStart = offset + (s.startIndex ?? 0);
const globalEnd = offset + (s.endIndex ?? 0);
// Drop sections that begin in the next window's body — the
// next window's analysis is authoritative for that range.
if (globalStart >= nextBody) continue;
out.push({
startIndex: globalStart,
endIndex: globalEnd,
title: s.title,
summary: s.summary,
});
}
}
// Order + repair overlaps (defensive — shouldn't trigger with
// well-behaved model output, but the existing single-shot path
// doesn't either and this matches its robustness).
out.sort((a, b) => a.startIndex - b.startIndex);
for (let i = 0; i < out.length - 1; i++) {
if (out[i].endIndex >= out[i + 1].startIndex) {
out[i].endIndex = out[i + 1].startIndex - 1;
}
}
return out.filter((s) => s.endIndex >= s.startIndex);
}
// ── Public entry point ──────────────────────────────────────────────────────
// Runs chunked analysis end-to-end. Returns the same envelope shape
// callers expect from a single-shot analyzer.analyzeText() call:
// {
// text: "<JSON string with .sections>", // for prompt/result parity
// model: "<which model served the most windows>",
// cost: { total cost across all windows, summed },
// usage: null, // no aggregate usage
// attempts: { windows: N, failed: K } // diagnostic
// }
// The caller parses .text the same way it parses a single-shot
// response — no changes to the downstream chunk-building code.
//
// Falls back to single-shot if planning produces just one window
// (i.e., content is below the chunking cutoff). If all windows fail,
// throws so the caller's existing fallback (try next model) kicks in.
export async function runChunkedAnalysis({
entries,
analyzer,
fallbackModels,
concurrency = DEFAULT_CONCURRENCY,
onProgress = () => {},
onWindowComplete = null,
signal,
jobId,
}) {
const windows = planAnalysisWindows(entries);
if (windows.length === 1) {
// Single-shot path — same as the legacy code does, but routed
// through here so callers have one entry point. Log message
// distinguishes the two reasons we end up here:
// (a) totalSec ≤ cutoff — short content, intentionally not chunked
// (b) entries are too sparse for multi-window planning — the loop
// broke after one window. Surfaces an awkward state that's
// usually a sign of bad upstream data (e.g. transcribe emitted
// bogus far-future timestamps that the sanity-cap dropped).
const lastEntry = entries[entries.length - 1];
const totalSec = (lastEntry?.offset || 0) + (lastEntry?.duration || 0);
if (totalSec <= CHUNKING_CUTOFF_SECONDS) {
onProgress(
`Content ≤${Math.round(CHUNKING_CUTOFF_SECONDS / 60)} min — running single-shot analysis`
);
} else {
onProgress(
`Single window planned over ${entries.length} entries (last @ ${Math.round(totalSec / 60)} min) — running single-shot analysis`
);
}
return await runSingleShot({
entries,
analyzer,
fallbackModels,
onProgress,
signal,
jobId,
});
}
onProgress(
`Chunked analysis: ${windows.length} windows of ~18 min each, up to ${concurrency} in parallel`
);
// Compute total audio duration from the last entry's offset so the
// section-count target (in buildAnalysisPrompt) scales with the
// FULL video length, not just per-window. Matches recap-relay's
// per-video-duration target methodology for consistent segmentation
// density across both pipelines.
const totalAudioSec = entries.length > 0
? (entries[entries.length - 1].offset || 0) + (entries[entries.length - 1].duration || 0)
: 0;
const results = await analyzeWindowsInParallel({
entries,
windows,
analyzer,
fallbackModels,
concurrency,
onProgress,
onWindowComplete,
signal,
jobId,
totalAudioSec,
});
// If the caller aborted mid-flight, some result slots may be empty.
// Surface cancellation cleanly to the outer pipeline.
if (signal?.aborted) {
const e = new Error("aborted");
e.name = "AbortError";
throw e;
}
const completed = results.filter(Boolean);
const failures = completed.filter((r) => !r.ok);
if (completed.length === 0 || failures.length === completed.length) {
throw new Error(
`All ${results.length} analyze windows failed. First error: ${
failures[0]?.error?.message || "unknown"
}`
);
}
const stitched = stitchAnalysisResults(results);
// Aggregate model attribution: pick the most-used successful model.
const modelTally = new Map();
let totalCost = 0;
for (const r of results) {
if (!r.ok) continue;
modelTally.set(r.model, (modelTally.get(r.model) || 0) + 1);
const c = typeof r.cost?.totalCost === "string"
? parseFloat(r.cost.totalCost)
: r.cost?.totalCost || 0;
if (Number.isFinite(c)) totalCost += c;
}
const dominantModel = [...modelTally.entries()].sort((a, b) => b[1] - a[1])[0]?.[0] || null;
onProgress(
`Chunked analysis complete — ${results.length - failures.length}/${results.length} windows succeeded, ${stitched.length} topics`
);
return {
text: JSON.stringify({ sections: stitched }),
model: dominantModel,
cost: {
totalCost: totalCost.toFixed(6),
totalCostDisplay: totalCost < 0.01
? `$${(totalCost * 100).toFixed(3)}¢`
: `$${totalCost.toFixed(4)}`,
},
usage: null,
attempts: { windows: results.length, failed: failures.length },
};
}
async function runSingleShot({
entries,
analyzer,
fallbackModels,
onProgress,
signal,
jobId,
}) {
// Single-shot path: the whole transcript IS the "window". Compute
// totalAudioSec from the entries so the section-count target picker
// chooses the right bucket (<30 min → 6 sections, 30-60 → 8, etc.).
const totalAudioSec = entries.length > 0
? (entries[entries.length - 1].offset || 0) + (entries[entries.length - 1].duration || 0)
: 0;
const prompt = buildAnalysisPrompt(entries, { totalAudioSec });
let lastErr = null;
for (const tryModel of fallbackModels) {
try {
const result = await analyzer.analyzeText({
prompt,
model: tryModel,
onProgress,
signal,
jobId,
});
return result;
} catch (err) {
if (signal?.aborted) throw err;
lastErr = err;
}
}
throw lastErr || new Error("All analysis models failed");
}
+80
View File
@@ -25,6 +25,14 @@ let startosConfigPath = null;
export let serverApiKey = "";
// Core-decoupling shared "operator key" — read live from the StartOS
// config sidecar the same way serverApiKey is, so the operator can set it
// via the "Set Relay Operator Key" action without a service restart.
// `RECAP_RELAY_OPERATOR_KEY` env pins the value (local dev). Consumed by
// relay-default.js's getRelayOperatorKey(); see that for the semantics.
let envRelayOperatorKey = "";
export let relayOperatorKey = "";
// ── Init ────────────────────────────────────────────────────────────────────
// Call once at boot. Sets up paths, reads the initial value, kicks off the
// poll loop. Idempotent if you really want to call it twice (the interval
@@ -35,13 +43,17 @@ export async function initConfig({ dataDir }) {
startosConfigPath = path.join(configDir, "startos-config.json");
envApiKey = process.env.GEMINI_API_KEY || "";
serverApiKey = envApiKey;
envRelayOperatorKey = (process.env.RECAP_RELAY_OPERATOR_KEY || "").trim();
relayOperatorKey = envRelayOperatorKey;
await fs.mkdir(configDir, { recursive: true }).catch(() => {});
await refreshServerApiKey("startup");
await refreshRelayOperatorKey("startup");
const pollMs = parseInt(process.env.RECAP_CONFIG_POLL_MS || "3000", 10);
setInterval(() => {
refreshServerApiKey("config poll").catch(() => {});
refreshRelayOperatorKey("config poll").catch(() => {});
}, pollMs);
}
@@ -75,6 +87,27 @@ async function refreshServerApiKey(reason) {
}
}
async function readRelayOperatorKeyFromConfig() {
try {
const content = await fs.readFile(startosConfigPath, "utf-8");
const config = JSON.parse(content);
return (config.recap_relay_operator_key || "").trim();
} catch {
return "";
}
}
async function refreshRelayOperatorKey(reason) {
if (envRelayOperatorKey) return; // env var pins the value
const next = await readRelayOperatorKeyFromConfig();
if (next !== relayOperatorKey) {
relayOperatorKey = next;
console.log(
`[config] relay operator key ${next ? "loaded" : "cleared"} (${reason})`,
);
}
}
// ── Public helpers ──────────────────────────────────────────────────────────
// Resolves the per-request key — either the client's own (BYO) or the
// server's stored key (when the client signals USE_SERVER_KEY or sends
@@ -89,3 +122,50 @@ export function resolveApiKey(clientKey) {
export function getEnvPath() {
return envPath;
}
// Snapshot of the full StartOS config blob — keys for every provider
// (gemini, anthropic, openai, openai-compatible, ollama) plus the
// admin-auth fields. Each request reads it once and passes it into
// resolveProviderOpts() per provider. Returns {} if the file doesn't
// exist or is unreadable.
export async function getConfigSnapshot() {
try {
const content = await fs.readFile(startosConfigPath, "utf-8");
return JSON.parse(content) || {};
} catch {
return {};
}
}
// Patch the StartOS config file in place. Reads current, merges in the
// given fields, writes atomically (tmp + rename). Used by the picker
// UI's Delete button to clear server-side credentials for a provider.
// The next config poll picks up the changes within CONFIG_POLL_MS;
// resolveProviderOpts already reads getConfigSnapshot per-request, so
// effectively the change is immediate.
//
// `patch` is a plain object of { config_field: value } pairs.
// Pass empty strings to clear a field rather than deleting the key —
// the StartOS schema declares every field with a default of '', so
// empty string is the canonical "unset" representation.
export async function mergeConfig(patch) {
if (!patch || typeof patch !== "object") return;
let current = {};
try {
const content = await fs.readFile(startosConfigPath, "utf-8");
current = JSON.parse(content) || {};
} catch {}
const merged = { ...current, ...patch };
const tmp = startosConfigPath + ".tmp";
await fs.mkdir(path.dirname(startosConfigPath), { recursive: true });
await fs.writeFile(tmp, JSON.stringify(merged, null, 2), { mode: 0o600 });
await fs.rename(tmp, startosConfigPath);
// Re-run the gemini-key refresher so serverApiKey reflects the
// patch immediately (otherwise it'd lag until the poll tick).
if (Object.prototype.hasOwnProperty.call(patch, "gemini_api_key")) {
await refreshServerApiKey("merge config");
}
if (Object.prototype.hasOwnProperty.call(patch, "recap_relay_operator_key")) {
await refreshRelayOperatorKey("merge config");
}
}
+671
View File
@@ -0,0 +1,671 @@
// Recap-side proxy to the relay's credit-purchase endpoints.
//
// Architecture is identical to license-purchase.js: Recap doesn't
// hold the BTCPay credentials, the relay does. Recap just forwards
// the buyer's pick to the relay and proxies the polling. The relay
// returns the BTCPay checkout URL which the Recap UI displays in a
// modal styled to match the license-purchase modal.
//
// Endpoints:
// GET /api/credits/packages → relay GET /relay/credits/packages
// POST /api/credits/buy → relay POST /relay/credits/buy
// GET /api/credits/invoice/:id → relay GET /relay/credits/invoice/:id
//
// Auth headers (X-Recap-Install-Id + Authorization Bearer LIC1-...)
// are added by this proxy, not by the buyer-side JS — keeping the
// install identity + license key out of any client-side code.
import { getRelayBaseURL } from "./relay-default.js";
import { getInstallId } from "./install-id.js";
import { getRawLicenseKey } from "./license.js";
// Multi-mode toggle. In multi mode every credit purchase is recorded
// in pending_purchases so we know WHO (signed-in user vs. anon trial
// cookie) to credit locally when the invoice settles. Single mode is
// the legacy "operator-pool only" flow — no local accounting layer,
// the relay's credits.json IS the source of truth.
const RECAP_MODE = process.env.RECAP_MODE === "multi" ? "multi" : "single";
function relayHeaders({ json = false, req = null } = {}) {
const h = {};
// Identity routing for the credit-purchase + credit-poll flow:
//
// Pro/Max signed-in tenant (req.user.keysat_license set)
// → use THEIR install ID + license. The buy invoice gets
// stashed with THEIR license_fingerprint so the BTCPay
// settle-webhook credits THEIR license-keyed pool — the
// same pool /api/relay/status reads when displaying their
// balance. Without this, credits land on the operator's
// pool and the buyer sees their balance unchanged after
// paying (the bug Grant hit on 2026-05-18).
//
// Anon visitor (trial cookie only) / free signed-in tenant
// (no license) / single-mode operator
// → fall back to operator identity. The operator's pool is
// what's being topped up; Recaps' own accounting layer
// (anon_trials / tenant_credits) handles the per-user
// attribution locally via pending_purchases.
let installId = null;
let licenseKey = null;
if (req?.user?.keysat_license && req.user.synthetic_install_id) {
installId = req.user.synthetic_install_id;
licenseKey = req.user.keysat_license;
}
if (!installId) {
try {
const id = getInstallId();
if (id) installId = id;
} catch {}
}
if (!licenseKey) {
try {
const key = getRawLicenseKey();
if (key) licenseKey = key;
} catch {}
}
if (installId) h["X-Recap-Install-Id"] = installId;
if (licenseKey) h["Authorization"] = `Bearer ${licenseKey}`;
if (json) h["Content-Type"] = "application/json";
return h;
}
export function setupCreditsPurchaseRoutes(app) {
// List bundles the operator has configured. Cheap, no auth gating —
// the buyer needs the price menu before they decide whether to pay.
app.get("/api/credits/packages", async (_req, res) => {
const base = getRelayBaseURL();
if (!base) {
return res.status(503).json({
error: "relay_not_configured",
message: "Relay base URL not set on this Recaps install.",
});
}
try {
// 10s timeout — was 5s, but a cold relay request from mobile
// cellular can take 6-8s, and Safari iOS surfaces the abort as
// a generic "Load failed" with no other info, so the buyer
// sees an error and has to manually retry. 10s is still snappy
// enough that a legit failure doesn't hang the UI for long.
const r = await fetch(`${base.replace(/\/$/, "")}/relay/credits/packages`, {
signal: AbortSignal.timeout(10000),
});
const text = await r.text();
let body = null;
try {
body = text ? JSON.parse(text) : null;
} catch {}
if (!r.ok) {
return res.status(r.status).json(body || { error: "relay_packages_failed" });
}
res.json(body || { packages: [] });
} catch (err) {
console.error(`[credits/packages] failed: ${err?.message || err}`);
res.status(502).json({
error: "packages_fetch_failed",
message: (err?.message || String(err)).slice(0, 300),
});
}
});
// Initiate a purchase. Body: { credits: 1|5|10|20 }. Returns the
// raw relay envelope (so the UI sees credits_remaining + tier +
// result.checkout_url + result.invoice_id).
//
// Multi-mode: identifies the buyer (signed-in user or anon trial
// cookie), records a pending_purchases row keyed by the invoice_id
// the relay returns. The settle-handler (in /api/credits/invoice/:id
// below) uses that row to know WHERE to apply the credits locally.
//
// Single-mode: skips the pending_purchases bookkeeping entirely;
// the operator IS the buyer and the relay's credits.json directly
// tracks their pool.
app.post("/api/credits/buy", async (req, res) => {
const base = getRelayBaseURL();
if (!base) {
return res
.status(503)
.json({ error: "relay_not_configured" });
}
const credits = Number(req.body?.credits);
const returnUrl =
typeof req.body?.return_url === "string" && req.body.return_url.startsWith("http")
? req.body.return_url
: null;
if (!Number.isFinite(credits) || credits <= 0) {
return res
.status(400)
.json({ error: "credits_required" });
}
// Identify the buyer for multi-mode. Either a signed-in user OR
// an anon trial cookie. If neither, attempt to auto-mint a trial
// cookie — anon visitors who click "Buy more" from the toolbar
// (before they've spent their pre-trial allowance) shouldn't be
// dead-ended into a sign-in nag. Same auto-mint pattern as
// /api/process for pre-trial visitors. Only refuse if trials are
// disabled or the IP is over its lifetime cap.
let buyerType = null;
let buyerId = null;
if (RECAP_MODE === "multi") {
if (req.user && req.user.id) {
buyerType = "user";
buyerId = req.user.id;
} else if (req.trial && req.trial.cookie_id) {
buyerType = "anon";
buyerId = req.trial.cookie_id;
} else {
// Try to mint a fresh trial cookie so the purchase has
// somewhere to land. forceMint=true bypasses the lifetime
// IP cap and the trials-disabled config — a paying buyer is
// by definition not abusing a free quota, and without a
// tracking cookie the settle handler has nowhere to credit
// the purchase locally (the relay still credits the operator
// pool; we just lose the visibility to apply it to this
// specific buyer).
try {
const { issueIfEligible } = await import("./anon-trial.js");
const trial = await issueIfEligible({ req, res, forceMint: true });
if (trial) {
buyerType = "anon";
buyerId = trial.cookie_id;
// Stash on req for downstream code paths
req.trial = trial;
}
} catch (err) {
console.warn(
"[credits/buy] anon-trial mint failed:",
err?.message || err,
);
}
if (!buyerId) {
return res.status(401).json({
error: "buyer_unknown",
message:
"Couldn't create a buyer record for this purchase. Sign up for a free account so we have somewhere to credit it.",
});
}
}
}
try {
const r = await fetch(`${base.replace(/\/$/, "")}/relay/credits/buy`, {
method: "POST",
headers: relayHeaders({ json: true, req }),
body: JSON.stringify({ credits, return_url: returnUrl || undefined }),
signal: AbortSignal.timeout(15_000),
});
const text = await r.text();
let body = null;
try {
body = text ? JSON.parse(text) : null;
} catch {}
if (!r.ok) {
return res
.status(r.status)
.json(body || { error: "relay_buy_failed" });
}
// Record the pending purchase BEFORE we respond, so even if the
// browser refreshes / crashes between buy + settle, the next
// poll for this invoice id will still know who to credit.
// Invoice id lives under result.invoice_id per the relay's
// envelope contract (same shape license-purchase uses).
const invoiceId =
body?.result?.invoice_id ||
body?.invoice_id ||
body?.btcpay_invoice_id ||
null;
if (RECAP_MODE === "multi") {
if (!invoiceId) {
// Loud warning — without an invoice id we can't reconcile
// on settle. Surface the response shape so we can see what
// the relay actually returned and fix the field-name
// assumption if this fires.
console.warn(
`[credits/buy] NO invoice_id in relay response — skipping pending_purchases. Top-level keys: ${Object.keys(body || {}).join(", ")} | result keys: ${Object.keys(body?.result || {}).join(", ")}`,
);
} else if (!buyerType || !buyerId) {
console.warn(
`[credits/buy] invoice ${invoiceId}: buyer identity missing — won't auto-apply on settle.`,
);
} else {
try {
const { getDb } = await import("./db.js");
const result = getDb()
.prepare(
`INSERT OR IGNORE INTO pending_purchases
(invoice_id, buyer_type, buyer_id, credits, created_at)
VALUES (?, ?, ?, ?, ?)`,
)
.run(invoiceId, buyerType, buyerId, credits, Date.now());
console.log(
`[credits/buy] tracked pending purchase invoice=${invoiceId} buyer=${buyerType}:${buyerId} credits=${credits} rowsInserted=${result.changes}`,
);
} catch (err) {
console.error(
`[credits/buy] failed to record pending purchase ${invoiceId}: ${err?.message || err}`,
);
}
}
}
res.json(body || {});
} catch (err) {
console.error(`[credits/buy] failed: ${err?.message || err}`);
res.status(502).json({
error: "purchase_failed",
message: (err?.message || String(err)).slice(0, 300),
});
}
});
// Poll an invoice's status. Returns the relay envelope; the UI
// reads `result.status` ("new" | "processing" | "settled" |
// "expired" | "invalid") and refreshes when settled.
//
// Multi-mode side effect: when the relay reports settled, we look
// up the matching pending_purchases row and apply the credits to
// the right local balance. Idempotent via applied_at — if the same
// invoice is polled multiple times after settle, only the first
// application takes effect.
app.get("/api/credits/invoice/:id", async (req, res) => {
const base = getRelayBaseURL();
if (!base) {
return res.status(503).json({ error: "relay_not_configured" });
}
const id = (req.params.id || "").trim();
if (!id) {
return res.status(400).json({ error: "missing_invoice_id" });
}
try {
const r = await fetch(
`${base.replace(/\/$/, "")}/relay/credits/invoice/${encodeURIComponent(id)}`,
{
headers: relayHeaders({ req }),
signal: AbortSignal.timeout(10_000),
}
);
const text = await r.text();
let body = null;
try {
body = text ? JSON.parse(text) : null;
} catch {}
if (!r.ok) {
return res
.status(r.status)
.json(body || { error: "relay_poll_failed" });
}
// Multi-mode: settle-and-apply. Status path mirrors the
// license-purchase poll-settle handler.
const status =
body?.result?.status || body?.status || null;
if (RECAP_MODE === "multi" && status === "settled") {
try {
await applyPendingPurchase(id);
} catch (err) {
console.error(
`[credits/invoice] apply failed for ${id}: ${err?.message || err}`,
);
// Don't fail the response — the relay reported settled and
// the operator pool has the credits. Local apply can be
// retried by hitting this endpoint again, or by a future
// reconciliation tool.
}
}
res.json(body || {});
} catch (err) {
console.error(`[credits/invoice] failed: ${err?.message || err}`);
res.status(502).json({
error: "poll_failed",
message: (err?.message || String(err)).slice(0, 300),
});
}
});
// POST /api/credits/claim { invoice_id }
// Manual self-service recovery: a signed-in user pastes the BTCPay
// invoice ID of a purchase they made anonymously (e.g., Safari
// Private mode where the trial cookie didn't survive the magic-
// link click). We verify the invoice is settled at the relay AND
// the pending_purchases row is anon-buyer + unapplied, then credit
// their account.
//
// Safety:
// - Requires authenticated user (req.user.id must be set)
// - Only claims buyer_type='anon' rows (no user-to-user takeover)
// - applied_at idempotency guard prevents double-credit
// - BTCPay invoice IDs are 30+ char random — not enumerable
// - User-buyer rows are never claimable here, regardless of
// ownership — those are the cookie sweep's job
app.post("/api/credits/claim", async (req, res) => {
if (RECAP_MODE !== "multi") {
return res.status(404).json({ error: "not_available" });
}
if (!req.user || !req.user.id) {
return res.status(401).json({
error: "auth_required",
message: "Sign in first to claim a purchase to your account.",
});
}
const invoiceId = String(req.body?.invoice_id || "").trim();
if (!invoiceId) {
return res.status(400).json({
error: "missing_invoice_id",
message: "Paste the invoice ID from your purchase email.",
});
}
const { getDb } = await import("./db.js");
const db = getDb();
const row = db
.prepare(
`SELECT invoice_id, buyer_type, buyer_id, credits, applied_at
FROM pending_purchases WHERE invoice_id = ?`,
)
.get(invoiceId);
if (!row) {
return res.status(404).json({
error: "invoice_not_found",
message:
"We don't have a record of that invoice ID. Double-check it — the ID is shown in your BTCPay payment confirmation.",
});
}
if (row.buyer_type !== "anon") {
// user-buyer rows are claimable only by their original buyer
// (cookie sweep) — refusing this avoids user-to-user takeover.
return res.status(403).json({
error: "not_anon_purchase",
message:
"This invoice was bought from a signed-in account and can only be claimed by that account.",
});
}
if (row.applied_at) {
return res.status(409).json({
error: "already_applied",
message:
"Those credits were already applied. Check your balance — they may have transferred automatically.",
});
}
// Verify settled at the relay before crediting. We do NOT trust
// the local row alone — the buyer could have initiated the
// invoice and never paid; without this check, anyone could
// claim N credits just by knowing an invoice ID.
const base = getRelayBaseURL();
if (!base) {
return res.status(503).json({ error: "relay_not_configured" });
}
let status = null;
try {
const r = await fetch(
`${base.replace(/\/$/, "")}/relay/credits/invoice/${encodeURIComponent(invoiceId)}`,
{ headers: relayHeaders({ req }), signal: AbortSignal.timeout(10_000) },
);
if (r.ok) {
const body = await r.json().catch(() => ({}));
status = body?.result?.status || body?.status || null;
}
} catch (err) {
console.warn(
`[credits/claim] relay status check failed for ${invoiceId}: ${err?.message || err}`,
);
return res.status(502).json({
error: "relay_unreachable",
message:
"Couldn't verify the invoice with the payment server. Try again in a minute.",
});
}
if (status !== "settled") {
return res.status(409).json({
error: "not_settled",
message: `That invoice is not settled (status: ${status || "unknown"}). If you just paid, wait a minute and try again.`,
});
}
try {
await applyPendingPurchase(invoiceId, { forceUserId: req.user.id });
} catch (err) {
console.error(
`[credits/claim] apply failed for ${invoiceId}: ${err?.message || err}`,
);
return res.status(500).json({
error: "apply_failed",
message: "Something went wrong applying the credits. Try again.",
});
}
console.log(
`[credits/claim] user ${req.user.id} claimed invoice ${invoiceId} (${row.credits} credits)`,
);
res.json({ ok: true, credits: row.credits });
});
}
// applyPendingPurchase(invoiceId, opts?) — credit the buyer's local
// balance for a settled invoice. Idempotent: bails if the row is
// already marked applied. If the buyer was an anon trial that has
// since been converted to a real user, credits route to the user
// instead.
//
// opts.forceUserId (optional) — route credits to this user instead
// of the row's recorded buyer. Used by the manual-claim endpoint:
// when a signed-in user pastes a BTCPay invoice ID for an anon
// purchase whose trial cookie was lost (e.g., Safari Private mode
// where the magic-link click landed in a different cookie jar), we
// trust the invoice ID as proof-of-ownership and direct the credits
// to their tenant_credits.
//
// Exported so the sweep helper below — and any future server-side
// flow that wants to reconcile a known-settled invoice — can call it
// without going through the /api/credits/invoice/:id route.
export async function applyPendingPurchase(invoiceId, opts = {}) {
const { getDb } = await import("./db.js");
const db = getDb();
const row = db
.prepare(
`SELECT invoice_id, buyer_type, buyer_id, credits, applied_at
FROM pending_purchases WHERE invoice_id = ?`,
)
.get(invoiceId);
if (!row) {
// Either the buy came from a different Recap instance, or the
// bookkeeping insert in /api/credits/buy failed earlier. Nothing
// to do; the operator pool still has the credits from BTCPay.
// Log so operator can reconcile manually if this fires.
console.warn(
`[credits/invoice] settled invoice ${invoiceId} has NO matching pending_purchases row — local balance NOT auto-applied. The credits ARE in the operator pool at the relay; operator should grant manually to the buyer.`,
);
return;
}
if (row.applied_at) {
return; // already applied, idempotent no-op
}
// Resolve buyer → target user_id (for tenant_credits) or trial
// cookie_id (for anon_trials.credits_total). Anon-buyers who have
// since converted to a real user get their credits routed to the
// user's tenant_credits — that's the cleaner outcome and matches
// the "credits transfer on signup" semantics the design promises.
let targetUserId = null;
let targetCookieId = null;
if (opts.forceUserId) {
targetUserId = opts.forceUserId;
} else if (row.buyer_type === "user") {
targetUserId = row.buyer_id;
} else if (row.buyer_type === "anon") {
const trial = db
.prepare(
"SELECT cookie_id, converted_to_user_id FROM anon_trials WHERE cookie_id = ?",
)
.get(row.buyer_id);
if (trial?.converted_to_user_id) {
targetUserId = trial.converted_to_user_id;
} else {
targetCookieId = row.buyer_id;
}
}
// Apply + mark applied in one transaction so a crash mid-way
// doesn't leave a half-credited buyer. Purchased credits land in
// the PERMANENT bucket (purchased_balance) so they're not wiped on
// the next replenishment refresh.
const tx = db.transaction(() => {
if (targetUserId) {
const existing = db
.prepare("SELECT user_id FROM tenant_credits WHERE user_id = ?")
.get(targetUserId);
if (existing) {
db.prepare(
`UPDATE tenant_credits
SET purchased_balance = purchased_balance + ?,
lifetime_granted = lifetime_granted + ?
WHERE user_id = ?`,
).run(row.credits, row.credits, targetUserId);
} else {
db.prepare(
`INSERT INTO tenant_credits
(user_id, purchased_balance, replenish_balance, last_replenish_at,
lifetime_granted, lifetime_consumed)
VALUES (?, ?, 0, ?, ?, 0)`,
).run(targetUserId, row.credits, Date.now(), row.credits);
}
} else if (targetCookieId) {
// Anon trial: credits go into the trial's credits_total (single
// bucket — anons don't have the purchased/replenish split).
// They'll move to purchased_balance on signup via linkToUser.
db.prepare(
`UPDATE anon_trials
SET credits_total = credits_total + ?
WHERE cookie_id = ?`,
).run(row.credits, targetCookieId);
}
db.prepare(
"UPDATE pending_purchases SET applied_at = ? WHERE invoice_id = ?",
).run(Date.now(), invoiceId);
});
tx();
console.log(
`[credits/invoice] applied ${row.credits} credits for ${row.buyer_type}:${row.buyer_id}${
targetUserId ? "user " + targetUserId : "anon " + targetCookieId
}`,
);
}
// sweepUnappliedPurchases({ buyerType, buyerId, cookieIds }) — catch
// up on settled-but-unapplied purchases for a buyer.
//
// Why this exists: the buy → BTCPay → settle → apply pipeline depends
// on the buyer's browser tab polling /api/credits/invoice/:id after
// BTCPay redirects back. But BTCPay redirects in the SAME tab (the
// poll loop dies before it gets a chance to see "settled"), and even
// when the redirect lands back on Recap the buyer might close it
// before the next poll tick. Result: the relay knows the invoice is
// settled and the operator pool has the credits, but the LOCAL
// pending_purchases row never flips to applied — so the buyer's
// balance stays stale until they manually re-poll, which they have no
// way to do.
//
// Fix: opportunistically sweep on every /api/account/whoami and
// /api/relay/status. Cheap (small bounded query + a few relay HTTP
// calls), idempotent (applyPendingPurchase no-ops on already-applied
// rows), and self-healing.
//
// Also called from anon-trial.js linkToUser BEFORE the transfer, so
// any anon-bought credits that hadn't yet been applied locally are
// rolled into anon_trials.credits_total before we copy them over to
// the new user's tenant_credits.
//
// Scope: only sweeps the buyer's OWN pending rows. cookieIds is an
// optional list of additional anon cookie_ids the caller wants
// swept on this buyer's behalf (used by /whoami for the new-signup
// case where the just-converted cookie may still have unapplied
// purchases). Cap at 5 invoices per sweep + 30-minute lookback so a
// degenerate case can't fan out into hundreds of relay calls per
// request.
export async function sweepUnappliedPurchases({
buyerType,
buyerId,
cookieIds = [],
req = null,
} = {}) {
if (RECAP_MODE !== "multi") return;
if (!buyerType && (!cookieIds || cookieIds.length === 0)) return;
const base = getRelayBaseURL();
if (!base) return; // no relay configured, nothing to sweep against
const { getDb } = await import("./db.js");
const db = getDb();
// 30-minute lookback. Older unapplied purchases probably failed for
// a reason we don't want to keep retrying every page-load (relay
// unreachable, invoice expired, etc.). Operator can reconcile
// manually if they fire.
const since = Date.now() - 30 * 60 * 1000;
// Build the WHERE clause. Always include the primary buyer; OR in
// any extra cookieIds the caller passed.
const conditions = [];
const params = [];
if (buyerType && buyerId) {
conditions.push("(buyer_type = ? AND buyer_id = ?)");
params.push(buyerType, buyerId);
}
for (const cid of cookieIds) {
if (typeof cid === "string" && cid) {
conditions.push("(buyer_type = 'anon' AND buyer_id = ?)");
params.push(cid);
}
}
if (conditions.length === 0) return;
params.push(since);
let rows = [];
try {
rows = db
.prepare(
`SELECT invoice_id FROM pending_purchases
WHERE (${conditions.join(" OR ")})
AND applied_at IS NULL
AND created_at >= ?
ORDER BY created_at DESC
LIMIT 5`,
)
.all(...params);
} catch (err) {
console.warn(
`[credits/sweep] query failed: ${err?.message || err}`,
);
return;
}
if (rows.length === 0) return;
for (const { invoice_id: invoiceId } of rows) {
try {
const r = await fetch(
`${base.replace(/\/$/, "")}/relay/credits/invoice/${encodeURIComponent(invoiceId)}`,
{
headers: relayHeaders({ req }),
signal: AbortSignal.timeout(5_000),
},
);
if (!r.ok) continue;
const text = await r.text();
let body = null;
try {
body = text ? JSON.parse(text) : null;
} catch {}
const status = body?.result?.status || body?.status || null;
if (status === "settled") {
await applyPendingPurchase(invoiceId);
}
} catch (err) {
// Best-effort; swallow per-invoice errors so one bad invoice
// doesn't block the others (or the page-load).
console.warn(
`[credits/sweep] invoice ${invoiceId} check failed: ${err?.message || err}`,
);
}
}
}
+426
View File
@@ -0,0 +1,426 @@
// Daily Digest — per-episode overview synthesis (multi-mode / cloud).
//
// Phase 2 of the Daily Digest feature: turn a saved recap's stored topic
// summaries into a 12 paragraph overview via the relay LLM, then cache
// the result back onto the session JSON (`digestOverview`) so it's
// generated at most once per episode. The daily scan + email (phase 3)
// will call getOrCreateEpisodeOverview(); no scheduler lives here yet.
//
// Cost ownership (Q1 = operator-absorbed): the synthesis call uses the
// OPERATOR's relay identity — the same credit pool that free signed-in
// users' summaries already draw from (resolveProviderOpts with req=null
// → operator install identity). A retention email shouldn't silently
// drain the recipient's quota for recaps they already made. To bill the
// recipient instead, build the provider with their cloud identity at the
// one marked line below.
import { randomBytes } from "crypto";
import { getProvider, resolveProviderOpts } from "./providers/index.js";
import { patchSession, loadSession, listScopeSessions } from "./history.js";
import { getDb } from "./db.js";
import { sendMail, isSmtpReady } from "./smtp.js";
import { renderDigestEmail } from "./email-template.js";
import { getConfigSnapshot } from "./config.js";
// Operator-internal vocabulary the sibling relay could surface in model
// output (backend / hardware names, LAN hosts). Scrubbed before any
// digest text reaches a cloud user — the same error-boundary rule the
// rest of the app follows. This is a backstop: the synthesis input is
// the recap's own (already user-facing) topic summaries, so a leak here
// is unlikely. Kept conservative to avoid mangling legitimate prose —
// only unambiguous infra tokens and private/LAN hosts, never common
// words or public data.
const OPERATOR_TERMS = [
/\bspark[\s-]?control\b/gi,
/\bparakeet\b/gi,
/\bsortformer\b/gi,
/\btitanet\b/gi,
/\bvllm\b/gi,
];
const LAN_HOST_RE = /\bhttps?:\/\/[^\s)]*\.local\b[^\s)]*/gi;
const PRIVATE_IP_RE =
/\b(?:10|127)\.\d{1,3}\.\d{1,3}\.\d{1,3}\b|\b192\.168\.\d{1,3}\.\d{1,3}\b|\b172\.(?:1[6-9]|2\d|3[01])\.\d{1,3}\.\d{1,3}\b/g;
export function scrubOperatorStrings(text) {
if (!text) return "";
let out = String(text);
for (const re of OPERATOR_TERMS) out = out.replace(re, "");
out = out.replace(LAN_HOST_RE, "");
out = out.replace(PRIVATE_IP_RE, "");
// Tidy whitespace / orphaned punctuation the removals may have left.
return out
.replace(/[ \t]{2,}/g, " ")
.replace(/\s+([.,;:])/g, "$1")
.trim();
}
// Build the LLM prompt from a saved recap record. Pure — exported for
// testing. Uses each topic's title + summary (the chunk shape is
// { title, summary, … }); a topic with no summary still contributes its
// title so the overview knows it was covered.
export function buildOverviewPrompt(record) {
const title = (record?.title || "Untitled").trim();
const type =
record?.type === "podcast"
? "podcast episode"
: record?.type === "youtube"
? "video"
: "recording";
const topics = Array.isArray(record?.chunks) ? record.chunks : [];
const topicBlock = topics
.map((c, i) => {
const t = (c?.title || `Topic ${i + 1}`).trim();
const s = (c?.summary || "").trim();
return s ? `- ${t}: ${s}` : `- ${t}`;
})
.join("\n");
return [
`Below are the per-topic summaries of a ${type} titled "${title}".`,
"",
"Write a tight 12 paragraph overview (about 100150 words) that captures " +
"the main throughline and the few most important takeaways, as if briefing " +
"a busy reader who hasn't seen it. Do not invent anything beyond the " +
"summaries below, use no headings or bullet points, and write in plain prose.",
"",
"Topic summaries:",
topicBlock,
].join("\n");
}
// Operator-identity relay provider for synthesis (operator-absorbed).
// Throws if the relay isn't configured (no install id / base URL) — the
// caller treats that as "skip this episode", not a fatal error.
function buildSynthesisProvider() {
// req=null → operator install identity. Swap in a per-recipient cloud
// identity here to bill the user instead of the operator.
const opts = resolveProviderOpts("relay", { req: null });
return getProvider("relay", opts);
}
// Synthesize (no cache) — call the relay, scrub, return the overview
// text. Throws on no-topics or an empty model result. `provider` is
// injectable for testing; defaults to the operator-identity relay.
export async function synthesizeEpisodeOverview(record, { provider } = {}) {
const topics = Array.isArray(record?.chunks) ? record.chunks : [];
if (topics.length === 0) {
throw new Error("no topic summaries to synthesize");
}
const p = provider || buildSynthesisProvider();
const prompt = buildOverviewPrompt(record);
const result = await p.analyzeText({
prompt,
retries: 1,
// Stable per-episode billing key: a retry within the relay's job
// window reuses the same credit rather than charging twice.
jobId: record?.id ? `digest-${record.id}` : undefined,
});
const text = scrubOperatorStrings(result?.text || "");
if (!text) throw new Error("empty synthesis result");
return text;
}
// Get-or-generate the cached overview. Returns { overview, cached }. On a
// cache miss it synthesizes and (unless save:false) writes the result
// back onto the session JSON so the next caller is a cache hit. The
// write-back is best-effort — a failed patch just means we re-synthesize
// next time, never a user-visible error.
export async function getOrCreateEpisodeOverview({
scope,
id,
record,
provider,
save = true,
}) {
const cached = (record?.digestOverview || "").trim();
if (cached) return { overview: cached, cached: true };
const overview = await synthesizeEpisodeOverview(record, { provider });
if (save && scope && id) {
try {
await patchSession(scope, id, { digestOverview: overview });
} catch {
// best-effort cache; ignore
}
}
return { overview, cached: false };
}
// ── Daily scan + scheduler (mirrors subscription-reminders.js) ──────────
const SEND_HOUR = 8; // 08:00 server-local — when the daily scan acts
const SCAN_INTERVAL_MS = 60 * 60 * 1000; // tick hourly; act only at SEND_HOUR
const BOOT_DELAY_MS = 2 * 60 * 1000;
// A user gets at most one digest per ~day even if the loop ticks more
// than once inside the send hour or they add content right after a send.
const MIN_RESEND_MS = 20 * 60 * 60 * 1000;
const MAX_EPISODES = 10; // cap per email; the rest become an overflow count
let scanning = false;
let scheduled = false;
// Which library scope a user's recaps live under. Mirrors
// history.js scopeForRequest: the multi-mode admin keeps the "owner"
// scope; everyone else is scoped by their user id. Pure — exported for
// testing.
export function scopeForUser(user) {
return user?.is_admin ? "owner" : user?.id;
}
// Pick the recaps created after the watermark, oldest first, capped.
// Pure — exported for testing. Returns { episodes, overflow, total }.
export function selectDigestEpisodes(sessions, watermarkMs, cap = MAX_EPISODES) {
const since = typeof watermarkMs === "number" ? watermarkMs : 0;
const fresh = (sessions || [])
.filter((s) => {
const t = new Date(s?.createdAt).getTime();
return Number.isFinite(t) && t > since;
})
.sort((a, b) => new Date(a.createdAt) - new Date(b.createdAt));
return {
episodes: fresh.slice(0, cap),
overflow: Math.max(0, fresh.length - cap),
total: fresh.length,
};
}
function maskEmail(email) {
return String(email).replace(/^(.).*(@.*)$/, "$1***$2");
}
// Mint (and persist) a user's unsubscribe token if they don't have one
// yet. Returns the token. Stable per user — re-enabling reuses it.
function ensureUnsubToken(db, user) {
if (user.digest_unsub_token) return user.digest_unsub_token;
const token = randomBytes(32).toString("base64url");
db.prepare("UPDATE users SET digest_unsub_token = ? WHERE id = ?").run(
token,
user.id,
);
return token;
}
// Build one user's digest: synthesize an overview per selected episode
// (operator-absorbed, cached). Returns { built, failed } where built are
// the episode payloads ready for the template (each carrying its source
// createdAt) and failed is the createdAt list of episodes that errored —
// the caller uses both to set a watermark that never skips a failure.
async function buildUserEpisodes(scope, selected) {
const built = [];
const failed = [];
for (const ep of selected) {
try {
const record = await loadSession(scope, ep.id);
if (!record) {
failed.push(ep.createdAt);
continue;
}
const { overview } = await getOrCreateEpisodeOverview({
scope,
id: ep.id,
record,
});
built.push({
title: ep.title,
type: ep.type,
url: ep.url,
overview,
createdAt: ep.createdAt,
});
} catch (err) {
failed.push(ep.createdAt);
console.warn(
`[digest] synthesis failed for ${scope}/${ep.id}: ${err?.message || err}`,
);
}
}
return { built, failed };
}
// The watermark to stamp after a send. Advances to the newest
// successfully-sent recap, but never past the oldest one that FAILED (so
// the next scan retries gaps) and never past un-synthesized overflow
// recaps (their createdAt is newer than any sent one, so they're picked
// up next scan too). Pure — exported for testing. Returns null when
// nothing was sent (caller should not advance). createdAt inputs are ISO
// strings; output is ms epoch.
export function nextDigestWatermark(sentCreatedAts, failedCreatedAts) {
const toMs = (x) => new Date(x).getTime();
const sent = (sentCreatedAts || []).map(toMs).filter(Number.isFinite);
if (sent.length === 0) return null;
const failed = (failedCreatedAts || []).map(toMs).filter(Number.isFinite);
const newestSent = Math.max(...sent);
const oldestFailed = failed.length ? Math.min(...failed) : Infinity;
return Math.min(newestSent, oldestFailed - 1);
}
// One scan pass. Self-gating, deduped, NEVER throws — returns a small
// summary so the scheduler stays alive. `force` bypasses the send-hour
// gate (used by the operator test trigger), not the per-user resend gate.
export async function runDigestScan({ force = false } = {}) {
// `force` bypasses the send-hour gate (operator test trigger), NOT the
// in-progress lock — a forced run alongside the scheduled tick would
// otherwise double-send to every opted-in user.
if (scanning) return { skipped: "already_running" };
scanning = true;
try {
const now = Date.now();
if (!force && new Date(now).getHours() !== SEND_HOUR) {
return { skipped: "off_hour" };
}
if (!isSmtpReady()) return { skipped: "smtp_not_ready" };
const snap = await getConfigSnapshot();
const publicUrl = (snap.recap_public_url || "").trim().replace(/\/$/, "");
if (!publicUrl) return { skipped: "public_url_not_set" };
const db = getDb();
const users = db
.prepare(
"SELECT id, email, is_admin, last_digest_at, digest_unsub_token FROM users WHERE digest_enabled = 1",
)
.all();
let sent = 0;
let skipped = 0;
for (const user of users) {
try {
const email = (user.email || "").trim();
if (!email) {
skipped++;
continue;
}
// Defensive: a row with no watermark (set via SQL, not the opt-in
// endpoint) would dump the whole backlog — start the clock now
// and pick up new recaps next scan instead.
if (typeof user.last_digest_at !== "number") {
db.prepare("UPDATE users SET last_digest_at = ? WHERE id = ?").run(
now,
user.id,
);
skipped++;
continue;
}
if (now - user.last_digest_at < MIN_RESEND_MS) {
skipped++;
continue;
}
const scope = scopeForUser(user);
if (!scope) {
// No usable id (shouldn't happen for a real row) — skip rather
// than read an "undefined" scope dir.
skipped++;
continue;
}
const sessions = await listScopeSessions(scope);
const { episodes: selected, overflow } = selectDigestEpisodes(
sessions,
user.last_digest_at,
MAX_EPISODES,
);
if (selected.length === 0) {
skipped++; // nothing new — skip the email, leave the watermark
continue;
}
const { built, failed } = await buildUserEpisodes(scope, selected);
if (built.length === 0) {
// Synthesis failed for all of them — don't advance the
// watermark, so the next scan retries the same recaps.
skipped++;
continue;
}
const token = ensureUnsubToken(db, user);
const message = renderDigestEmail({
brandName: "Recaps",
episodes: built,
overflowCount: overflow,
manageUrl: `${publicUrl}/`,
unsubscribeUrl: `${publicUrl}/api/digest/unsubscribe?token=${encodeURIComponent(token)}`,
});
await sendMail({
to: email,
subject: message.subject,
text: message.text,
html: message.html,
});
// Advance the watermark only after a successful send — to the
// newest sent recap, but never past a failed or deferred one, so
// the next scan retries gaps instead of skipping them.
const watermark = nextDigestWatermark(
built.map((e) => e.createdAt),
failed,
);
db.prepare("UPDATE users SET last_digest_at = ? WHERE id = ?").run(
watermark ?? now,
user.id,
);
sent++;
console.log(
`[digest] sent to ${maskEmail(email)} (${episodes.length} recap${episodes.length === 1 ? "" : "s"}${overflow ? `, +${overflow} more` : ""})`,
);
} catch (err) {
console.warn(
`[digest] user ${user.id} failed: ${err?.message || err}`,
);
skipped++;
}
}
if (sent) {
console.log(`[digest] scan complete: ${sent} sent, ${skipped} skipped`);
}
return { sent, skipped };
} catch (err) {
console.warn(`[digest] scan error: ${err?.message || err}`);
return { skipped: "error", error: err?.message || String(err) };
} finally {
scanning = false;
}
}
// Start the hourly scan loop. Idempotent; self-gates inside the scan, so
// it's safe to call whenever multi mode boots.
export function startDigestScheduler() {
if (scheduled) return;
scheduled = true;
setTimeout(() => {
runDigestScan().catch(() => {});
}, BOOT_DELAY_MS);
setInterval(() => {
runDigestScan().catch(() => {});
}, SCAN_INTERVAL_MS);
console.log("[digest] daily-digest scheduler started");
}
// One-click unsubscribe — a public GET (no session) keyed by the per-user
// token in the email. Mounted in index.js (multi mode) and whitelisted in
// tenant-auth's public paths. Flips digest_enabled off; the in-app toggle
// can turn it back on.
export function setupDigestRoutes(app) {
app.get("/api/digest/unsubscribe", (req, res) => {
const token = String(req.query?.token || "").trim();
const page = (msg) =>
`<!doctype html><html><body style="margin:0;padding:48px 16px;background:#fafafa;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;text-align:center;color:#333;"><div style="max-width:420px;margin:0 auto;background:#fff;border-radius:8px;padding:32px;font-size:15px;line-height:1.6;">${msg}</div></body></html>`;
if (!token) {
return res.status(400).send(page("Invalid unsubscribe link."));
}
try {
const r = getDb()
.prepare(
"UPDATE users SET digest_enabled = 0 WHERE digest_unsub_token = ?",
)
.run(token);
if (r.changes === 0) {
return res
.status(404)
.send(page("This unsubscribe link is no longer valid."));
}
return res.send(
page(
"You've been unsubscribed from the daily digest. You can turn it back on anytime in Settings.",
),
);
} catch (err) {
console.error("[digest] unsubscribe failed:", err);
return res
.status(500)
.send(page("Something went wrong. Please try again later."));
}
});
}
+444
View File
@@ -0,0 +1,444 @@
// Multi-tenant SQLite store — single source of truth for users,
// sessions, magic-link tokens, subscriptions, tenant credits, and
// the library_meta index over /data/history/<userId>/*.json files.
//
// Created only when RECAP_MODE === 'multi'. In single mode this module
// is never imported — `getDb()` would crash trying to require
// better-sqlite3 anyway, but the auth-middleware short-circuits before
// reaching it. Keep all SQLite access funneled through `getDb()` so
// single-mode boots don't touch the native binding at all.
//
// Forward-only schema. No migration framework — every release is one
// `db.exec(SCHEMA_SQL)` at boot. New columns get `ALTER TABLE …`
// statements appended below the original CREATEs and guarded with an
// existence check; new tables just go in fresh. Rollback is
// "checkpoint your /data dir before upgrading."
import path from "path";
let dbInstance = null;
const SCHEMA_SQL = `
-- ── users ──────────────────────────────────────────────────────────────
-- One row per authenticated end-user. The operator-owner is also a row
-- here (is_admin = 1) so per-user library scoping works uniformly.
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
email TEXT NOT NULL UNIQUE,
password_hash TEXT,
created_at INTEGER NOT NULL,
last_signin_at INTEGER,
synthetic_install_id TEXT NOT NULL UNIQUE,
keysat_license TEXT,
display_name TEXT,
is_admin INTEGER NOT NULL DEFAULT 0,
-- Core-decoupling: the user's subscription tier ("core" | "pro" | "max").
-- The Recap Relay is the source of truth (keyed by user-id); this is the
-- Recaps-side cache used for feature gating, kept in sync by the operator
-- grant flow (which writes here AND POSTs the relay's /relay/user-tier).
tier TEXT NOT NULL DEFAULT 'core',
-- Captured at first signup for forensic / abuse-investigation use.
-- NOT used for auth decisions — just data for the operator to grep
-- when an abuse pattern shows up in the admin dashboard.
signup_ip TEXT,
signup_user_agent TEXT,
-- Daily Digest (opt-in, multi-mode): a daily email of the user's last
-- ~24h of library recaps. Off by default. last_digest_at is the
-- ms-epoch watermark of the last send; the scan covers recaps created
-- AFTER it (dedup), and opt-in stamps it to "now" so the first digest
-- doesn't dump the whole backlog. NULL = never sent.
-- digest_unsub_token is a per-user random string for the one-click
-- unsubscribe link in each digest email (no login needed); minted
-- lazily on first send.
digest_enabled INTEGER NOT NULL DEFAULT 0,
last_digest_at INTEGER,
digest_unsub_token TEXT
);
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
CREATE INDEX IF NOT EXISTS idx_users_signup_ip ON users(signup_ip);
-- NB: idx_users_unsub_token is created in migrateUserDigestPrefs, not here
-- — SCHEMA_SQL runs before the column migration on existing DBs, so an
-- index over digest_unsub_token here would fail with "no such column".
-- ── sessions ───────────────────────────────────────────────────────────
-- Server-side session store so we can revoke individual sessions from
-- the dashboard. Cookies carry only the random session id.
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL,
last_used_at INTEGER,
user_agent TEXT,
ip_address TEXT
);
CREATE INDEX IF NOT EXISTS idx_sessions_user ON sessions(user_id);
CREATE INDEX IF NOT EXISTS idx_sessions_expires ON sessions(expires_at);
-- ── magic_link_tokens ──────────────────────────────────────────────────
-- Plaintext token only ever exists in the outbound email and the
-- inbound verify URL — what we persist is the SHA-256 hash. Tokens are
-- single-use (used_at NOT NULL = spent) and short-lived (15 min).
CREATE TABLE IF NOT EXISTS magic_link_tokens (
token_hash TEXT PRIMARY KEY,
email TEXT NOT NULL,
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL,
used_at INTEGER,
intent TEXT NOT NULL,
-- Request context for abuse investigation. Captured at /auth/request-link
-- time, never used for auth decisions — just for the recent-signups admin
-- view to surface scripted abuse patterns.
request_ip TEXT,
request_ua TEXT,
-- Anon trial cookie that was present at /auth/request-link time.
-- Stored server-side (NOT in the magic-link URL itself — that would
-- leak it to anyone who saw the email) so that at /auth/verify we
-- can link the trial → user even when the magic-link click lands
-- in a different browser / cookie jar than the one that initiated
-- the request (Safari Private mode + email-app in-app browser is
-- the canonical case). Server-side binding means the cookie ID
-- can't be spoofed: an attacker who intercepts the magic link
-- still can't change which trial gets linked.
trial_cookie_id TEXT
);
CREATE INDEX IF NOT EXISTS idx_magic_email ON magic_link_tokens(email);
CREATE INDEX IF NOT EXISTS idx_magic_ip ON magic_link_tokens(request_ip, created_at);
-- ── subscriptions ──────────────────────────────────────────────────────
-- One row per paid period. Multiple rows accumulate as a user renews.
-- We don't try to model "the active subscription" — joins to MAX(started_at)
-- with status='active' do the job and stay honest about history.
CREATE TABLE IF NOT EXISTS subscriptions (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
tier TEXT NOT NULL,
started_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL,
cancelled_at INTEGER,
btcpay_invoice_id TEXT,
amount_sats INTEGER,
status TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_subs_user ON subscriptions(user_id);
-- ── tenant_credits ─────────────────────────────────────────────────────
-- Per-tenant local credit ledger. Cloud users with their OWN keysat
-- license bill the relay directly (via the license-keyed pool); this
-- table is the source of truth for everyone else — signed-in users on
-- the free / cloud-default tier, and family-share tenants on a self-
-- hosted multi-tenant Recap.
--
-- Two buckets per user:
-- purchased_balance — a la carte purchases + admin grants + carry-over
-- from anon trial conversions. PERMANENT — never
-- wiped or refilled.
-- replenish_balance — initial signup allowance + periodic refills.
-- REFILLED to tenant_default_credits on each
-- anniversary period boundary (period set via
-- the tenant_credit_replenish_period config).
-- Leftover replenish credits at the end of a
-- period are FORFEIT (use-it-or-lose-it).
--
-- Spend order: debit replenish_balance first (it'll refresh anyway),
-- then purchased_balance only when the refillable bucket is empty.
-- last_replenish_at: epoch-ms of the most recent refill, used to compute
-- the next anniversary boundary.
CREATE TABLE IF NOT EXISTS tenant_credits (
user_id TEXT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
purchased_balance INTEGER NOT NULL DEFAULT 0,
replenish_balance INTEGER NOT NULL DEFAULT 0,
last_replenish_at INTEGER,
lifetime_granted INTEGER NOT NULL DEFAULT 0,
lifetime_consumed INTEGER NOT NULL DEFAULT 0
);
-- ── anon_trials ────────────────────────────────────────────────────────
-- Cookie-gated "taste before sign-up" trial. The first time an
-- unauthenticated visitor hits /api/process, we issue a recap_anon_trial
-- cookie (32-byte random), insert a row here with N credits (set by
-- the trial_credits_per_visitor operator config), and let them
-- summarize without signing up. After credits_used >= credits_total,
-- the UI nudges them to sign up for more.
--
-- Trial requests forward the OPERATOR's install_id + license to the
-- relay, so the operator's credit pool is what actually pays for the
-- Gemini call. tenant_credits.balance is irrelevant for trials —
-- the credits_total field on this row is the only gate.
--
-- ip_address rate-limits trial-cookie issuance: trials_per_ip_per_day
-- caps how many fresh trial cookies one IP can mint in 24h. Doesn't
-- stop sophisticated abuse (IP rotation), but raises the floor for
-- scripted laptop attacks and gives the operator a column to grep on.
--
-- converted_to_user_id is set when the trial holder signs up — links
-- the trial summary into their library and lets the operator measure
-- the trial → signup conversion rate.
CREATE TABLE IF NOT EXISTS anon_trials (
cookie_id TEXT PRIMARY KEY,
ip_address TEXT,
user_agent TEXT,
created_at INTEGER NOT NULL,
credits_total INTEGER NOT NULL,
credits_used INTEGER NOT NULL DEFAULT 0,
last_used_at INTEGER,
converted_to_user_id TEXT REFERENCES users(id) ON DELETE SET NULL
);
CREATE INDEX IF NOT EXISTS idx_anon_trials_ip ON anon_trials(ip_address, created_at);
CREATE INDEX IF NOT EXISTS idx_anon_trials_created ON anon_trials(created_at);
-- ── pending_purchases ──────────────────────────────────────────────────
-- Tracks every credit-purchase invoice initiated through Recap so that
-- when the invoice settles (via BTCPay webhook → relay → poll round-
-- trip back to us) we know WHO to credit locally.
--
-- The BTCPay invoice on the relay side credits the OPERATOR's pool —
-- the operator paid for the underlying Gemini/etc capacity at the
-- relay. Recap's local accounting layer (tenant_credits for signed-in
-- users, anon_trials.credits_total for trial cookies) is what gates
-- the actual buyer's spend, so we mark this row applied once the
-- relevant local balance is incremented. applied_at being non-null is
-- the idempotency guard — a poll firing twice doesn't double-credit.
--
-- buyer_type values:
-- "user" → buyer_id is users.id; credits land in tenant_credits
-- "anon" → buyer_id is anon_trials.cookie_id; credits land in
-- anon_trials.credits_total. If the cookie has since been
-- converted to a user (anon_trials.converted_to_user_id),
-- credits route to that user's tenant_credits instead.
CREATE TABLE IF NOT EXISTS pending_purchases (
invoice_id TEXT PRIMARY KEY,
buyer_type TEXT NOT NULL,
buyer_id TEXT NOT NULL,
credits INTEGER NOT NULL,
created_at INTEGER NOT NULL,
applied_at INTEGER
);
CREATE INDEX IF NOT EXISTS idx_pending_purchases_buyer ON pending_purchases(buyer_type, buyer_id);
CREATE INDEX IF NOT EXISTS idx_pending_purchases_unapplied ON pending_purchases(applied_at) WHERE applied_at IS NULL;
-- ── pending_signups ────────────────────────────────────────────────────
-- Buyer-creates-account flow: when an anon visitor picks Pro / Max
-- from the tier signup modal, they enter an email and pay BTCPay
-- BEFORE any user account exists. We record the (invoice_id, email,
-- policy_slug) here so the poll-settle handler can create the user +
-- attach the issued license + send a magic-link email once payment
-- lands. applied_at is the idempotency guard — multiple polls after
-- settle don't double-create the user.
--
-- Distinct from pending_purchases (credit-pack buys) because the
-- settle effects are completely different: pending_signups creates
-- a USER and sends an email; pending_purchases just credits an
-- existing buyer's local balance.
CREATE TABLE IF NOT EXISTS pending_signups (
invoice_id TEXT PRIMARY KEY,
email TEXT NOT NULL,
policy_slug TEXT NOT NULL,
created_at INTEGER NOT NULL,
applied_at INTEGER
);
CREATE INDEX IF NOT EXISTS idx_pending_signups_email ON pending_signups(email);
CREATE INDEX IF NOT EXISTS idx_pending_signups_unapplied ON pending_signups(applied_at) WHERE applied_at IS NULL;
-- ── subscription_reminders ─────────────────────────────────────────────
-- Dedup ledger for the self-serve expiry-reminder emails. The relay owns
-- the subscription expiry; a daily Recaps scan asks it who's expiring and
-- emails them. This table guarantees each (user, period, kind) email goes
-- out at most once. period_expires_at is the ISO expiry instant the
-- reminder is for — when the user renews, expiry changes, so a fresh set
-- of reminders re-arms for the new period without re-sending old ones.
-- kind is one of 'upcoming_7d', 'upcoming_1d', or 'lapsed'.
CREATE TABLE IF NOT EXISTS subscription_reminders (
user_id TEXT NOT NULL,
period_expires_at TEXT NOT NULL,
kind TEXT NOT NULL,
sent_at INTEGER NOT NULL,
PRIMARY KEY (user_id, period_expires_at, kind)
);
-- ── library_meta ───────────────────────────────────────────────────────
-- Index over /data/history/<userId>/<sessionId>.json. The summary
-- content stays on disk; this table is just for fast listing without
-- scanning the filesystem.
CREATE TABLE IF NOT EXISTS library_meta (
session_id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
video_id TEXT,
url TEXT,
title TEXT,
type TEXT,
topic_count INTEGER,
segment_count INTEGER,
created_at INTEGER NOT NULL,
upload_date TEXT
);
CREATE INDEX IF NOT EXISTS idx_library_user ON library_meta(user_id, created_at DESC);
`;
// initDb({ dataDir })
// Idempotent. Opens /data/recap.db, applies the schema, returns the
// connection. Safe to call multiple times — repeat calls return the
// existing handle.
export async function initDb({ dataDir }) {
if (dbInstance) return dbInstance;
// Lazy import so single-mode never loads the native binding.
const { default: Database } = await import("better-sqlite3");
const dbPath = path.join(dataDir, "recap.db");
const db = new Database(dbPath);
// WAL mode for the obvious reasons: concurrent reads while a write
// is in flight, and durable enough for our small write volume
// (signups, sessions, library inserts). `synchronous = NORMAL` is
// the standard pairing — fsync on checkpoint, not every commit.
db.pragma("journal_mode = WAL");
db.pragma("synchronous = NORMAL");
db.pragma("foreign_keys = ON");
db.exec(SCHEMA_SQL);
// ── In-place schema migrations ──────────────────────────────────────
// SCHEMA_SQL above is the FRESH-INSTALL schema. Existing installs
// may have an older shape (e.g. tenant_credits with the legacy
// `balance` column). We bring them up to current by introspecting
// PRAGMA table_info and ALTER-ing only where needed. Each migration
// is idempotent — running boot multiple times is safe.
migrateTenantCreditsSchema(db);
migrateMagicLinkTokensTrialCookie(db);
migrateUsersTier(db);
migrateUserDigestPrefs(db);
dbInstance = db;
console.log(`[db] opened ${dbPath} (multi-tenant store)`);
return db;
}
// Core-decoupling — add users.tier to existing DBs (fresh installs get it
// from SCHEMA_SQL). Idempotent: ALTERs only when the column is missing.
function migrateUsersTier(db) {
let cols;
try {
cols = db.prepare("PRAGMA table_info(users)").all();
} catch {
return;
}
if (!cols.some((c) => c.name === "tier")) {
db.exec("ALTER TABLE users ADD COLUMN tier TEXT NOT NULL DEFAULT 'core'");
console.log("[db] added users.tier column (core-decoupling)");
}
}
// Daily Digest — add the opt-in columns to existing DBs (fresh installs get
// them from SCHEMA_SQL). Idempotent: ALTERs only the columns still missing.
function migrateUserDigestPrefs(db) {
let cols;
try {
cols = db.prepare("PRAGMA table_info(users)").all();
} catch {
return;
}
if (!cols.some((c) => c.name === "digest_enabled")) {
db.exec("ALTER TABLE users ADD COLUMN digest_enabled INTEGER NOT NULL DEFAULT 0");
console.log("[db] added users.digest_enabled column (daily-digest)");
}
if (!cols.some((c) => c.name === "last_digest_at")) {
db.exec("ALTER TABLE users ADD COLUMN last_digest_at INTEGER");
console.log("[db] added users.last_digest_at column (daily-digest)");
}
if (!cols.some((c) => c.name === "digest_unsub_token")) {
db.exec("ALTER TABLE users ADD COLUMN digest_unsub_token TEXT");
console.log("[db] added users.digest_unsub_token column (daily-digest)");
}
// Created here (not in SCHEMA_SQL) so it runs AFTER the column exists on
// both fresh and migrated DBs. Idempotent. Keeps the public unsubscribe
// token lookup off a full-table scan.
db.exec(
"CREATE INDEX IF NOT EXISTS idx_users_unsub_token ON users(digest_unsub_token)",
);
}
// v0.2.92 — split the single tenant_credits.balance into two buckets
// (purchased + replenish) so we can refill the latter periodically
// without wiping the former.
function migrateTenantCreditsSchema(db) {
let cols;
try {
cols = db.prepare("PRAGMA table_info(tenant_credits)").all();
} catch {
return; // table doesn't exist yet (shouldn't happen post-SCHEMA_SQL)
}
const colNames = new Set(cols.map((c) => c.name));
// 1. Rename legacy `balance` → `purchased_balance`. Existing balances
// were a mix of signup-grant + admin-grant + purchase; treating
// them all as "purchased" (permanent) is the safe interpretation
// — we'd rather over-preserve than wipe credits on upgrade.
if (colNames.has("balance") && !colNames.has("purchased_balance")) {
db.exec(
"ALTER TABLE tenant_credits RENAME COLUMN balance TO purchased_balance",
);
console.log(
"[db] migrated tenant_credits.balance → tenant_credits.purchased_balance",
);
colNames.delete("balance");
colNames.add("purchased_balance");
}
if (!colNames.has("replenish_balance")) {
db.exec(
"ALTER TABLE tenant_credits ADD COLUMN replenish_balance INTEGER NOT NULL DEFAULT 0",
);
console.log("[db] added tenant_credits.replenish_balance");
}
if (!colNames.has("last_replenish_at")) {
db.exec(
"ALTER TABLE tenant_credits ADD COLUMN last_replenish_at INTEGER",
);
console.log("[db] added tenant_credits.last_replenish_at");
}
}
// v0.2.104 — add trial_cookie_id to magic_link_tokens so cross-cookie-
// jar magic-link clicks (Safari Private → Gmail webview, etc.) still
// link the anon trial to the new user at /auth/verify time. Existing
// installs get the column added in-place; pre-existing rows just keep
// trial_cookie_id = NULL (no linking via the new path, falls back to
// the legacy req.cookies path).
function migrateMagicLinkTokensTrialCookie(db) {
let cols;
try {
cols = db.prepare("PRAGMA table_info(magic_link_tokens)").all();
} catch {
return;
}
const colNames = new Set(cols.map((c) => c.name));
if (!colNames.has("trial_cookie_id")) {
db.exec(
"ALTER TABLE magic_link_tokens ADD COLUMN trial_cookie_id TEXT",
);
console.log("[db] added magic_link_tokens.trial_cookie_id");
}
}
// Returns the open handle. Throws if initDb hasn't run — that's a
// programming error (some single-mode caller reached a multi-mode
// codepath). Callers in multi-mode should assume the handle exists.
export function getDb() {
if (!dbInstance) {
throw new Error(
"[db] getDb() called before initDb(); check RECAP_MODE wiring",
);
}
return dbInstance;
}
// Test/teardown helper. Closes the connection so the next initDb()
// call reopens fresh. Not used in production.
export function closeDb() {
if (dbInstance) {
dbInstance.close();
dbInstance = null;
}
}
+285
View File
@@ -0,0 +1,285 @@
// Magic-link email body builder. Returns { subject, text, html } for
// nodemailer. Keeps the HTML and text in sync — both carry the same
// verifyUrl and the same expiry copy.
//
// Style is deliberately minimal: one paragraph, one button, no images,
// no fancy CSS. Spam filters like simple emails; users skim them and
// click the link. Anything fancier risks the email landing in spam,
// which is fatal to a magic-link auth flow.
// renderMagicLinkEmail({ verifyUrl, brandName, expiresInMinutes })
// → { subject, text, html }
export function renderMagicLinkEmail({
verifyUrl,
brandName = "Recaps",
expiresInMinutes = 15,
}) {
const subject = `Sign in to ${brandName}`;
const text = [
`Sign in to ${brandName} by opening this link:`,
"",
verifyUrl,
"",
`This link expires in ${expiresInMinutes} minutes and can only be used once.`,
"",
`If you didn't request this, you can safely ignore this email — no one else can use this link without access to your inbox.`,
].join("\n");
// Inline-styled HTML. Most email clients strip <style> blocks, so
// everything that needs to look right has to be inline.
const html = `<!doctype html>
<html>
<body style="margin:0;padding:0;background:#fafafa;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background:#fafafa;padding:32px 0;">
<tr>
<td align="center">
<table role="presentation" width="480" cellpadding="0" cellspacing="0" style="background:#ffffff;border-radius:8px;padding:32px;max-width:90%;">
<tr>
<td style="font-size:18px;font-weight:600;color:#111;padding-bottom:16px;">
Sign in to ${escapeHtml(brandName)}
</td>
</tr>
<tr>
<td style="font-size:15px;line-height:1.5;color:#444;padding-bottom:24px;">
Click the button below to sign in. This link expires in ${expiresInMinutes} minutes and can only be used once.
</td>
</tr>
<tr>
<td align="center" style="padding-bottom:24px;">
<a href="${escapeAttr(verifyUrl)}" style="display:inline-block;background:#111;color:#fff;text-decoration:none;font-size:15px;font-weight:500;padding:12px 24px;border-radius:6px;">Sign in</a>
</td>
</tr>
<tr>
<td style="font-size:13px;line-height:1.5;color:#888;padding-bottom:8px;">
Or copy and paste this URL into your browser:
</td>
</tr>
<tr>
<td style="font-size:12px;color:#888;word-break:break-all;padding-bottom:24px;">
${escapeHtml(verifyUrl)}
</td>
</tr>
<tr>
<td style="font-size:12px;line-height:1.5;color:#888;border-top:1px solid #eee;padding-top:16px;">
If you didn't request this, you can safely ignore this email — no one can use this link without access to your inbox.
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>`;
return { subject, text, html };
}
// renderSubscriptionReminderEmail({ brandName, tier, expiresAt, daysLeft,
// kind, manageUrl }) → { subject, text, html }
// kind: 'upcoming_7d' | 'upcoming_1d' | 'lapsed'. Same minimal,
// spam-filter-friendly style as the magic-link email: one message, one
// button. `expiresAt` is an ISO string or Date; `daysLeft` is a number
// (<= 0 means already expired).
export function renderSubscriptionReminderEmail({
brandName = "Recaps",
tier = "pro",
expiresAt,
daysLeft = 0,
kind = "upcoming_7d",
manageUrl,
}) {
const tierLabel = tier === "max" ? "Max" : "Pro";
const lapsed = kind === "lapsed";
let when;
if (lapsed) when = "has expired";
else if (daysLeft <= 1) when = "expires tomorrow";
else when = `expires in ${daysLeft} days`;
let expiryDateStr = "";
try {
const d = expiresAt instanceof Date ? expiresAt : new Date(expiresAt);
if (!Number.isNaN(d.getTime())) {
expiryDateStr = d.toLocaleDateString("en-US", {
year: "numeric",
month: "long",
day: "numeric",
});
}
} catch {}
const subject = lapsed
? `Your ${brandName} ${tierLabel} plan has expired`
: `Your ${brandName} ${tierLabel} plan ${when}`;
const lead = lapsed
? `Your ${brandName} ${tierLabel} plan has expired${expiryDateStr ? ` (${expiryDateStr})` : ""}, so your account is back on the free Core tier. Renew anytime to restore ${tierLabel} — it's a one-time payment, no auto-charges.`
: `Your ${brandName} ${tierLabel} plan ${when}${expiryDateStr ? ` (${expiryDateStr})` : ""}. Renew to keep your ${tierLabel} features — it's a one-time payment for another period, no auto-charges.`;
const cta = lapsed ? `Renew ${tierLabel}` : `Renew now`;
const text = [
lapsed
? `Your ${brandName} ${tierLabel} plan has expired.`
: `Your ${brandName} ${tierLabel} plan ${when}.`,
"",
lead,
"",
`${cta}: ${manageUrl}`,
"",
`You're receiving this because you have a ${brandName} account. Prepaid plans never auto-renew — you're only charged when you choose to.`,
].join("\n");
const html = `<!doctype html>
<html>
<body style="margin:0;padding:0;background:#fafafa;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background:#fafafa;padding:32px 0;">
<tr>
<td align="center">
<table role="presentation" width="480" cellpadding="0" cellspacing="0" style="background:#ffffff;border-radius:8px;padding:32px;max-width:90%;">
<tr>
<td style="font-size:18px;font-weight:600;color:#111;padding-bottom:16px;">
Your ${escapeHtml(brandName)} ${escapeHtml(tierLabel)} plan ${escapeHtml(lapsed ? "has expired" : when)}
</td>
</tr>
<tr>
<td style="font-size:15px;line-height:1.5;color:#444;padding-bottom:24px;">
${escapeHtml(lead)}
</td>
</tr>
<tr>
<td align="center" style="padding-bottom:24px;">
<a href="${escapeAttr(manageUrl)}" style="display:inline-block;background:#111;color:#fff;text-decoration:none;font-size:15px;font-weight:500;padding:12px 24px;border-radius:6px;">${escapeHtml(cta)}</a>
</td>
</tr>
<tr>
<td style="font-size:12px;line-height:1.5;color:#888;border-top:1px solid #eee;padding-top:16px;">
You're receiving this because you have a ${escapeHtml(brandName)} account. Prepaid plans never auto-renew — you're only charged when you choose to.
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>`;
return { subject, text, html };
}
// renderDigestEmail({ brandName, episodes, overflowCount, manageUrl,
// unsubscribeUrl }) → { subject, text, html }
// episodes: [{ title, type, url, overview }] — already capped + synthesized
// by the scan. overflowCount: how many more are in the library beyond the
// shown set (0 = none). Same minimal, spam-filter-friendly style as the
// other emails: no images, inline CSS, one CTA. The unsubscribe link is a
// one-click GET (no login) — required for deliverability + consent.
export function renderDigestEmail({
brandName = "Recaps",
episodes = [],
overflowCount = 0,
manageUrl,
unsubscribeUrl,
}) {
const n = episodes.length;
const subject =
n === 1
? `Your ${brandName} digest: 1 new recap`
: `Your ${brandName} digest: ${n} new recaps`;
const typeLabel = (t) =>
t === "podcast" ? "Podcast" : t === "youtube" ? "Video" : "Recording";
const epText = episodes
.map((ep) =>
[
`${ep.title || "Untitled"} (${typeLabel(ep.type)})`,
ep.overview || "",
ep.url || "",
]
.filter(Boolean)
.join("\n"),
)
.join("\n\n");
const text = [
`Here's what you added to ${brandName} in the last day:`,
"",
epText,
"",
overflowCount > 0
? `…and ${overflowCount} more in your library: ${manageUrl}`
: `Open your library: ${manageUrl}`,
"",
`You're receiving this because you turned on the daily digest. Unsubscribe: ${unsubscribeUrl}`,
].join("\n");
const episodeBlocks = episodes
.map((ep) => {
const title = escapeHtml(ep.title || "Untitled");
const titleHtml = ep.url
? `<a href="${escapeAttr(ep.url)}" style="color:#111;text-decoration:none;">${title}</a>`
: title;
return `
<tr>
<td style="padding-bottom:20px;border-bottom:1px solid #eee;">
<div style="font-size:11px;text-transform:uppercase;letter-spacing:0.5px;color:#999;padding-bottom:4px;">${escapeHtml(typeLabel(ep.type))}</div>
<div style="font-size:16px;font-weight:600;color:#111;padding-bottom:8px;line-height:1.35;">${titleHtml}</div>
<div style="font-size:14px;line-height:1.55;color:#444;">${escapeHtml(ep.overview || "")}</div>
</td>
</tr>
<tr><td style="height:20px;"></td></tr>`;
})
.join("");
const overflowHtml =
overflowCount > 0
? `<tr><td style="font-size:13px;color:#888;padding-bottom:16px;">…and ${overflowCount} more in your library.</td></tr>`
: "";
const html = `<!doctype html>
<html>
<body style="margin:0;padding:0;background:#fafafa;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background:#fafafa;padding:32px 0;">
<tr>
<td align="center">
<table role="presentation" width="520" cellpadding="0" cellspacing="0" style="background:#ffffff;border-radius:8px;padding:32px;max-width:90%;">
<tr>
<td style="font-size:18px;font-weight:600;color:#111;padding-bottom:20px;">
Your ${escapeHtml(brandName)} digest
</td>
</tr>
${episodeBlocks}
${overflowHtml}
<tr>
<td align="center" style="padding:8px 0 24px;">
<a href="${escapeAttr(manageUrl)}" style="display:inline-block;background:#111;color:#fff;text-decoration:none;font-size:15px;font-weight:500;padding:12px 24px;border-radius:6px;">Open your library</a>
</td>
</tr>
<tr>
<td style="font-size:12px;line-height:1.5;color:#888;border-top:1px solid #eee;padding-top:16px;">
You're receiving this because you turned on the daily digest. <a href="${escapeAttr(unsubscribeUrl)}" style="color:#888;">Unsubscribe</a> anytime, or manage it in Settings.
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>`;
return { subject, text, html };
}
function escapeHtml(s) {
return String(s)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#39;");
}
function escapeAttr(s) {
return escapeHtml(s);
}
+64 -10
View File
@@ -9,10 +9,13 @@ import { formatTime } from "./util.js";
// numbers are operational data, not configuration. Update when Google
// changes published rates.
export const PRICING = {
"gemini-3-flash-preview": { input: 0.50, output: 3.00, thinking: 3.00 },
"gemini-3-pro-preview": { input: 2.00, output: 12.00, thinking: 12.00 },
// The five Gemini models we support. Verified against Google's
// official docs on 2026-05-12. Retired/never-existed IDs omitted.
"gemini-3.1-pro-preview": { input: 2.00, output: 12.00, thinking: 12.00 },
"gemini-2.5-pro": { input: 1.25, output: 10.00, thinking: 10.00 },
"gemini-3-flash-preview": { input: 0.50, output: 3.00, thinking: 3.00 },
"gemini-2.5-flash": { input: 0.15, output: 0.60, thinking: 0.60 },
"gemini-3.1-flash-lite": { input: 0.10, output: 0.40, thinking: 0.40 },
// Fallback for unknown / future models — better an estimate than nothing.
"default": { input: 1.00, output: 5.00, thinking: 5.00 },
};
@@ -46,16 +49,67 @@ export function calcCost(modelName, usage) {
};
}
// ── Section-count target by VIDEO duration ─────────────────────────────────
// Mirrors recap-relay's computePerWindowTarget() (server/chunked-analyze.js).
// Operator-tunable on the relay; baked into code defaults here on the
// Recap-app direct path. The defaults match the relay's defaults so
// segmentation density is consistent across both pipelines.
//
// Buckets are TOTAL video duration in minutes:
// <30 → 6 sections / 30-60 → 8 / 60-90 → 9 / 90-120 → 10
// 120-150 → 11 / 150-180 → 12 / >=180 → 12
// Per-window target = total_target × window_sec / total_audio_sec
// (clamped to ≥1 for single-shot runs).
function pickTotalSectionsTarget(totalAudioSec) {
const m = (totalAudioSec || 0) / 60;
if (m < 30) return 6;
if (m < 60) return 8;
if (m < 90) return 9;
if (m < 120) return 10;
if (m < 150) return 11;
if (m < 180) return 12;
return 12;
}
function formatTargetSectionsLabel(avg) {
if (avg <= 1.2) return "1 section";
const lo = Math.max(1, Math.floor(avg));
const hi = Math.max(lo, Math.ceil(avg));
if (lo === hi) return "around " + lo + " sections";
return lo + "" + hi + " sections";
}
// ── Topic-analysis prompt builder ───────────────────────────────────────────
// Takes the parsed transcript entries and builds the JSON-output prompt
// fed to the analysis model. Indices in the response are positional into
// the same `entries` array — the caller relies on that contract.
export function buildAnalysisPrompt(entries) {
// Takes the parsed transcript entries for a WINDOW and builds the
// JSON-output prompt fed to the analysis model. Indices in the response
// are positional into the same window-entries array — the caller relies
// on that contract.
//
// `opts.totalAudioSec` is the FULL audio duration (not just this window),
// used to scale the section-count target via the per-video-duration table
// above. When omitted, falls back to deriving from the windowEntries
// themselves (legacy callers / unit tests / single-shot path).
export function buildAnalysisPrompt(entries, opts = {}) {
const numbered = entries
.map((e, i) => `[${i}] (${formatTime(e.offset)}) ${e.text}`)
.join("\n");
return `You are analyzing a video transcript. Your job is to identify natural topic boundaries and group the transcript into discussion-based sections.
// Window length in minutes (this window's own transcript span).
const windowSec = entries.length > 1
? (entries[entries.length - 1].offset || 0) - (entries[0].offset || 0)
: 0;
const windowMin = Math.max(1, Math.round(windowSec / 60));
const maxIndex = Math.max(0, entries.length - 1);
// Total audio duration drives the per-video-duration target picker.
// If the caller didn't supply it, assume this is a single-shot run
// and the window IS the whole audio.
const totalAudioSec = opts.totalAudioSec || windowSec || 60;
const totalTarget = pickTotalSectionsTarget(totalAudioSec);
const numWindows = Math.max(1, totalAudioSec / Math.max(60, windowSec || 60));
const avgPerWindow = totalTarget / numWindows;
const targetSections = formatTargetSectionsLabel(avgPerWindow);
return `You are analyzing a ~${windowMin}-minute section of a longer transcript. Your job is to identify natural topic boundaries and group the transcript into discussion-based sections — aim for ${targetSections}.
TRANSCRIPT (each line is numbered with a timestamp):
${numbered}
@@ -67,13 +121,13 @@ INSTRUCTIONS:
4. For each section, write:
- A short, specific topic title (3-8 words)
- A 1-3 sentence summary of what's discussed
- The start and end segment indices (inclusive)
- The start and end segment indices (inclusive), counted as the bracketed [N] number at the start of each transcript line above.
IMPORTANT:
- Sections must be chronological and non-overlapping.
- Every segment index from 0 to ${entries.length - 1} must belong to exactly one section.
- Every segment index from 0 to ${maxIndex} must belong to exactly one section.
- startIndex of section N+1 must equal endIndex of section N plus 1.
- Create as many or as few sections as the content naturally requires.
- Create as many or as few sections as the content naturally requires — but lean toward broad, substantive topics rather than minute-by-minute breakdowns. A natural topic that spans several minutes of dialogue should be one section, not several.
- Titles should be descriptive and specific, not generic like "Introduction" unless it truly is one.
Respond with ONLY valid JSON in this exact format, no other text:
+510 -87
View File
@@ -1,33 +1,136 @@
// History storage + routes. Sessions are written as one JSON file per
// summary in /data/history/<id>.json. Folder structure / ordering lives
// in a sidecar `_meta.json`.
// History storage + routes. Per-user-scoped under /data/history/<scope>/.
//
// Module-private state: just the historyDir path, set by initHistory().
// The DELETE route needs to add the deleted videoId to the skip list
// (so subscriptions don't re-queue it) — that's a cross-module concern,
// so it's injected as a callback by setupHistoryRoutes.
// "Scope" is:
// - single mode: "owner" (always)
// - multi mode signed-in user: "<user_id>"
// - multi mode anonymous trial: "anon/<trial_cookie_id>"
//
// Each scope has its own folder with one *.json file per summary and a
// `_meta.json` for the folder/ordering UI. Scope isolation is enforced
// here at the path level — handlers in this file refuse to read across
// scopes, period. The auth middleware populates req.userId; we derive
// the scope via scopeForRequest(req) and never trust raw URL input.
//
// On a brand-new install nothing exists. Single-mode installs created
// before 0.2.77 wrote files flat to /data/history/*.json; the migration
// hook (see migrateLegacyLibrary below) moves those into the "owner"
// scope on first multi-mode boot.
//
// Module-private state: historyDir (the root path). All per-scope
// paths are derived per-call so adding a new user doesn't need a
// re-init.
import fs from "fs/promises";
import path from "path";
let historyDir = null;
let metaPath = null;
// ── Initialization ──────────────────────────────────────────────────────────
// Call once at boot. Creates the directory and stores the path. Idempotent.
export async function initHistory({ dataDir }) {
// Call once at boot. Creates the root directory and stores the path.
// In single mode also ensures /data/history/owner exists so the
// owner-scope writes don't race on first-summary mkdir.
export async function initHistory({ dataDir, mode = "single" }) {
historyDir = path.join(dataDir, "history");
metaPath = path.join(historyDir, "_meta.json");
await fs.mkdir(historyDir, { recursive: true }).catch(() => {});
if (mode === "single") {
await fs.mkdir(ownerScopeDir(), { recursive: true }).catch(() => {});
}
}
// ── Scope helpers ───────────────────────────────────────────────────────────
// Files that live at the root of /data/history/ (not inside a per-user
// scope) — subscription state, skip lists, etc. Filtered out when
// listing sessions so they don't appear as phantom library items.
// Files that live inside a scope dir but are NOT session records:
// the folder/order meta + the per-scope subscription state (moved here by
// the 0.2.147 migration). They must be filtered out of every place that
// lists `.json` files as sessions, or they show up as phantom "Invalid
// Date · undefined topics" library entries.
export const ROOT_SIDECARS = new Set([
"_meta.json",
"subscriptions.json",
"skip-list.json",
"seen-list.json",
"auto-queue.json",
]);
// Sanitize a user-supplied scope component so it can't escape the
// history root via path traversal. Allows the alphabet that user_ids
// and trial cookie_ids use (base64url + hex chars + the literal "anon"
// and "owner" prefixes). Anything else → throw.
function safeComponent(s) {
if (typeof s !== "string" || !s) throw new Error("invalid_scope_component");
if (!/^[A-Za-z0-9_-]+$/.test(s)) throw new Error("invalid_scope_component");
return s;
}
// scopeForRequest(req) — single string identifying the writer/reader
// of a library. Used as a subpath under /data/history/. Throws if the
// request has no usable identity (caller should 401 in that case).
//
// Returned strings:
// "owner" — single mode, OR the multi-mode admin (so
// a multi→single mode flip preserves the
// operator's library at the same path)
// "<user_id>" — multi mode non-admin signed-in user
// "anon/<cookie_id>" — multi mode anonymous-trial cookie
//
// Why admin → "owner": before v0.2.91 we renamed /data/history/owner/
// → /data/history/<admin_user_id>/ on first multi-mode signup, which
// made switching back to single mode hide the operator's library
// (single mode reads "owner"). Keeping admin's scope at "owner"
// regardless of mode makes mode-switching lossless.
export function scopeForRequest(req) {
if (req.recapMode !== "multi") return "owner";
if (req.user && req.user.is_admin) return "owner";
if (req.user && req.user.id) return safeComponent(req.user.id);
if (typeof req.userId === "string" && req.userId.startsWith("anon:")) {
return `anon/${safeComponent(req.userId.slice(5))}`;
}
if (req.userId === "owner") return "owner"; // pre-multi-mode legacy shim
throw new Error("no_scope");
}
function scopeDir(scope) {
// `scope` may contain a slash for the "anon/<id>" case — split into
// segments so path.join doesn't treat it as one component (and so
// safeComponent enforcement covers each piece).
const parts = scope.split("/").map(safeComponent);
return path.join(historyDir, ...parts);
}
function ownerScopeDir() {
return path.join(historyDir, "owner");
}
function metaPathFor(scope) {
return path.join(scopeDir(scope), "_meta.json");
}
// ── Storage ─────────────────────────────────────────────────────────────────
// saveToHistory persists a completed summary. Returns the generated id.
// Used by /api/process. The id encodes the timestamp + a content hint
// (videoId for YouTube, base64-truncated guid/url for podcasts) so files
// sort chronologically by name.
export async function saveToHistory(videoId, url, title, chunks, entries, logs, uploadDate, type) {
const idSuffix = type === "podcast"
// Caller (the /api/process handler) is responsible for passing the
// right scope — derived via scopeForRequest(req) up the call stack.
//
// The id encodes the timestamp + a content hint (videoId for YouTube,
// base64-truncated guid/url for podcasts) so files sort chronologically
// by name.
export async function saveToHistory(
scope,
videoId,
url,
title,
chunks,
entries,
logs,
uploadDate,
type,
speakers = null,
speakerNames = null,
) {
const idSuffix =
type === "podcast"
? Buffer.from(videoId).toString("base64url").slice(0, 16)
: videoId;
const id = `${Date.now()}-${idSuffix}`;
@@ -44,101 +147,389 @@ export async function saveToHistory(videoId, url, title, chunks, entries, logs,
chunks,
entries,
logs,
// Phase 1E — speaker legend summary keyed by global speaker ID
// (Speaker_A, Speaker_B, ...). Each chunk's entries also carry
// `.speaker` and `.speaker_confidence` fields inline. Null when
// diarization wasn't available (older relay, off, or no
// fingerprints collected). Persisting at the record level lets
// the library card show "2 speakers" without scanning entries.
speakers: speakers || null,
// Phase 2 — inferred speaker names from the relay's post-cluster
// polish pass. Map { Speaker_A: "Matt Hill", ... } with null
// values for unidentified speakers. Reopening a saved session
// restores names alongside the cluster IDs.
speakerNames: speakerNames || null,
};
await fs.writeFile(path.join(historyDir, `${id}.json`), JSON.stringify(record));
const dir = scopeDir(scope);
await fs.mkdir(dir, { recursive: true }).catch(() => {});
await fs.writeFile(path.join(dir, `${id}.json`), JSON.stringify(record));
return id;
}
// ── Meta ────────────────────────────────────────────────────────────────────
// `_meta.json` shape: { folders: [{ id, name, order, collapsed,
// items: [sessionId, ...] }], uncategorized: [sessionId, ...] }
export async function loadMeta() {
// Each scope has its own `_meta.json` for folder/ordering UI state.
// New scope = empty meta on read (no file yet).
export async function loadMeta(scope) {
try {
return JSON.parse(await fs.readFile(metaPath, "utf-8"));
return JSON.parse(await fs.readFile(metaPathFor(scope), "utf-8"));
} catch {
return { folders: [], uncategorized: [] };
}
}
export async function saveMeta(meta) {
await fs.writeFile(metaPath, JSON.stringify(meta, null, 2));
export async function saveMeta(scope, meta) {
const dir = scopeDir(scope);
await fs.mkdir(dir, { recursive: true }).catch(() => {});
await fs.writeFile(metaPathFor(scope), JSON.stringify(meta, null, 2));
}
// getHistoryDir() — root /data/history/. Some callers (subscriptions,
// skip-list, etc.) write sidecar files here that aren't scoped.
export function getHistoryDir() {
return historyDir;
}
// ── Routes ──────────────────────────────────────────────────────────────────
// Pass `addToSkipList` so the DELETE route can suppress re-queueing of
// videos the user has explicitly removed. Decoupled from subscriptions
// to keep this module standalone.
export function setupHistoryRoutes(app, { addToSkipList } = {}) {
// Get all history: sessions + folder structure
app.get("/api/history", async (req, res) => {
// getScopeHistoryDir(scope) — the per-scope directory. Used by handlers
// that need raw filesystem access (e.g. delete).
export function getScopeHistoryDir(scope) {
return scopeDir(scope);
}
// ── Audio-first ("walking mode") TTS cache helpers ──────────────────────────
// Per-topic synthesized summary clips live alongside the session JSON in
// a sibling folder: /data/history/<scope>/<id>-audio/topic-<i>.mp3. Same
// scope-isolation guarantees as the session record (safeFilename guards
// the id; scopeDir guards the scope).
// Directory holding a session's cached summary-audio clips.
export function sessionAudioDir(scope, id) {
return path.join(scopeDir(scope), `${safeFilename(id)}-audio`);
}
// Load a full session record by id within a scope. Returns null if it
// doesn't exist (or can't be parsed) — callers 404 on null.
export async function loadSession(scope, id) {
try {
const files = await fs.readdir(historyDir);
const sessionsMap = {};
// Skip the meta + state files; everything else is a session.
for (const file of files.filter(f =>
f.endsWith(".json") &&
!f.startsWith("_") &&
f !== "subscriptions.json" &&
f !== "skip-list.json" &&
f !== "seen-list.json" &&
f !== "auto-queue.json"
const raw = await fs.readFile(
path.join(scopeDir(scope), `${safeFilename(id)}.json`),
"utf-8",
);
return JSON.parse(raw);
} catch {
return null;
}
}
// List a scope's saved sessions as lightweight metadata (no entries /
// chunks), oldest first. The daily-digest scan uses this to pick recaps
// created after a watermark before loading each full record for
// synthesis. Returns [] when the scope has no library yet (or the id is
// malformed — safeComponent throws inside scopeDir, caught here).
export async function listScopeSessions(scope) {
let dir;
try {
dir = scopeDir(scope);
} catch {
return [];
}
let files = [];
try {
files = await fs.readdir(dir);
} catch {
return [];
}
const out = [];
for (const file of files.filter(
(f) => f.endsWith(".json") && !f.startsWith("_") && !ROOT_SIDECARS.has(f),
)) {
try {
const raw = await fs.readFile(path.join(historyDir, file), "utf-8");
const data = JSON.parse(await fs.readFile(path.join(dir, file), "utf-8"));
out.push({
id: data.id,
title: data.title,
type: data.type || "youtube",
url: data.url,
createdAt: data.createdAt,
});
} catch {}
}
out.sort((a, b) => new Date(a.createdAt) - new Date(b.createdAt));
return out;
}
// Shallow-merge `patch` into a session record on disk (e.g. to stamp
// `summaryAudio` availability). No-op-safe: returns null if the record
// is missing rather than throwing.
export async function patchSession(scope, id, patch) {
const file = path.join(scopeDir(scope), `${safeFilename(id)}.json`);
let rec;
try {
rec = JSON.parse(await fs.readFile(file, "utf-8"));
} catch {
return null;
}
const merged = { ...rec, ...patch };
await fs.writeFile(file, JSON.stringify(merged));
return merged;
}
// ── Legacy library migration (single → multi) ───────────────────────────────
// Pre-0.2.77 single-mode installs wrote summaries flat to
// /data/history/*.json with a single _meta.json. On first boot in
// multi mode we move all of that into /data/history/owner/ so the
// operator's library is accessible under the "owner" scope. After the
// first real user signs up (is_admin=1), auth-routes.js renames that
// folder to the user's actual id so they own their original library.
//
// Idempotent — writes a sentinel after the first migration. Safe to
// call on every boot; no-op if there's nothing flat to move.
export async function migrateLegacyLibrary() {
const sentinel = path.join(historyDir, ".migrated_to_multi");
try {
await fs.access(sentinel);
return { migrated: 0, skipped: "already_migrated" };
} catch {}
let entries = [];
try {
entries = await fs.readdir(historyDir);
} catch {
return { migrated: 0, skipped: "no_history_dir" };
}
const flatFiles = entries.filter(
(f) => f.endsWith(".json") && !ROOT_SIDECARS.has(f),
);
if (flatFiles.length === 0 && !entries.includes("_meta.json")) {
// Truly empty. Write the sentinel so future boots don't keep
// checking, but flag this as a non-migration.
await fs.writeFile(sentinel, new Date().toISOString());
return { migrated: 0, skipped: "empty_legacy_library" };
}
const target = ownerScopeDir();
await fs.mkdir(target, { recursive: true }).catch(() => {});
let moved = 0;
for (const f of flatFiles) {
try {
await fs.rename(
path.join(historyDir, f),
path.join(target, f),
);
moved += 1;
} catch (err) {
console.warn(`[history] failed to migrate ${f}:`, err?.message || err);
}
}
// Move _meta.json too if it exists at the root. Folder/ordering state
// belongs to the same library.
try {
await fs.rename(
path.join(historyDir, "_meta.json"),
path.join(target, "_meta.json"),
);
} catch {} // no _meta.json is fine
await fs.writeFile(sentinel, new Date().toISOString());
console.log(
`[history] migrated ${moved} legacy session(s) to /data/history/owner/`,
);
return { migrated: moved };
}
// reclaimAdminLibraryToOwner({ db }) — one-time fixup for installs
// upgraded from <0.2.91 where the first admin's library got renamed
// from /data/history/owner/ → /data/history/<admin_user_id>/ on their
// signup. With the new admin-scope-is-always-owner rule (above), we
// need that library back under "owner" so multi-mode admin reads it
// AND a future single-mode flip can still find it. Idempotent — runs
// the rename only if BOTH (a) an admin user exists in SQLite AND
// (b) /data/history/<admin_user_id>/ exists AND (c) /data/history/owner/
// does NOT already exist. Otherwise no-op.
//
// Pass in the better-sqlite3 db handle from db.js — we don't import
// here to avoid a dep cycle (db.js is multi-mode only, history.js is
// loaded in single mode too).
export async function reclaimAdminLibraryToOwner({ db }) {
if (!db) return { reclaimed: false, reason: "no_db" };
let admin;
try {
admin = db
.prepare(
"SELECT id, email FROM users WHERE is_admin = 1 ORDER BY created_at ASC LIMIT 1",
)
.get();
} catch {
return { reclaimed: false, reason: "no_users_table" };
}
if (!admin) return { reclaimed: false, reason: "no_admin" };
const ownerDir = path.join(historyDir, "owner");
const adminDir = path.join(historyDir, safeComponent(admin.id));
try {
await fs.access(ownerDir);
// /data/history/owner/ already exists → either a fresh install or
// the fixup already ran. Either way, do nothing.
return { reclaimed: false, reason: "owner_already_exists" };
} catch {}
try {
await fs.access(adminDir);
} catch {
return { reclaimed: false, reason: "admin_dir_missing" };
}
await fs.rename(adminDir, ownerDir);
console.log(
`[history] reclaimed admin library: /data/history/${admin.id}/ → /data/history/owner/`,
);
return { reclaimed: true, admin_id: admin.id, email: admin.email };
}
// renameScopeDir(fromScope, toScope) — atomic rename of a per-scope
// folder. Used when:
// - the first multi-mode signup claims the "owner" legacy library
// (auth-routes.js calls this with fromScope="owner", toScope=user.id)
// - an anonymous trial converts to a real user (auth-routes.js,
// fromScope="anon/<cookie_id>", toScope=user.id)
//
// If `fromScope` doesn't exist, no-op (returns false). If `toScope`
// already exists, we don't clobber — the caller has to merge manually
// (which currently only matters in edge cases, since fresh user ids
// are uuids that won't collide). Returns true on actual rename.
export async function renameScopeDir(fromScope, toScope) {
const from = scopeDir(fromScope);
const to = scopeDir(toScope);
try {
await fs.access(from);
} catch {
return false;
}
try {
await fs.access(to);
console.warn(
`[history] renameScopeDir: ${toScope} already exists; refusing to clobber. Leaving ${fromScope} in place for manual reconciliation.`,
);
return false;
} catch {}
// Ensure parent of `to` exists (for the "anon/<id>" case the parent
// is /data/history/anon/, which won't be there in fresh installs).
await fs.mkdir(path.dirname(to), { recursive: true }).catch(() => {});
await fs.rename(from, to);
console.log(`[history] renamed scope ${fromScope}${toScope}`);
return true;
}
// ── Routes ──────────────────────────────────────────────────────────────────
// All routes are scoped to req — they read scopeForRequest(req) and
// refuse to operate outside that scope. No request body or URL param
// can reference another user's library.
//
// `addToSkipList(scope, videoId)` is injected so the DELETE route can
// suppress re-queueing of a subscription video the user explicitly removed.
// It's scope-keyed (./subscriptions.js): the skip applies to the same
// scope's subscription store.
export function setupHistoryRoutes(app, { addToSkipList } = {}) {
function requireScope(req, res) {
try {
return scopeForRequest(req);
} catch {
res.status(401).json({ error: "auth_required" });
return null;
}
}
// Get all history: sessions + folder structure for THIS user.
app.get("/api/history", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const dir = scopeDir(scope);
let files = [];
try {
files = await fs.readdir(dir);
} catch {
files = []; // no library yet — render an empty state
}
const sessionsMap = {};
for (const file of files.filter(
(f) =>
f.endsWith(".json") && !f.startsWith("_") && !ROOT_SIDECARS.has(f),
)) {
try {
const raw = await fs.readFile(path.join(dir, file), "utf-8");
const data = JSON.parse(raw);
sessionsMap[data.id] = {
id: data.id, videoId: data.videoId, url: data.url,
title: data.title, topicCount: data.topicCount,
id: data.id,
videoId: data.videoId,
url: data.url,
title: data.title,
topicCount: data.topicCount,
type: data.type || "youtube",
segmentCount: data.segmentCount, createdAt: data.createdAt,
segmentCount: data.segmentCount,
createdAt: data.createdAt,
uploadDate: data.uploadDate || "",
};
} catch {}
}
const meta = await loadMeta();
const meta = await loadMeta(scope);
// Clean up: remove references to deleted sessions
for (const folder of meta.folders) {
folder.items = folder.items.filter(id => sessionsMap[id]);
folder.items = folder.items.filter((id) => sessionsMap[id]);
}
meta.uncategorized = meta.uncategorized.filter(id => sessionsMap[id]);
meta.uncategorized = meta.uncategorized.filter((id) => sessionsMap[id]);
// Add any sessions not in meta (newly created)
const allReferenced = new Set([
...meta.uncategorized,
...meta.folders.flatMap(f => f.items),
...meta.folders.flatMap((f) => f.items),
]);
const allIds = Object.keys(sessionsMap);
const orphans = allIds.filter(id => !allReferenced.has(id))
.sort((a, b) => new Date(sessionsMap[b].createdAt) - new Date(sessionsMap[a].createdAt));
const orphans = allIds
.filter((id) => !allReferenced.has(id))
.sort(
(a, b) =>
new Date(sessionsMap[b].createdAt) -
new Date(sessionsMap[a].createdAt),
);
meta.uncategorized = [...orphans, ...meta.uncategorized];
await saveMeta(meta);
await saveMeta(scope, meta);
res.json({ sessions: sessionsMap, meta });
} catch (err) {
res.json({ sessions: {}, meta: { folders: [], uncategorized: [] } });
res.json({
sessions: {},
meta: { folders: [], uncategorized: [] },
});
}
});
// Get a single session (full data)
// Get a single session (full data) — scoped to current user.
app.get("/api/history/:id", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const raw = await fs.readFile(path.join(historyDir, `${req.params.id}.json`), "utf-8");
const raw = await fs.readFile(
path.join(scopeDir(scope), `${safeFilename(req.params.id)}.json`),
"utf-8",
);
res.json(JSON.parse(raw));
} catch {
res.status(404).json({ error: "Session not found" });
}
});
// Rename a session title
// Rename a session title — scoped.
app.put("/api/history/:id/title", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const filePath = path.join(historyDir, `${req.params.id}.json`);
const filePath = path.join(
scopeDir(scope),
`${safeFilename(req.params.id)}.json`,
);
const raw = await fs.readFile(filePath, "utf-8");
const data = JSON.parse(raw);
data.title = req.body.title || data.title;
@@ -149,11 +540,16 @@ export function setupHistoryRoutes(app, { addToSkipList } = {}) {
}
});
// Delete a session — also adds the videoId to the skip list so any
// subscriptions don't re-queue it.
// Delete a session — scoped. Also adds the videoId to the (global)
// skip list so subscriptions don't re-queue it.
app.delete("/api/history/:id", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const filePath = path.join(historyDir, `${req.params.id}.json`);
const filePath = path.join(
scopeDir(scope),
`${safeFilename(req.params.id)}.json`,
);
let videoId = null;
try {
const raw = await fs.readFile(filePath, "utf-8");
@@ -163,26 +559,30 @@ export function setupHistoryRoutes(app, { addToSkipList } = {}) {
await fs.unlink(filePath);
if (videoId && typeof addToSkipList === "function") {
await addToSkipList(videoId);
await addToSkipList(scope, videoId);
}
const meta = await loadMeta();
meta.uncategorized = meta.uncategorized.filter(id => id !== req.params.id);
const meta = await loadMeta(scope);
meta.uncategorized = meta.uncategorized.filter(
(id) => id !== req.params.id,
);
for (const folder of meta.folders) {
folder.items = folder.items.filter(id => id !== req.params.id);
folder.items = folder.items.filter((id) => id !== req.params.id);
}
await saveMeta(meta);
await saveMeta(scope, meta);
res.json({ ok: true });
} catch {
res.status(404).json({ error: "Session not found" });
}
});
// Update meta (folders, ordering) — the frontend sends the full structure
// Update meta (folders, ordering) — scoped.
app.put("/api/history/meta", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const meta = req.body;
await saveMeta(meta);
await saveMeta(scope, meta);
res.json({ ok: true });
} catch (err) {
res.status(500).json({ error: err.message });
@@ -190,11 +590,18 @@ export function setupHistoryRoutes(app, { addToSkipList } = {}) {
});
app.post("/api/history/folders", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const meta = await loadMeta();
const folder = { id: `folder-${Date.now()}`, name: req.body.name || "New Folder", collapsed: false, items: [] };
const meta = await loadMeta(scope);
const folder = {
id: `folder-${Date.now()}`,
name: req.body.name || "New Folder",
collapsed: false,
items: [],
};
meta.folders.push(folder);
await saveMeta(meta);
await saveMeta(scope, meta);
res.json(folder);
} catch (err) {
res.status(500).json({ error: err.message });
@@ -202,12 +609,14 @@ export function setupHistoryRoutes(app, { addToSkipList } = {}) {
});
app.put("/api/history/folders/:id", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const meta = await loadMeta();
const folder = meta.folders.find(f => f.id === req.params.id);
const meta = await loadMeta(scope);
const folder = meta.folders.find((f) => f.id === req.params.id);
if (!folder) return res.status(404).json({ error: "Folder not found" });
folder.name = req.body.name || folder.name;
await saveMeta(meta);
await saveMeta(scope, meta);
res.json(folder);
} catch (err) {
res.status(500).json({ error: err.message });
@@ -215,48 +624,52 @@ export function setupHistoryRoutes(app, { addToSkipList } = {}) {
});
app.put("/api/history/folders/:id/collapsed", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const meta = await loadMeta();
const folder = meta.folders.find(f => f.id === req.params.id);
const meta = await loadMeta(scope);
const folder = meta.folders.find((f) => f.id === req.params.id);
if (!folder) return res.status(404).json({ error: "Folder not found" });
folder.collapsed = !!req.body.collapsed;
await saveMeta(meta);
await saveMeta(scope, meta);
res.json({ ok: true, collapsed: folder.collapsed });
} catch (err) {
res.status(500).json({ error: err.message });
}
});
// Delete a folder — items move back to uncategorized
// Delete a folder — items move back to uncategorized.
app.delete("/api/history/folders/:id", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const meta = await loadMeta();
const idx = meta.folders.findIndex(f => f.id === req.params.id);
const meta = await loadMeta(scope);
const idx = meta.folders.findIndex((f) => f.id === req.params.id);
if (idx === -1) return res.status(404).json({ error: "Folder not found" });
const [folder] = meta.folders.splice(idx, 1);
meta.uncategorized = [...folder.items, ...meta.uncategorized];
await saveMeta(meta);
await saveMeta(scope, meta);
res.json({ ok: true });
} catch (err) {
res.status(500).json({ error: err.message });
}
});
// Move a session to a folder (or uncategorized if folderId is null)
// Move a session to a folder (or uncategorized if folderId is null).
app.put("/api/history/move", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const { sessionId, folderId, index } = req.body;
const meta = await loadMeta();
const meta = await loadMeta(scope);
// Remove from current location
meta.uncategorized = meta.uncategorized.filter(id => id !== sessionId);
meta.uncategorized = meta.uncategorized.filter((id) => id !== sessionId);
for (const folder of meta.folders) {
folder.items = folder.items.filter(id => id !== sessionId);
folder.items = folder.items.filter((id) => id !== sessionId);
}
// Add to new location
if (folderId) {
const folder = meta.folders.find(f => f.id === folderId);
const folder = meta.folders.find((f) => f.id === folderId);
if (folder) {
const i = typeof index === "number" ? index : folder.items.length;
folder.items.splice(i, 0, sessionId);
@@ -266,10 +679,20 @@ export function setupHistoryRoutes(app, { addToSkipList } = {}) {
meta.uncategorized.splice(i, 0, sessionId);
}
await saveMeta(meta);
await saveMeta(scope, meta);
res.json({ ok: true });
} catch (err) {
res.status(500).json({ error: err.message });
}
});
}
// Allow the same character set as scope components for session ids.
// Belt-and-suspenders against ../../ in :id; ids generated by
// saveToHistory always match.
export function safeFilename(s) {
if (typeof s !== "string" || !/^[A-Za-z0-9_-]+$/.test(s)) {
throw new Error("invalid_session_id");
}
return s;
}
+2861 -456
View File
File diff suppressed because it is too large Load Diff
+75
View File
@@ -0,0 +1,75 @@
// Persistent per-install identifier. Generated once on first boot and
// stashed at `<DATA_DIR>/install-id` (typically /data/install-id on
// StartOS). Survives container restarts and Recap upgrades; lost on a
// full uninstall + reinstall.
//
// What it's for: the upcoming relay backend will use this ID as the
// owner of comped/paid relay credits. Without a stable client identity
// the relay can't tell whether a request belongs to a credited install
// or a fresh one. Direct install-ID auth is the v1 choice (see the
// project roadmap discussion) — simple, sufficient for low-count free
// credits, can be hardened later with license-server-minted JWTs.
//
// What it is NOT: a license key. The license system (./license.js) is
// completely separate — license keys are user-facing strings that
// authorize Pro features, while install-IDs are opaque per-install
// UUIDs the relay backend uses for credit accounting.
import fs from "fs/promises";
import path from "path";
import { randomUUID } from "crypto";
let cachedId = null;
// Initialize on boot. Reads the existing ID off disk; if there's no
// file, generates a fresh UUIDv4 and writes it. Subsequent calls to
// getInstallId() return the cached value without touching disk.
//
// `dataDir` must be writable — on StartOS that's /data (the persistent
// volume), on local dev it's the project root.
export async function initInstallId({ dataDir }) {
if (!dataDir) throw new Error("initInstallId: dataDir is required");
const filePath = path.join(dataDir, "install-id");
try {
const raw = await fs.readFile(filePath, "utf8");
const trimmed = raw.trim();
if (isValidInstallId(trimmed)) {
cachedId = trimmed;
console.log(`[install-id] loaded ${redact(cachedId)} from ${filePath}`);
return cachedId;
}
console.warn(
`[install-id] file at ${filePath} contained an invalid value — regenerating`
);
} catch (err) {
if (err.code !== "ENOENT") {
console.warn(`[install-id] read failed (${err.code}): ${err.message}`);
}
}
// No valid file — mint a new one. UUIDv4 is plenty: 122 bits of
// randomness, no collision risk across realistic install counts, and
// it's opaque enough to share over the wire without leaking system
// info (unlike e.g. a machine-id).
const fresh = randomUUID();
await fs.writeFile(filePath, fresh + "\n", { mode: 0o600 });
cachedId = fresh;
console.log(`[install-id] generated ${redact(cachedId)}${filePath}`);
return cachedId;
}
export function getInstallId() {
return cachedId;
}
// Loose UUID shape check — accepts any reasonable UUID-ish string.
// Avoids requiring v4 specifically in case operators want to seed
// non-standard IDs.
function isValidInstallId(s) {
return typeof s === "string" && /^[0-9a-f-]{32,40}$/i.test(s);
}
// Log-safe display: first 8 + last 4 chars only.
function redact(id) {
if (!id || id.length < 12) return "(short)";
return `${id.slice(0, 8)}${id.slice(-4)}`;
}
+95 -35
View File
@@ -4,45 +4,89 @@
// directly because library import predates the subscriptions module
// having a public 'merge' helper (and the merge logic is library-
// specific anyway).
//
// As of 0.2.77 (multi-tenant): all reads/writes are scoped to the
// requesting user via scopeForRequest(req). In single mode the scope
// is always "owner" — preserved single-user behavior. In multi mode
// each tenant exports/imports their own library; the operator's
// admin status doesn't grant cross-user export (a separate operator-
// only "export everyone's library" endpoint can be added later if
// the operator ever needs it).
//
// Subscriptions remain global (one /data/history/subscriptions.json
// per install) for now. Per-user subscriptions are a Phase 1D/2 task.
import fs from "fs/promises";
import path from "path";
import express from "express";
import { getHistoryDir, loadMeta, saveMeta } from "./history.js";
import {
getHistoryDir,
getScopeHistoryDir,
loadMeta,
saveMeta,
scopeForRequest,
safeFilename,
ROOT_SIDECARS,
} from "./history.js";
// ── Routes ──────────────────────────────────────────────────────────────────
// Both routes are gated by the Pro 'library' entitlement (the gate runs
// upstream in license-middleware.js). They assume initHistory() has
// already been called.
export function setupLibraryRoutes(app) {
function requireScope(req, res) {
try {
return scopeForRequest(req);
} catch {
res.status(401).json({ error: "auth_required" });
return null;
}
}
// Bulk export everything: meta, sessions, subscriptions.
app.get("/api/library/export", async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const historyDir = getHistoryDir();
const meta = await loadMeta();
const files = await fs.readdir(historyDir);
const scopeDir = getScopeHistoryDir(scope);
const meta = await loadMeta(scope);
let files = [];
try {
files = await fs.readdir(scopeDir);
} catch {
files = [];
}
const sessions = {};
for (const file of files) {
if (
!file.endsWith(".json") ||
file === "_meta.json" ||
file === "subscriptions.json" ||
file === "auto-queue.json" ||
file === "skip-list.json" ||
file === "seen-list.json"
) continue;
// Skip non-sessions: meta + the subscription sidecar files that now
// live inside the scope dir (else they'd export as phantom sessions).
if (!file.endsWith(".json") || ROOT_SIDECARS.has(file)) continue;
try {
const raw = await fs.readFile(path.join(historyDir, file), "utf-8");
const raw = await fs.readFile(path.join(scopeDir, file), "utf-8");
const id = file.replace(".json", "");
sessions[id] = JSON.parse(raw);
} catch {}
}
// Subscriptions live at the install-wide history root and are
// operator-owned (single global store). Only the operator exports
// them — in single mode (the operator owns the box) or, in multi
// mode, the admin. A non-admin tenant's export must NOT leak the
// operator's subscription list. Future per-user subscriptions move
// this into the scope dir.
let subscriptions = [];
const ownsSubscriptions =
req.recapMode !== "multi" || !!(req.user && req.user.is_admin);
if (ownsSubscriptions) {
try {
subscriptions = JSON.parse(
await fs.readFile(path.join(historyDir, "subscriptions.json"), "utf-8")
subscriptions =
JSON.parse(
await fs.readFile(
path.join(getHistoryDir(), "subscriptions.json"),
"utf-8",
),
).subscriptions || [];
} catch {}
}
const exportData = {
version: 1,
@@ -54,7 +98,7 @@ export function setupLibraryRoutes(app) {
res.setHeader("Content-Type", "application/json");
res.setHeader(
"Content-Disposition",
'attachment; filename="recap-library.json"'
'attachment; filename="recap-library.json"',
);
res.json(exportData);
} catch (err) {
@@ -68,6 +112,8 @@ export function setupLibraryRoutes(app) {
"/api/library/import",
express.json({ limit: "200mb" }),
async (req, res) => {
const scope = requireScope(req, res);
if (!scope) return;
try {
const data = req.body;
if (!data || !data.sessions) {
@@ -76,13 +122,25 @@ export function setupLibraryRoutes(app) {
.json({ error: "Invalid library file — missing sessions data" });
}
const historyDir = getHistoryDir();
const scopeDir = getScopeHistoryDir(scope);
await fs.mkdir(scopeDir, { recursive: true }).catch(() => {});
let imported = 0;
let skipped = 0;
// Sessions — skip if already present.
for (const [id, session] of Object.entries(data.sessions)) {
const filePath = path.join(historyDir, `${id}.json`);
// The import file is fully attacker-controlled; validate the key
// before using it as a filename. A "../../" id would otherwise
// escape the scope dir and write anywhere the process can reach.
let safeId;
try {
safeId = safeFilename(id);
} catch {
skipped++;
continue;
}
const filePath = path.join(scopeDir, `${safeId}.json`);
try {
await fs.access(filePath);
skipped++;
@@ -94,20 +152,20 @@ export function setupLibraryRoutes(app) {
// Meta — merge folders + add new uncategorized at the top.
if (data.meta) {
const existingMeta = await loadMeta();
const existingMeta = await loadMeta(scope);
const allExistingIds = new Set([
...existingMeta.uncategorized,
...existingMeta.folders.flatMap(f => f.items),
...existingMeta.folders.flatMap((f) => f.items),
]);
if (data.meta.folders) {
for (const folder of data.meta.folders) {
const existingFolder = existingMeta.folders.find(
f => f.id === folder.id
(f) => f.id === folder.id,
);
if (!existingFolder) {
existingMeta.folders.push(folder);
folder.items.forEach(id => allExistingIds.add(id));
folder.items.forEach((id) => allExistingIds.add(id));
}
}
}
@@ -120,29 +178,31 @@ export function setupLibraryRoutes(app) {
}
}
await saveMeta(existingMeta);
await saveMeta(scope, existingMeta);
}
// Subscriptions — merge, dedupe by URL.
if (data.subscriptions && data.subscriptions.length > 0) {
// Subscriptions — install-wide + operator-owned. Only the operator
// (single mode, or multi-mode admin) may import them; otherwise a
// tenant's import would inject into the operator's global list.
const ownsSubscriptions =
req.recapMode !== "multi" || !!(req.user && req.user.is_admin);
if (ownsSubscriptions && data.subscriptions && data.subscriptions.length > 0) {
const subsPath = path.join(getHistoryDir(), "subscriptions.json");
let existingSubs = [];
try {
existingSubs = JSON.parse(
await fs.readFile(
path.join(historyDir, "subscriptions.json"),
"utf-8"
)
).subscriptions || [];
existingSubs =
JSON.parse(await fs.readFile(subsPath, "utf-8"))
.subscriptions || [];
} catch {}
const existingUrls = new Set(existingSubs.map(s => s.url));
const existingUrls = new Set(existingSubs.map((s) => s.url));
for (const sub of data.subscriptions) {
if (!existingUrls.has(sub.url)) {
existingSubs.push(sub);
}
}
await fs.writeFile(
path.join(historyDir, "subscriptions.json"),
JSON.stringify({ subscriptions: existingSubs })
subsPath,
JSON.stringify({ subscriptions: existingSubs }),
);
}
@@ -155,6 +215,6 @@ export function setupLibraryRoutes(app) {
} catch (err) {
res.status(500).json({ error: err.message });
}
}
},
);
}
+215 -29
View File
@@ -20,9 +20,16 @@ console.log(
// Free-tier concurrency lock. Unlicensed users may process one video at
// a time — second submission while another is in flight returns 409 from
// /api/process. The /api/process handler calls tryAcquireFreeSlot() at
// entry and releaseFreeSlot() in its finally block.
let freeJobInFlight = false;
// /api/process with details about what's running. The /api/process
// handler calls tryAcquireFreeSlot() at entry and releaseFreeSlot() in
// its finally block.
//
// The current-job object also drives:
// - /api/process/current — UI status banner after a browser refresh
// - /api/process/cancel — sets `aborted: true` AND fires the request's
// AbortController so in-flight model API calls are interrupted
// immediately (not just at the next pipeline checkpoint).
let currentFreeJob = null; // { url, title, startedAt, aborted, abortController } | null
// ── Online validation tunables ──────────────────────────────────────────────
// 30 min default scheduled cycle catches revocations / suspensions /
@@ -103,22 +110,101 @@ export function startLicenseRefresh() {
}
// ── Free-tier slot management ───────────────────────────────────────────────
// Whether the current LIC counts as a "free" (unlicensed / no core) user.
// Whether the current LIC counts as a paid user — i.e. holds either the
// `pro` or `max` entitlement. Keysat policy cards mint Pro licenses with
// `pro` and Max licenses with `max`; both unlock the same Recap-side
// gates today (subscriptions, no free-tier concurrency lock), with the
// relay layer responsible for the Pro-vs-Max quota split.
export function isPaidUser() {
if (LIC.state !== "licensed") return false;
return LIC.entitlements.has("pro") || LIC.entitlements.has("max");
}
// Inverse of isPaidUser — kept as a separate export because that's how
// most callers phrase the check ("if free, apply rate limits / show
// upgrade banner / etc.").
export function isFreeUser() {
return !(LIC.state === "licensed" && LIC.entitlements.has("core"));
return !isPaidUser();
}
// Returns true if the slot was acquired, false if another free job is in
// flight. The /api/process handler must release via releaseFreeSlot()
// in a finally block on every exit path.
export function tryAcquireFreeSlot() {
if (freeJobInFlight) return false;
freeJobInFlight = true;
//
// `abortController` is the request's AbortController — abortCurrentFreeJob
// calls .abort() on it so in-flight provider SDK calls are interrupted at
// the network layer, not just at the next pipeline checkpoint.
//
// `logs` is a server-side buffer the pipeline appends to (via
// appendCurrentJobLog) as each progress message is sent over SSE. After
// a browser refresh the client re-fetches /api/process/current and uses
// these to repopulate the activity log — without it, a refresh during
// a long pipeline silently drops everything the user has already seen.
export function tryAcquireFreeSlot({ url = "", title = "", abortController = null } = {}) {
if (currentFreeJob) return false;
currentFreeJob = {
url,
title,
startedAt: Date.now(),
aborted: false,
abortController,
logs: [],
};
return true;
}
// Push one entry onto the in-flight job's log buffer. No-op if there's
// no current job (e.g. licensed user — no free-tier tracking). Kept
// bounded so a multi-hour run doesn't grow the buffer without limit.
const MAX_LIVE_LOG_ENTRIES = 500;
export function appendCurrentJobLog(entry) {
if (!currentFreeJob || !entry) return;
currentFreeJob.logs.push(entry);
if (currentFreeJob.logs.length > MAX_LIVE_LOG_ENTRIES) {
currentFreeJob.logs.splice(0, currentFreeJob.logs.length - MAX_LIVE_LOG_ENTRIES);
}
}
export function releaseFreeSlot() {
freeJobInFlight = false;
currentFreeJob = null;
}
// Returns a JSON-friendly snapshot of the in-flight free job, or null.
// `includeLogs` is opt-in because the typical poll (banner refresh) only
// cares about the small header fields — logs are only needed when the
// client is rehydrating after a browser refresh.
export function getCurrentFreeJob({ includeLogs = false } = {}) {
if (!currentFreeJob) return null;
const out = {
url: currentFreeJob.url,
title: currentFreeJob.title,
startedAt: currentFreeJob.startedAt,
elapsedMs: Date.now() - currentFreeJob.startedAt,
aborted: currentFreeJob.aborted,
};
if (includeLogs) out.logs = [...currentFreeJob.logs];
return out;
}
// Mark the current job as cancelled AND fire its AbortController so any
// in-flight provider SDK call rejects immediately. Pipeline code also
// polls isFreeJobAborted() at major checkpoints — that handles the gaps
// between awaitable calls (e.g. while looping over yt-dlp retry delays).
// The handler's finally block runs releaseFreeSlot(), so we don't clear
// currentFreeJob here — that avoids a race where a follow-up /api/process
// request acquires the slot while the cancelled call is still cleaning up.
// Returns true if there was a job to cancel.
export function abortCurrentFreeJob() {
if (!currentFreeJob) return false;
currentFreeJob.aborted = true;
try {
currentFreeJob.abortController?.abort();
} catch {}
return true;
}
export function isFreeJobAborted() {
return !!(currentFreeJob && currentFreeJob.aborted);
}
// ── Endpoints reachable without a license ───────────────────────────────────
@@ -132,34 +218,77 @@ const LICENSE_OPEN_PATHS = new Set([
"/api/license-status",
"/api/license/activate",
"/api/license/deactivate",
"/api/process",
// Install identity — needed by the relay client before any license
// exists, and by the UI's settings panel for verification.
"/api/install-id",
// Relay balance display — the UI needs to render credit counts even
// for unlicensed (Core) users since they get free lifetime credits.
"/api/relay/status",
// Tier-policy lookup powers dynamic copy on the activation screen
// (e.g. "N relay credits" pulled live from the relay). Unlicensed
// users see the activation screen, so this must be open to them.
"/api/relay/policy",
]);
// Prefix-based open list: any /api/* path that startsWith one of these
// is reachable without a license. Library + saved summaries are part of
// the free experience (the app would feel broken without them — you'd
// summarize a video and never be able to find it again). Subscriptions,
// clips, and the relay remain paid. /api/providers/* is open so any
// user (including unlicensed) can test connectivity to their LLM
// providers before deciding whether to buy. /api/process is a prefix
// (not an exact-match in LICENSE_OPEN_PATHS) because /api/process,
// /api/process/current, and /api/process/cancel all need to be reachable
// for the free-tier flow — without /current the in-flight banner can't
// clear after the pipeline finishes, and without /cancel the Cancel
// button silently fails for unlicensed users.
const LICENSE_OPEN_PREFIXES = [
"/api/history",
"/api/library",
"/api/providers",
"/api/process",
// Audio-first ("walking mode") TTS. The /api/tts routes self-gate
// access (Max entitlement in multi mode; operator-only otherwise), so
// the blanket license middleware must let them through to that gate
// rather than 402-ing single-mode operators or Max users here.
"/api/tts",
// In-app purchase flow: GET /api/license/policies, POST
// /api/license/purchase, GET /api/license/poll/<invoiceId>. Buyers
// are unlicensed by definition — they must reach these before any
// license exists.
"/api/license/policies",
"/api/license/purchase",
"/api/license/poll",
// Relay credit top-up purchases: GET /api/credits/packages, POST
// /api/credits/buy, GET /api/credits/invoice/<id>. Buying credits
// doesn't require a license — Core (free) users should be able to
// top up just as easily as Pro/Max. The relay itself enforces
// billing via BTCPay; we just proxy.
"/api/credits",
// Self-serve subscription purchase: POST /api/billing/buy, GET
// /api/billing/status. A Core (free) user buying their way UP to
// Pro/Max is unlicensed by definition, so the activation gate must
// let them reach the buy + poll routes. The routes self-gate to a
// real signed-in user (req.user.id).
"/api/billing",
];
// ── Pro-tier feature gates ──────────────────────────────────────────────────
// Each entry maps URL prefixes → required entitlement; first match wins.
// A licensed user without the right entitlement gets a clean 402
// feature_not_in_tier (vs. the generic activation gate above).
//
// History + library used to be gated here. They moved to the free tier
// (see LICENSE_OPEN_PREFIXES above) — without saved summaries the app
// feels broken on first use, and the real paid value is auto-queue +
// relay credits.
const PRO_FEATURE_GATES = [
{
prefixes: ["/api/subscriptions", "/api/auto-queue", "/api/sub-check-log"],
entitlement: "subscriptions",
feature: "subscriptions",
message:
"Channel subscriptions and auto-queue require a Pro license. Upgrade to unlock.",
},
{
prefixes: ["/api/history"],
entitlement: "history",
feature: "history",
message:
"Summary history requires a Pro license. Upgrade to unlock.",
},
{
prefixes: ["/api/library"],
entitlement: "library",
feature: "library",
message:
"Library import/export requires a Pro license. Upgrade to unlock.",
"Channel subscriptions and auto-queue require a Pro or Max plan. Upgrade to unlock.",
},
];
@@ -174,13 +303,14 @@ export function setupLicenseMiddleware(app) {
app.use((req, res, next) => {
if (!req.path.startsWith("/api/")) return next();
if (LICENSE_OPEN_PATHS.has(req.path)) return next();
if (LIC.state === "licensed" && LIC.entitlements.has("core")) return next();
if (LICENSE_OPEN_PREFIXES.some((p) => req.path.startsWith(p))) return next();
if (isPaidUser()) return next();
return res.status(402).json({
error: "license_required",
message:
LIC.state === "licensed"
? "Your license is missing the 'core' entitlement. Contact the seller."
: "This feature requires a Recap license. Upgrade to unlock.",
? "Your license is missing the 'pro' or 'max' entitlement. Contact the seller."
: "This feature requires a Recaps license. Upgrade to unlock.",
state: LIC.state,
reason: LIC.reason,
activate_url: "/#activate",
@@ -193,6 +323,25 @@ export function setupLicenseMiddleware(app) {
app.use((req, res, next) => {
for (const gate of PRO_FEATURE_GATES) {
if (gate.prefixes.some((p) => req.path.startsWith(p))) {
// Multi mode (cloud): per-tenant — the user's relay-owned tier
// decides. Pro/Max (or the admin/operator) get in; free tenants get
// a clear 402. This is the per-tenant subscriptions gate.
if (req.recapMode === "multi") {
const tier = req.user?.tier;
if (
(req.user && req.user.is_admin) ||
tier === "pro" ||
tier === "max"
) {
return next();
}
return res.status(402).json({
error: "feature_not_in_tier",
feature: gate.feature,
message: gate.message,
});
}
// Single mode: the operator's own license carries the entitlement.
if (LIC.entitlements.has(gate.entitlement)) return next();
return res.status(402).json({
error: "feature_not_in_tier",
@@ -211,7 +360,44 @@ export function setupLicenseMiddleware(app) {
// Open by virtue of being in LICENSE_OPEN_PATHS — the gate lets them
// through unauthenticated.
export function setupLicenseRoutes(app) {
app.get("/api/license-status", (_req, res) => {
app.get("/api/license-status", (req, res) => {
// ── Multi-mode: return per-user view ────────────────────────────────
// The OPERATOR's license at /data/license.txt is "the install" — the
// pool that pays for free + trial users. Each signed-in cloud user
// has their own state:
// - paid user (users.tier = pro|max) → synthesize a license view
// from their relay-owned tier (core-decoupling: no Keysat license)
// - free tenant (signed in, Core tier) → unlicensed view (they're
// not Pro; their balance comes from tenant_credits via /relay/status)
// - anonymous/trial → unlicensed view (the badge should show trial
// credits, NOT the operator's PRO tier)
// - admin (is_admin = 1) → the operator's LIC (they ARE the
// operator; same UX as single mode)
if (req.recapMode === "multi") {
if (req.user && req.user.is_admin) {
// Operator viewing their own server: full operator license view.
return res.json(license.publicView(LIC));
}
// Paid cloud user — the tier is the relay-owned subscription tier,
// cached on the Recaps account (req.user.tier) and kept in sync by
// the operator grant flow. Synthesize a license view from it so the
// badge + per-user gates match a license-bearing user. Core-
// decoupling: this is the SOLE source of paid status in multi mode —
// a leftover per-user keysat_license is deliberately NOT consulted
// (licenses are moot in the cloud path), so the badge always agrees
// with the relay-owned tier shown in the operator's Tenants panel.
const tier = req.user?.tier;
if (req.user && (tier === "pro" || tier === "max")) {
return res.json(license.viewForTier(tier));
}
// Free tenant (tier core), trial, or fully anonymous — return an
// unlicensed view. Frontend uses this to hide the PRO badge / "manage
// license" affordances. Balance display comes from /api/relay/status
// (which is also multi-mode-aware).
return res.json(license.publicView(license.parseLicenseKey("")));
}
// ── Single-mode (the existing path) ─────────────────────────────────
// Opportunistic refresh: if the cached state is more than
// OPPORTUNISTIC_REFRESH_THRESHOLD_MS old, fire a validateOnline in
// the background. Doesn't block the response — the next status hit
+423
View File
@@ -0,0 +1,423 @@
// In-app buy flow. Proxies the Keysat public API (policies + purchase
// + poll) through Recap so the frontend renders tier cards in Recap's
// own visual style instead of being redirected to Keysat's hosted
// `/buy/<slug>` page.
//
// API shape implemented here matches the Keysat spec exactly:
// GET /api/license/policies → listPublicPolicies(PRODUCT_SLUG)
// POST /api/license/purchase → startPurchase(PRODUCT_SLUG, opts)
// GET /api/license/poll/:invoiceId → pollPurchase(invoiceId); on a
// settled invoice this also
// ACTIVATES the issued license
// on this Recap install so the
// next license-status refresh
// sees the new entitlements.
//
// All three routes are open to unlicensed users — buyers need to reach
// them before they have a license, by definition.
import { Client } from "@keysat/licensing-client";
import * as license from "./license.js";
import { randomBytes } from "crypto";
const KEYSAT_BASE_URL = license.KEYSAT_BASE_URL;
const PRODUCT_SLUG = license.PRODUCT_SLUG;
// Multi-mode toggle. In single mode (the default), an activated license
// is written to /data/license.txt — operator-install-wide. In multi
// mode, the buyer is a logged-in user and the license attaches to
// THEIR users row instead. Set at module load; the file is re-imported
// by tests so module-level state isn't an issue.
const RECAP_MODE = process.env.RECAP_MODE === "multi" ? "multi" : "single";
// Lazy-init the SDK client so a missing import doesn't crash boot.
let _client = null;
function getClient() {
if (!_client) _client = new Client(KEYSAT_BASE_URL);
return _client;
}
// Tiny in-memory cache for the policies response. Keysat's admin can
// change tiers any time so don't cache long — 30s strikes the balance
// between "operator edit takes effect quickly" and "don't hammer the
// licensing server on every browser-side modal mount". Pair this
// with using ?refresh=1 on the client if you need to invalidate
// faster than the TTL.
const POLICIES_TTL_MS = 30 * 1000;
let _policiesCache = { at: 0, body: null };
export function setupLicensePurchaseRoutes(app, { onLicenseActivated } = {}) {
// ── List tiers ────────────────────────────────────────────────────────────
// We deliberately bypass the SDK's listPublicPolicies() here because
// it strips fields we need to render the buy page: marketing_bullets,
// marketing_bullets_position, hidden_entitlements, and
// featured_discount. Hitting the public HTTP endpoint directly gives
// us the full snake_case JSON with everything the Keysat hosted /buy
// page renders, so we can match its data parity in Recap's own
// visual style.
app.get("/api/license/policies", async (_req, res) => {
if (_policiesCache.body && Date.now() - _policiesCache.at < POLICIES_TTL_MS) {
return res.json({
keysat_base_url: KEYSAT_BASE_URL,
product_slug: PRODUCT_SLUG,
..._policiesCache.body,
});
}
try {
const url = `${KEYSAT_BASE_URL.replace(/\/$/, "")}/v1/products/${encodeURIComponent(PRODUCT_SLUG)}/policies`;
const r = await fetch(url, {
signal: AbortSignal.timeout(8000),
});
if (!r.ok) {
const body = await r.text().catch(() => "");
throw new Error(`HTTP ${r.status}: ${body.slice(0, 200)}`);
}
const data = await r.json();
_policiesCache = { at: Date.now(), body: data };
res.json({
keysat_base_url: KEYSAT_BASE_URL,
product_slug: PRODUCT_SLUG,
...data,
});
} catch (err) {
console.error(`[license/policies] failed: ${err?.message || err}`);
res.status(502).json({
error: "policies_fetch_failed",
message: (err?.message || String(err)).slice(0, 300),
});
}
});
// ── Start purchase ────────────────────────────────────────────────────────
// Body: { policySlug: string, buyerEmail?: string, code?: string,
// redirectUrl?: string, buyerNote?: string }
// Returns the FULL raw Keysat /v1/purchase response, including
// discount fields: { invoice_id, checkout_url, final_price_sats,
// discount_applied_sats, amount_sats, btcpay_invoice_id, poll_url }.
// We bypass the SDK (which strips final_price_sats /
// discount_applied_sats from PurchaseSession) so the UI can show
// "you saved N sats" before sending the buyer to BTCPay.
//
// Multi-mode anon-signup flow: if there's no signed-in user AND a
// buyer_email is supplied, we treat this as a "create-account-via-
// purchase" flow. We record a pending_signups row keyed by the
// invoice id so the poll-settle handler can create the user + send
// the license-ready magic-link email once payment lands.
app.post("/api/license/purchase", async (req, res) => {
const {
policySlug,
buyerEmail,
code,
redirectUrl,
buyerNote,
} = req.body || {};
if (!policySlug || typeof policySlug !== "string") {
return res.status(400).json({
error: "missing_policy_slug",
message:
"policySlug is required — get it from /api/license/policies, never hardcode.",
});
}
// Anon-signup gate: in multi-mode, if no signed-in user, an email
// is REQUIRED — that's how the settle handler knows who to send
// the "your account is ready" link to. Signed-in users in multi
// mode have their email implicit via req.user.email (still passed
// via buyerEmail prefill from the frontend), so this only blocks
// the truly-anon-no-email edge case.
const isAnonSignup =
RECAP_MODE === "multi" && !req.user && !!buyerEmail;
if (RECAP_MODE === "multi" && !req.user && !buyerEmail) {
return res.status(400).json({
error: "email_required",
message:
"Enter your email — we'll send a sign-in link once your payment confirms.",
});
}
try {
const url = `${KEYSAT_BASE_URL.replace(/\/$/, "")}/v1/purchase`;
const r = await fetch(url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
// Keysat's /v1/purchase expects `product` (NOT `product_slug`,
// despite what its own developer-facing instructions show).
// The SDK confirms this — see vendor/keysat-licensing-client
// dist/index.js, startPurchase POST body.
product: PRODUCT_SLUG,
policy_slug: policySlug,
buyer_email: buyerEmail || undefined,
code: code || undefined,
redirect_url: redirectUrl || undefined,
buyer_note: buyerNote || undefined,
}),
signal: AbortSignal.timeout(15000),
});
const text = await r.text();
let body = null;
try { body = text ? JSON.parse(text) : null; } catch {}
if (!r.ok) {
return res.status(r.status === 400 ? 400 : 502).json({
error:
(body && (body.error || body.code)) || "purchase_failed",
message:
(body && (body.message || body.detail)) ||
text?.slice(0, 300) ||
`HTTP ${r.status}`,
});
}
// Stash pending_signups row BEFORE responding so a browser
// crash between buy + settle doesn't lose the buyer's email.
// Keysat's response shape: invoice_id at the root.
const invoiceId = body?.invoice_id || null;
if (isAnonSignup && invoiceId) {
try {
const { getDb } = await import("./db.js");
getDb()
.prepare(
`INSERT OR IGNORE INTO pending_signups
(invoice_id, email, policy_slug, created_at)
VALUES (?, ?, ?, ?)`,
)
.run(
invoiceId,
buyerEmail.trim().toLowerCase(),
policySlug,
Date.now(),
);
console.log(
`[license/purchase] tracked pending signup ${invoiceId}${buyerEmail} (${policySlug})`,
);
} catch (err) {
// Non-fatal — the BTCPay invoice still goes through. If we
// can't apply on settle, the operator can manually look up
// the Keysat invoice + activate the license + email the
// buyer.
console.error(
`[license/purchase] failed to record pending signup ${invoiceId}: ${err?.message || err}`,
);
}
}
// Pass the raw response straight through — the frontend reads
// final_price_sats + discount_applied_sats off it to render the
// discount preview before opening the checkout.
res.json(body || {});
} catch (err) {
console.error(`[license/purchase] failed: ${err?.message || err}`);
res.status(502).json({
error: "purchase_failed",
message: (err?.message || String(err)).slice(0, 300),
});
}
});
// ── Poll for license issuance ─────────────────────────────────────────────
// While the BTCPay invoice is pending, status="pending" and no
// licenseKey. Once paid + signed, status="settled" and the response
// carries the LIC1-... key. On settle, we ACTIVATE the key on this
// install (writes to /data/license.txt, refreshes module-scoped LIC
// state) so the next license-status fetch reflects the new
// entitlements. The frontend just calls /api/license-status after
// this returns settled.
app.get("/api/license/poll/:invoiceId", async (req, res) => {
const invoiceId = (req.params.invoiceId || "").trim();
if (!invoiceId) {
return res.status(400).json({ error: "missing_invoice_id" });
}
try {
const poll = await getClient().pollPurchase(invoiceId);
if (
poll.status === "settled" &&
typeof poll.licenseKey === "string" &&
poll.licenseKey.startsWith("LIC1-")
) {
// Settle path forks on mode + buyer identity:
// - single mode: write the key to /data/license.txt so the
// OPERATOR install is now licensed (existing behavior).
// - multi mode + signed-in buyer: attach the key to the
// buyer's user row (users.keysat_license). The next
// resolveProviderOpts() call for that user picks up the
// new license automatically (req.user is re-read
// per-request, so the change is immediate after the
// next signed-in request hits the server).
// - multi mode + anon buyer with pending_signups row:
// create-or-match a user by buyer_email, attach the
// license, send a magic-link email so they can sign in.
// Idempotent via pending_signups.applied_at.
// - multi mode + anon buyer with NO pending row: fall back
// to the single-mode path (write to /data/license.txt).
// This is defensive — shouldn't happen in normal flow.
try {
let routedToUser = false;
if (RECAP_MODE === "multi" && req?.user?.id) {
const { getDb } = await import("./db.js");
getDb()
.prepare("UPDATE users SET keysat_license = ? WHERE id = ?")
.run(poll.licenseKey, req.user.id);
// Mutate the in-memory req.user so subsequent calls in
// the same request see the new license. (next request
// re-reads from DB so this is belt-and-suspenders.)
req.user.keysat_license = poll.licenseKey;
routedToUser = true;
console.log(
`[license/poll] attached license to user ${req.user.id}`,
);
} else if (RECAP_MODE === "multi") {
const applied = await maybeApplyPendingSignup(
invoiceId,
poll.licenseKey,
req,
);
if (applied) routedToUser = true;
}
if (!routedToUser) {
license.activate(poll.licenseKey);
if (typeof onLicenseActivated === "function") {
await onLicenseActivated();
}
}
} catch (e) {
console.error(
`[license/poll] activate-on-settle failed: ${e?.message || e}`
);
// Don't fail the response — the buyer's checkout succeeded,
// they shouldn't see a 500. They can hit "I have a key" and
// paste the key manually as a fallback (or, in multi mode,
// operator can run an admin SQL update).
}
}
res.json(poll);
} catch (err) {
console.error(`[license/poll] failed: ${err?.message || err}`);
res.status(502).json({
error: "poll_failed",
message: (err?.message || String(err)).slice(0, 300),
});
}
});
}
// maybeApplyPendingSignup(invoiceId, licenseKey, req) — multi-mode
// anon-signup settle path. Looks up the pending_signups row for this
// invoice and, if not already applied:
// 1. Create or match a user by buyer email (lowercased)
// 2. Attach the issued license to users.keysat_license
// 3. Send a "your Recap [Pro|Max] account is ready" magic-link email
// 4. Mark applied_at = now
// Returns true if applied, false if no pending row or already applied.
//
// Idempotent: a duplicate poll after settle is a no-op because
// applied_at is the guard.
async function maybeApplyPendingSignup(invoiceId, licenseKey, req) {
const { getDb } = await import("./db.js");
const db = getDb();
const pending = db
.prepare(
`SELECT invoice_id, email, policy_slug, applied_at
FROM pending_signups WHERE invoice_id = ?`,
)
.get(invoiceId);
if (!pending) return false;
if (pending.applied_at) return true; // already applied
const email = pending.email;
const policySlug = pending.policy_slug;
// Parse the license to derive the tier label (for the email copy).
// Falls back to "Pro" if parsing fails — same default as the UI.
let tierLabel = "Pro";
try {
const { parseLicenseKey } = await import("./license.js");
const parsed = parseLicenseKey(licenseKey);
if (parsed && parsed.entitlements) {
if (parsed.entitlements.has("max")) tierLabel = "Max";
else if (parsed.entitlements.has("pro")) tierLabel = "Pro";
}
} catch {}
// Create-or-match user. If a user with this email already exists,
// attach the license to their account (case 'a' from the design
// doc: "friendliest — handles the 'I forgot I had an account' case").
// Otherwise insert a new stub user.
let user = db
.prepare("SELECT id, email, keysat_license FROM users WHERE email = ?")
.get(email);
const now = Date.now();
const tx = db.transaction(() => {
let userId;
if (user) {
userId = user.id;
db.prepare(
"UPDATE users SET keysat_license = ?, last_signin_at = ? WHERE id = ?",
).run(licenseKey, now, userId);
} else {
userId = randomUuid();
const syntheticInstallId = randomUuid();
db.prepare(
`INSERT INTO users
(id, email, created_at, last_signin_at, synthetic_install_id,
keysat_license, is_admin)
VALUES (?, ?, ?, ?, ?, ?, 0)`,
).run(userId, email, now, now, syntheticInstallId, licenseKey);
console.log(
`[license/poll] anon-signup created user ${userId} <${email}> with ${tierLabel} license`,
);
}
db.prepare(
"UPDATE pending_signups SET applied_at = ? WHERE invoice_id = ?",
).run(now, invoiceId);
return userId;
});
const userId = tx();
// Send the "your account is ready" magic-link email. Best-effort —
// if SMTP fails, the user can hit /auth.html and request a regular
// sign-in link (their account now exists with the license attached).
try {
const { sendSignInLink } = await import("./auth-routes.js");
const { renderLicenseReadyEmail } = await import(
"./license-ready-email.js"
);
// Build the email body using the verifyUrl that sendSignInLink
// will produce. We pass emailBody to override the default sign-in
// copy with the celebratory purchase-confirmation framing.
// sendSignInLink generates verifyUrl internally and the body uses
// a placeholder we string-substitute below — but actually
// sendSignInLink already accepts emailBody. We need to construct
// it with the verifyUrl already inserted. So we'll pre-render
// emailBody with verifyUrl set to a token-placeholder string,
// BUT sendSignInLink doesn't expose verifyUrl... Use a small
// bridge: render emailBody after sendSignInLink runs by passing
// a callback. Simplest: just send the link via auth-routes'
// existing helper which has the right copy via emailBody override.
//
// Actually sendSignInLink computes verifyUrl after the token is
// generated. To inject custom copy, the helper needs verifyUrl
// in scope. We pass emailBody as a FUNCTION that receives the
// verifyUrl and returns the {subject,text,html} tuple — but the
// current signature accepts an object. Refactor in v0.2.94 if
// needed. For now the helper accepts either object or function.
await sendSignInLink({
email,
intent: "license_purchase",
emailBody: (verifyUrl) =>
renderLicenseReadyEmail({
verifyUrl,
tierLabel,
brandName: "Recaps",
expiresInMinutes: 15,
}),
});
} catch (err) {
console.error(
`[license/poll] license-ready email send failed for ${email}: ${err?.message || err}`,
);
}
return true;
}
// Local UUID helper — same shape we use in auth-routes for new users.
function randomUuid() {
// Same crypto.randomBytes(16).toString("hex") pattern used elsewhere.
return randomBytes(16).toString("hex");
}
+93
View File
@@ -0,0 +1,93 @@
// "Your Recap [Pro|Max] account is ready" email — sent after an
// anon buyer's BTCPay invoice settles. Distinct from the standard
// sign-in magic-link email because the framing is celebratory ("your
// purchase is confirmed, click here to access") rather than the
// transactional "click here to sign in." Both share the underlying
// magic-link mechanism — only the copy and subject differ.
//
// Returns { subject, text, html } in the nodemailer shape.
export function renderLicenseReadyEmail({
verifyUrl,
tierLabel = "Pro",
brandName = "Recaps",
expiresInMinutes = 15,
}) {
const subject = `Your ${brandName} ${tierLabel} account is ready`;
const text = [
`Thanks for upgrading to ${brandName} ${tierLabel} — your payment is confirmed.`,
"",
`Click the link below to sign in to your new account. ${tierLabel} is already active and your license is attached.`,
"",
verifyUrl,
"",
`This link expires in ${expiresInMinutes} minutes and can only be used once. If it expires, just go back to ${brandName} and request a fresh sign-in link from the same email.`,
"",
"Welcome aboard.",
].join("\n");
const html = `<!doctype html>
<html>
<body style="margin:0;padding:0;background:#fafafa;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="background:#fafafa;padding:32px 0;">
<tr>
<td align="center">
<table role="presentation" width="480" cellpadding="0" cellspacing="0" style="background:#ffffff;border-radius:8px;padding:32px;max-width:90%;">
<tr>
<td style="font-size:18px;font-weight:600;color:#111;padding-bottom:8px;">
Your ${escapeHtml(brandName)} ${escapeHtml(tierLabel)} account is ready
</td>
</tr>
<tr>
<td style="font-size:14px;line-height:1.55;color:#444;padding-bottom:8px;">
Thanks for upgrading — your payment is confirmed and your <strong>${escapeHtml(tierLabel)}</strong> license is attached to your new account.
</td>
</tr>
<tr>
<td style="font-size:14px;line-height:1.55;color:#444;padding-bottom:24px;">
Click the button below to sign in. The link expires in ${expiresInMinutes} minutes.
</td>
</tr>
<tr>
<td align="center" style="padding-bottom:24px;">
<a href="${escapeAttr(verifyUrl)}" style="display:inline-block;background:#3b82f6;color:#fff;text-decoration:none;font-size:15px;font-weight:600;padding:12px 24px;border-radius:6px;">Sign in to ${escapeHtml(brandName)}</a>
</td>
</tr>
<tr>
<td style="font-size:13px;line-height:1.5;color:#888;padding-bottom:8px;">
Or copy and paste this URL into your browser:
</td>
</tr>
<tr>
<td style="font-size:12px;color:#888;word-break:break-all;padding-bottom:24px;">
${escapeHtml(verifyUrl)}
</td>
</tr>
<tr>
<td style="font-size:12px;line-height:1.5;color:#888;border-top:1px solid #eee;padding-top:16px;">
If the link expires before you click it, you can request a fresh sign-in link from ${escapeHtml(brandName)} using this same email. Your ${escapeHtml(tierLabel)} license will already be on the account when you sign in.
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>`;
return { subject, text, html };
}
function escapeHtml(s) {
return String(s)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#39;");
}
function escapeAttr(s) {
return escapeHtml(s);
}
+98 -10
View File
@@ -10,17 +10,26 @@
// PRODUCT_SLUG → must match the product slug created in Keysat
// KEYSAT_BASE_URL → optional, only used by online validate() / purchase
//
// Tier model for this app (see KEYSAT_INTEGRATION.md §0):
// "core" required for any business endpoint; unlocks
// summarization and BYO Gemini API key
// "history" — saved summary library: /api/history*
// "library" — bulk import/export: /api/library/*
// "subscriptions" — Pro: channel subs, auto-queue, sub-check log
// "clips" — Pro: paperclip / clip-collection panel
// Tier model for this app:
// Core / Freeno license. Library + history + lifetime relay
// credits (count set by recap-relay's Adjust Tier
// Quotas action; default 10). BYO API keys or
// self-hosted model URL gives unlimited use.
// Pro — license with entitlements:
// "pro" flags the license as paid; unlocks the
// activation gate
// "subscriptions" — channel + podcast subs, auto-queue, sub-check log
// "relay_pro" — recap-relay reads this and applies the Pro
// monthly cap (defaults: 50/mo, 25 Gemini-served)
// Max — license with entitlements:
// "max" — same gate as "pro" (server treats either as paid)
// "subscriptions" — same feature set as Pro
// "relay_max" — recap-relay applies the Max monthly cap (default
// unlimited, with 50/month Gemini sub-cap)
//
// Tier policies:
// Core → ["core", "history", "library"]
// Pro → ["core", "history", "library", "subscriptions", "clips"]
// Older entitlements ("core", "library", "history", "clips") were used
// in pre-1.0 builds. They are unused by the current server; harmless to
// ship in legacy keys.
import fs from "fs";
import path from "path";
@@ -83,6 +92,15 @@ function getOnlineClient() {
// 1. RECAP_LICENSE_KEY env var (overrides everything; useful for tests)
// 2. license.txt at LICENSE_PATH (web-UI activation writes here)
// 3. recap_license_key in startos-config.json ("Set Recap License" action)
// Read-only accessor for the raw license key (LIC1-...). Returns null
// when no license is configured. Used by the relay provider when
// attaching the Authorization header so the relay can do its cached
// online check against keysat. Don't add this to publicView — it's
// server-side only and should never reach the browser.
export function getRawLicenseKey() {
return readLicenseString();
}
function readLicenseString() {
const fromEnv = (process.env.RECAP_LICENSE_KEY || "").trim();
if (fromEnv) return fromEnv;
@@ -212,6 +230,42 @@ export function checkLicense() {
return base;
}
// parseLicenseKey(rawKey) — verify an arbitrary LIC1- string without
// touching disk or persisted state. Used in multi-tenant mode to build
// a publicView for a license stored on a user's row (users.keysat_license)
// rather than the operator's /data/license.txt. Returns the same state
// shape as checkLicense() so publicView(state) works uniformly.
//
// Differences from checkLicense:
// - Doesn't read /data/license.txt (rawKey is passed in)
// - Doesn't layer persisted online state (each user's license has
// its own online status; we'd need per-user state files to track
// that — out of scope for MVP, accept "offline-verified but
// online-unknown" as good enough)
export function parseLicenseKey(rawKey) {
if (verifierError) {
return emptyState({ state: "invalid", reason: `bad embedded key: ${verifierError}` });
}
const raw = (rawKey || "").trim();
if (!raw || !raw.startsWith("LIC1-")) return emptyState();
try {
const ok = verifier.verify(raw);
const payload = ok.payload || {};
if (payload.productSlug && payload.productSlug !== PRODUCT_SLUG) {
return emptyState({ state: "invalid", reason: "product_mismatch" });
}
return emptyState({
state: "licensed",
licenseId: payload.licenseUuid || null,
entitlements: new Set(payload.entitlements || []),
expiresAt: payload.expiresAt ? new Date(payload.expiresAt * 1000) : null,
isTrial: !!(payload.flags & 1),
});
} catch (e) {
return emptyState({ state: "invalid", reason: e?.message || "verify_failed" });
}
}
// activate(rawKey) — write a pasted key to disk, then re-check.
// Returns the new license state. Throws on bad input format only;
// signature failures surface as state: 'invalid' with a reason.
@@ -383,6 +437,40 @@ export function publicView(state) {
};
}
// viewForTier(tier, opts) — a publicView-shaped object synthesized from a
// relay-owned subscription tier ("pro" | "max"), for core-decoupling cloud
// users who have NO Keysat license. The entitlement sets mirror the Pro/Max
// license keys documented at the top of this file, so the frontend badge AND
// the per-user feature gates behave identically to a license-bearing user.
// Anything other than "pro"/"max" yields an unlicensed view (no paid
// entitlements). `expiresAt` is an optional ISO string (the relay owns the
// authoritative expiry; Recaps caches only the tier, so this is usually null).
export function viewForTier(tier, { expiresAt = null } = {}) {
const t = (tier || "").toLowerCase();
let entitlements;
if (t === "max") {
entitlements = ["max", "subscriptions", "relay_max"];
} else if (t === "pro") {
entitlements = ["pro", "subscriptions", "relay_pro"];
} else {
entitlements = [];
}
return {
state: entitlements.length ? "licensed" : "unlicensed",
reason: null,
licenseId: null,
entitlements: entitlements.sort(),
expiresAt: expiresAt || null,
isTrial: false,
productSlug: PRODUCT_SLUG,
keysatBaseUrl: KEYSAT_BASE_URL,
licensePath: LICENSE_PATH,
lastValidatedAt: null,
serverStatus: null,
graceUntil: null,
};
}
// has(state, entitlement) — convenience wrapper for feature gates.
export function has(state, entitlement) {
return state && state.entitlements && state.entitlements.has(entitlement);
+528 -6
View File
@@ -8,15 +8,20 @@
"name": "youtube-summarizer-server",
"version": "1.0.0",
"dependencies": {
"@anthropic-ai/sdk": "^0.95.0",
"@google/genai": "^1.41.0",
"@keysat/licensing-client": "file:../vendor/keysat-licensing-client",
"better-sqlite3": "^11.5.0",
"cookie": "^1.0.1",
"cors": "^2.8.5",
"express": "^4.21.0"
"express": "^4.21.0",
"nodemailer": "^6.9.16",
"openai": "^6.37.0"
}
},
"../vendor/keysat-licensing-client": {
"name": "@keysat/licensing-client",
"version": "0.1.0",
"version": "0.2.0",
"license": "MIT",
"dependencies": {
"@noble/ed25519": "^2.0.0",
@@ -27,6 +32,36 @@
"node": ">=18"
}
},
"node_modules/@anthropic-ai/sdk": {
"version": "0.95.1",
"resolved": "https://registry.npmjs.org/@anthropic-ai/sdk/-/sdk-0.95.1.tgz",
"integrity": "sha512-OO9AF7hmAoU492c/mD7Q2cPqI2WNAj7rAPHlawgBeUgpwiboLRiDs+grsErGWeHHP9ZRWfzq2OVrODTt8aITVg==",
"license": "MIT",
"dependencies": {
"json-schema-to-ts": "^3.1.1",
"standardwebhooks": "^1.0.0"
},
"bin": {
"anthropic-ai-sdk": "bin/cli"
},
"peerDependencies": {
"zod": "^3.25.0 || ^4.0.0"
},
"peerDependenciesMeta": {
"zod": {
"optional": true
}
}
},
"node_modules/@babel/runtime": {
"version": "7.29.2",
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.2.tgz",
"integrity": "sha512-JiDShH45zKHWyGe4ZNVRrCjBz8Nh9TMmZG1kh4QTK8hCBTWBi8Da+i7s1fJw7/lYpM4ccepSNfqzZ/QvABBi5g==",
"license": "MIT",
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@google/genai": {
"version": "1.52.0",
"resolved": "https://registry.npmjs.org/@google/genai/-/genai-1.52.0.tgz",
@@ -119,6 +154,12 @@
"integrity": "sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==",
"license": "BSD-3-Clause"
},
"node_modules/@stablelib/base64": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@stablelib/base64/-/base64-1.0.1.tgz",
"integrity": "sha512-1bnPQqSxSuc3Ii6MhBysoWCg58j97aUjuCSZrGSmDxNqtytIi0k8utUenAwTZN4V5mXXYGsVUI9zeBqy+jBOSQ==",
"license": "MIT"
},
"node_modules/@types/node": {
"version": "25.6.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.6.2.tgz",
@@ -182,6 +223,17 @@
],
"license": "MIT"
},
"node_modules/better-sqlite3": {
"version": "11.10.0",
"resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz",
"integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",
"hasInstallScript": true,
"license": "MIT",
"dependencies": {
"bindings": "^1.5.0",
"prebuild-install": "^7.1.1"
}
},
"node_modules/bignumber.js": {
"version": "9.3.1",
"resolved": "https://registry.npmjs.org/bignumber.js/-/bignumber.js-9.3.1.tgz",
@@ -191,6 +243,26 @@
"node": "*"
}
},
"node_modules/bindings": {
"version": "1.5.0",
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
"integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
"license": "MIT",
"dependencies": {
"file-uri-to-path": "1.0.0"
}
},
"node_modules/bl": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
"integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
"license": "MIT",
"dependencies": {
"buffer": "^5.5.0",
"inherits": "^2.0.4",
"readable-stream": "^3.4.0"
}
},
"node_modules/body-parser": {
"version": "1.20.5",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.5.tgz",
@@ -230,6 +302,30 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/buffer": {
"version": "5.7.1",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"base64-js": "^1.3.1",
"ieee754": "^1.1.13"
}
},
"node_modules/buffer-equal-constant-time": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
@@ -274,6 +370,12 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/chownr": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"license": "ISC"
},
"node_modules/content-disposition": {
"version": "0.5.4",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
@@ -296,12 +398,16 @@
}
},
"node_modules/cookie": {
"version": "0.7.2",
"resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.2.tgz",
"integrity": "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==",
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/cookie/-/cookie-1.1.1.tgz",
"integrity": "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/cookie-signature": {
@@ -345,6 +451,30 @@
"ms": "2.0.0"
}
},
"node_modules/decompress-response": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
"integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
"license": "MIT",
"dependencies": {
"mimic-response": "^3.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/deep-extend": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
"license": "MIT",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/depd": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@@ -364,6 +494,15 @@
"npm": "1.2.8000 || >= 1.4.16"
}
},
"node_modules/detect-libc": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
"license": "Apache-2.0",
"engines": {
"node": ">=8"
}
},
"node_modules/dunder-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
@@ -402,6 +541,15 @@
"node": ">= 0.8"
}
},
"node_modules/end-of-stream": {
"version": "1.4.5",
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
"integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
"license": "MIT",
"dependencies": {
"once": "^1.4.0"
}
},
"node_modules/es-define-property": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
@@ -447,6 +595,15 @@
"node": ">= 0.6"
}
},
"node_modules/expand-template": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
"integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
"license": "(MIT OR WTFPL)",
"engines": {
"node": ">=6"
}
},
"node_modules/express": {
"version": "4.22.1",
"resolved": "https://registry.npmjs.org/express/-/express-4.22.1.tgz",
@@ -493,12 +650,27 @@
"url": "https://opencollective.com/express"
}
},
"node_modules/express/node_modules/cookie": {
"version": "0.7.2",
"resolved": "https://registry.npmjs.org/cookie/-/cookie-0.7.2.tgz",
"integrity": "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/extend": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz",
"integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==",
"license": "MIT"
},
"node_modules/fast-sha256": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/fast-sha256/-/fast-sha256-1.3.0.tgz",
"integrity": "sha512-n11RGP/lrWEFI/bWdygLxhI+pVeo1ZYIVwvvPkW7azl/rOy+F3HYRZ2K5zeE9mmkhQppyv9sQFx0JM9UabnpPQ==",
"license": "Unlicense"
},
"node_modules/fetch-blob": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.2.0.tgz",
@@ -522,6 +694,12 @@
"node": "^12.20 || >= 14.13"
}
},
"node_modules/file-uri-to-path": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
"integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
"license": "MIT"
},
"node_modules/finalhandler": {
"version": "1.3.2",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",
@@ -570,6 +748,12 @@
"node": ">= 0.6"
}
},
"node_modules/fs-constants": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
"integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
"license": "MIT"
},
"node_modules/function-bind": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
@@ -644,6 +828,12 @@
"node": ">= 0.4"
}
},
"node_modules/github-from-package": {
"version": "0.0.0",
"resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
"integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
"license": "MIT"
},
"node_modules/google-auth-library": {
"version": "10.6.2",
"resolved": "https://registry.npmjs.org/google-auth-library/-/google-auth-library-10.6.2.tgz",
@@ -774,12 +964,38 @@
"node": ">=0.10.0"
}
},
"node_modules/ieee754": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
"integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "BSD-3-Clause"
},
"node_modules/inherits": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
},
"node_modules/ini": {
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"license": "ISC"
},
"node_modules/ipaddr.js": {
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@ -798,6 +1014,19 @@
"bignumber.js": "^9.0.0"
}
},
"node_modules/json-schema-to-ts": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/json-schema-to-ts/-/json-schema-to-ts-3.1.1.tgz",
"integrity": "sha512-+DWg8jCJG2TEnpy7kOm/7/AxaYoaRbjVB4LFZLySZlWn8exGs3A4OLJR966cVvU26N7X9TWxl+Jsw7dzAqKT6g==",
"license": "MIT",
"dependencies": {
"@babel/runtime": "^7.18.3",
"ts-algebra": "^2.0.0"
},
"engines": {
"node": ">=16"
}
},
"node_modules/jwa": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/jwa/-/jwa-2.0.1.tgz",
@@ -894,12 +1123,45 @@
"node": ">= 0.6"
}
},
"node_modules/mimic-response": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
"integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/mkdirp-classic": {
"version": "0.5.3",
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
"license": "MIT"
},
"node_modules/ms": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"license": "MIT"
},
"node_modules/napi-build-utils": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
"integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",
"license": "MIT"
},
"node_modules/negotiator": {
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",
@@ -909,6 +1171,18 @@
"node": ">= 0.6"
}
},
"node_modules/node-abi": {
"version": "3.92.0",
"resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.92.0.tgz",
"integrity": "sha512-KdHvFWZjEKDf0cakgFjebl371GPsISX2oZHcuyKqM7DtogIsHrqKeLTo8wBHxaXRAQlY2PsPlZmfo+9ZCxEREQ==",
"license": "MIT",
"dependencies": {
"semver": "^7.3.5"
},
"engines": {
"node": ">=10"
}
},
"node_modules/node-domexception": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz",
@@ -947,6 +1221,15 @@
"url": "https://opencollective.com/node-fetch"
}
},
"node_modules/nodemailer": {
"version": "6.10.1",
"resolved": "https://registry.npmjs.org/nodemailer/-/nodemailer-6.10.1.tgz",
"integrity": "sha512-Z+iLaBGVaSjbIzQ4pX6XV41HrooLsQ10ZWPUehGmuantvzWoDVBnmsdUcOIDM1t+yPor5pDhVlDESgOMEGxhHA==",
"license": "MIT-0",
"engines": {
"node": ">=6.0.0"
}
},
"node_modules/object-assign": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
@@ -980,6 +1263,36 @@
"node": ">= 0.8"
}
},
"node_modules/once": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
"license": "ISC",
"dependencies": {
"wrappy": "1"
}
},
"node_modules/openai": {
"version": "6.37.0",
"resolved": "https://registry.npmjs.org/openai/-/openai-6.37.0.tgz",
"integrity": "sha512-0H5dEGFmmLv6KSd0W1w2nyL8WsLkX6yoLeQpU+dZAOuGcany5qkYQMmj35ZrKgb6yiyYqpUzFOpR8mZQkgqeEQ==",
"license": "Apache-2.0",
"bin": {
"openai": "bin/cli"
},
"peerDependencies": {
"ws": "^8.18.0",
"zod": "^3.25 || ^4.0"
},
"peerDependenciesMeta": {
"ws": {
"optional": true
},
"zod": {
"optional": true
}
}
},
"node_modules/p-retry": {
"version": "4.6.2",
"resolved": "https://registry.npmjs.org/p-retry/-/p-retry-4.6.2.tgz",
@@ -1008,6 +1321,33 @@
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT"
},
"node_modules/prebuild-install": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
"integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",
"deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",
"license": "MIT",
"dependencies": {
"detect-libc": "^2.0.0",
"expand-template": "^2.0.3",
"github-from-package": "0.0.0",
"minimist": "^1.2.3",
"mkdirp-classic": "^0.5.3",
"napi-build-utils": "^2.0.0",
"node-abi": "^3.3.0",
"pump": "^3.0.0",
"rc": "^1.2.7",
"simple-get": "^4.0.0",
"tar-fs": "^2.0.0",
"tunnel-agent": "^0.6.0"
},
"bin": {
"prebuild-install": "bin.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/protobufjs": {
"version": "7.5.6",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.5.6.tgz",
@@ -1045,6 +1385,16 @@
"node": ">= 0.10"
}
},
"node_modules/pump": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
"integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
"license": "MIT",
"dependencies": {
"end-of-stream": "^1.1.0",
"once": "^1.3.1"
}
},
"node_modules/qs": {
"version": "6.14.2",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.14.2.tgz",
@@ -1084,6 +1434,35 @@
"node": ">= 0.8"
}
},
"node_modules/rc": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
"license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
"dependencies": {
"deep-extend": "^0.6.0",
"ini": "~1.3.0",
"minimist": "^1.2.0",
"strip-json-comments": "~2.0.1"
},
"bin": {
"rc": "cli.js"
}
},
"node_modules/readable-stream": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
"license": "MIT",
"dependencies": {
"inherits": "^2.0.3",
"string_decoder": "^1.1.1",
"util-deprecate": "^1.0.1"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/retry": {
"version": "0.13.1",
"resolved": "https://registry.npmjs.org/retry/-/retry-0.13.1.tgz",
@@ -1119,6 +1498,18 @@
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
"license": "MIT"
},
"node_modules/semver": {
"version": "7.8.0",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.0.tgz",
"integrity": "sha512-AcM7dV/5ul4EekoQ29Agm5vri8JNqRyj39o0qpX6vDF2GZrtutZl5RwgD1XnZjiTAfncsJhMI48QQH3sN87YNA==",
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/send": {
"version": "0.19.2",
"resolved": "https://registry.npmjs.org/send/-/send-0.19.2.tgz",
@@ -1242,6 +1633,61 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/simple-concat": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
"integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/simple-get": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
"integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"decompress-response": "^6.0.0",
"once": "^1.3.1",
"simple-concat": "^1.0.0"
}
},
"node_modules/standardwebhooks": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/standardwebhooks/-/standardwebhooks-1.0.0.tgz",
"integrity": "sha512-BbHGOQK9olHPMvQNHWul6MYlrRTAOKn03rOe4A8O3CLWhNf4YHBqq2HJKKC+sfqpxiBY52pNeesD6jIiLDz8jg==",
"license": "MIT",
"dependencies": {
"@stablelib/base64": "^1.0.0",
"fast-sha256": "^1.3.0"
}
},
"node_modules/statuses": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@@ -1251,6 +1697,52 @@
"node": ">= 0.8"
}
},
"node_modules/string_decoder": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
"license": "MIT",
"dependencies": {
"safe-buffer": "~5.2.0"
}
},
"node_modules/strip-json-comments": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/tar-fs": {
"version": "2.1.4",
"resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.4.tgz",
"integrity": "sha512-mDAjwmZdh7LTT6pNleZ05Yt65HC3E+NiQzl672vQG38jIrehtJk/J3mNwIg+vShQPcLF/LV7CMnDW6vjj6sfYQ==",
"license": "MIT",
"dependencies": {
"chownr": "^1.1.1",
"mkdirp-classic": "^0.5.2",
"pump": "^3.0.0",
"tar-stream": "^2.1.4"
}
},
"node_modules/tar-stream": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
"integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
"license": "MIT",
"dependencies": {
"bl": "^4.0.3",
"end-of-stream": "^1.4.1",
"fs-constants": "^1.0.0",
"inherits": "^2.0.3",
"readable-stream": "^3.1.1"
},
"engines": {
"node": ">=6"
}
},
"node_modules/toidentifier": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
@@ -1260,6 +1752,24 @@
"node": ">=0.6"
}
},
"node_modules/ts-algebra": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ts-algebra/-/ts-algebra-2.0.0.tgz",
"integrity": "sha512-FPAhNPFMrkwz76P7cdjdmiShwMynZYN6SgOujD1urY4oNm80Ou9oMdmbR45LotcKOXoy7wSmHkRFE6Mxbrhefw==",
"license": "MIT"
},
"node_modules/tunnel-agent": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
"integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
"license": "Apache-2.0",
"dependencies": {
"safe-buffer": "^5.0.1"
},
"engines": {
"node": "*"
}
},
"node_modules/type-is": {
"version": "1.6.18",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",
@@ -1288,6 +1798,12 @@
"node": ">= 0.8"
}
},
"node_modules/util-deprecate": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
"license": "MIT"
},
"node_modules/utils-merge": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
@@ -1315,6 +1831,12 @@
"node": ">= 8"
}
},
"node_modules/wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
},
"node_modules/ws": {
"version": "8.20.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.20.0.tgz",
+6 -1
View File
@@ -8,9 +8,14 @@
"test": "node --test --test-reporter=spec 'test/**/*.test.js'"
},
"dependencies": {
"@anthropic-ai/sdk": "^0.95.0",
"@google/genai": "^1.41.0",
"@keysat/licensing-client": "file:../vendor/keysat-licensing-client",
"better-sqlite3": "^11.5.0",
"cors": "^2.8.5",
"express": "^4.21.0"
"cookie": "^1.0.1",
"express": "^4.21.0",
"nodemailer": "^6.9.16",
"openai": "^6.37.0"
}
}
+116
View File
@@ -0,0 +1,116 @@
// Anthropic (Claude) provider — analysis only.
//
// Claude does not natively transcribe audio, so transcribeAudio() throws.
// Mix-and-match users can pair this provider for analysis with Gemini
// (or future OpenAI Whisper) for transcription.
//
// Pricing reflects standard-context rates as of 2026-04-29 (cached in
// the claude-api skill). Update when Anthropic changes published rates.
import Anthropic from "@anthropic-ai/sdk";
import { retryAPI } from "../util.js";
import { formatCost, ratesFor } from "./cost.js";
// Per-1M-token rates in USD. Anthropic does not expose a separate
// "thinking" rate — thinking tokens are billed as output, so we let
// formatCost default thinking → output by omitting the thinking field.
export const ANTHROPIC_PRICING = {
"claude-opus-4-7": { input: 5.00, output: 25.00 },
"claude-opus-4-6": { input: 5.00, output: 25.00 },
"claude-sonnet-4-6": { input: 3.00, output: 15.00 },
"claude-haiku-4-5": { input: 1.00, output: 5.00 },
// Fallback for unknown / future models.
"default": { input: 3.00, output: 15.00 },
};
// Analysis model list. Order = default fallback chain (most capable first).
export const ANTHROPIC_ANALYSIS_MODELS = [
"claude-opus-4-7",
"claude-opus-4-6",
"claude-sonnet-4-6",
"claude-haiku-4-5",
];
// Analysis output cap. Generous — the topic-analysis prompt produces a
// JSON document scaled to transcript length, and truncation here loses
// trailing sections.
const ANALYSIS_MAX_TOKENS = 16000;
export function createAnthropicProvider({ apiKey, timeoutMs = 900_000 } = {}) {
if (!apiKey) {
throw new Error("createAnthropicProvider: apiKey is required");
}
const client = new Anthropic({ apiKey, timeout: timeoutMs });
return {
name: "anthropic",
capabilities: {
transcribe: false,
analyze: true,
listModels: true,
},
listAnalysisModels() {
return [...ANTHROPIC_ANALYSIS_MODELS];
},
listTranscriptionModels() {
return [];
},
async transcribeAudio() {
throw new Error(
"Anthropic models do not natively transcribe audio. Use Gemini or OpenAI (Whisper) for the transcription step."
);
},
async analyzeText({
prompt,
model,
onProgress = () => {},
retries = 2,
signal,
}) {
const result = await retryAPI(
() =>
client.messages.create(
{
model,
max_tokens: ANALYSIS_MAX_TOKENS,
messages: [{ role: "user", content: prompt }],
},
// The Anthropic SDK accepts a per-call signal as the second
// arg; abort() rejects the in-flight HTTP request immediately.
signal ? { signal } : undefined
),
{
retries,
delayMs: 5000,
label: "Anthropic analysis",
log: (msg) => onProgress(msg),
}
);
const text = (result.content || [])
.filter((b) => b.type === "text")
.map((b) => b.text)
.join("");
const usage = {
inputTokens: result.usage?.input_tokens || 0,
outputTokens: result.usage?.output_tokens || 0,
thinkingTokens: 0,
};
const cost = formatCost(ratesFor(ANTHROPIC_PRICING, model), usage);
return {
text,
usage,
cost,
finishReason: result.stop_reason || null,
raw: result,
};
},
};
}
+66
View File
@@ -0,0 +1,66 @@
// Shared cost-calculation helper for the provider abstraction.
//
// Each provider knows two things:
// 1. Its pricing table (per-1M-token rates per model).
// 2. How to map its native usage shape into the normalized
// { inputTokens, outputTokens, thinkingTokens, totalTokens } shape.
//
// This module then turns (rates, normalized usage) → the cost record
// the rest of the app already understands. Same shape gemini-helpers
// `calcCost` produces, so dashboards / logs don't care which provider
// was used.
// Format a normalized usage object against a per-model rate table into
// the shared cost record. `rates` is { input, output, thinking? } in
// USD per 1M tokens; `usage` is { inputTokens, outputTokens,
// thinkingTokens, totalTokens } counts.
export function formatCost(rates, usage) {
const inputTokens = usage.inputTokens || 0;
const outputTokens = usage.outputTokens || 0;
const thinkingTokens = usage.thinkingTokens || 0;
const thinkingRate = rates.thinking != null ? rates.thinking : rates.output;
const inputCost = (inputTokens / 1_000_000) * rates.input;
const outputCost = (outputTokens / 1_000_000) * rates.output;
const thinkingCost = (thinkingTokens / 1_000_000) * thinkingRate;
const totalCost = inputCost + outputCost + thinkingCost;
return {
inputTokens,
outputTokens,
thinkingTokens,
totalTokens: usage.totalTokens || (inputTokens + outputTokens + thinkingTokens),
inputCost: inputCost.toFixed(6),
outputCost: outputCost.toFixed(6),
thinkingCost: thinkingCost.toFixed(6),
totalCost: totalCost.toFixed(6),
totalCostDisplay: totalCost < 0.01
? `$${(totalCost * 100).toFixed(3)}¢`
: `$${totalCost.toFixed(4)}`,
};
}
// Look up rates for a model in a provider's pricing table, falling back
// to the table's "default" row. Each provider defines its own table.
export function ratesFor(pricingTable, model) {
return pricingTable[model] || pricingTable["default"] || { input: 0, output: 0 };
}
// Zero-cost record — used by providers that don't charge (Ollama,
// local, openai-compatible without a known pricing table).
export function zeroCost(usage = {}) {
const inputTokens = usage.inputTokens || 0;
const outputTokens = usage.outputTokens || 0;
const thinkingTokens = usage.thinkingTokens || 0;
return {
inputTokens,
outputTokens,
thinkingTokens,
totalTokens: usage.totalTokens || (inputTokens + outputTokens + thinkingTokens),
inputCost: "0.000000",
outputCost: "0.000000",
thinkingCost: "0.000000",
totalCost: "0.000000",
totalCostDisplay: "$0.0000",
};
}
+407
View File
@@ -0,0 +1,407 @@
// Gemini provider — wraps @google/genai behind the shared Provider
// interface. Stateless helpers + a per-request factory: each call to
// createGeminiProvider({ apiKey }) returns a provider instance bound to
// that key, mirroring how `new GoogleGenAI({ apiKey })` was used before.
//
// What lives here:
// - SDK init + per-request HTTP timeouts
// - File API upload + processing-state polling
// - generateContent calls for transcription + analysis
// - Empty-response retry loop
// - Safety settings + thinking-config selection
// - Cost calculation (delegated to gemini-helpers.calcCost)
// - Model lists for the two pipelines (transcription vs. analysis)
//
// What does NOT live here (stays in server/index.js as orchestration):
// - Audio chunking decisions + transcript merging
// - Analysis-output JSON parsing
// - Topic-analysis prompt construction (provider-neutral, in
// gemini-helpers.js)
import { GoogleGenAI } from "@google/genai";
import { safeText, retryGemini, formatTime } from "../util.js";
import { calcCost } from "../gemini-helpers.js";
// Models exposed to the analysis fallback chain. Order matters — first
// is the preferred default, the rest are tried in order if it fails.
// The five Gemini models we expose. Verified valid against
// ai.google.dev/gemini-api/docs/models — older IDs (gemini-3-pro-preview
// shut down 2026-03-09, gemini-2.0-flash deprecated, gemini-3.1-flash*
// never existed) are intentionally not in either list.
export const GEMINI_ANALYSIS_MODELS = [
"gemini-3.1-pro-preview",
"gemini-2.5-pro",
"gemini-3-flash-preview",
"gemini-2.5-flash",
"gemini-3.1-flash-lite",
];
// Transcription fallback order: Flash first (Flash is Google's
// natural audio fit), Pro only as last-resort because Pro on audio
// is significantly more expensive than Flash.
export const GEMINI_TRANSCRIPTION_MODELS = [
"gemini-3-flash-preview",
"gemini-2.5-flash",
"gemini-3.1-flash-lite",
"gemini-2.5-pro",
"gemini-3.1-pro-preview",
];
// Empty-response retries: when the SDK returns 200 with no text (which
// happens periodically with audio inputs), retry up to N times with
// linear backoff before giving up.
const EMPTY_RETRIES = 3;
// The @google/genai SDK does not accept a per-call AbortSignal, so when
// the user cancels a request we need to interrupt the in-flight promise
// ourselves. Race the SDK call against a promise that rejects when the
// caller's signal aborts — the rejection bubbles up immediately and the
// underlying HTTP request gets garbage-collected by the SDK on its own
// timeout. `signal` is optional; without it this is a no-op passthrough.
function withAbort(promise, signal) {
if (!signal) return promise;
if (signal.aborted) {
return Promise.reject(Object.assign(new Error("aborted"), { name: "AbortError" }));
}
return new Promise((resolve, reject) => {
const onAbort = () => {
reject(Object.assign(new Error("aborted"), { name: "AbortError" }));
};
signal.addEventListener("abort", onAbort, { once: true });
promise.then(
(v) => {
signal.removeEventListener("abort", onAbort);
resolve(v);
},
(e) => {
signal.removeEventListener("abort", onAbort);
reject(e);
}
);
});
}
// Safety filters disabled for transcription so the model doesn't refuse
// to transcribe sensitive but legitimate spoken content. Analysis
// inherits whatever Gemini's defaults are.
const TRANSCRIPTION_SAFETY = [
{ category: "HARM_CATEGORY_HARASSMENT", threshold: "BLOCK_NONE" },
{ category: "HARM_CATEGORY_HATE_SPEECH", threshold: "BLOCK_NONE" },
{ category: "HARM_CATEGORY_SEXUALLY_EXPLICIT", threshold: "BLOCK_NONE" },
{ category: "HARM_CATEGORY_DANGEROUS_CONTENT", threshold: "BLOCK_NONE" },
];
export function createGeminiProvider({ apiKey, timeoutMs = 900_000 } = {}) {
if (!apiKey) {
throw new Error("createGeminiProvider: apiKey is required");
}
const ai = new GoogleGenAI({
apiKey,
httpOptions: { timeout: timeoutMs, headersTimeout: timeoutMs },
});
// Analysis uses the same client — legitimate analysis on long
// transcripts can genuinely take 35+ minutes, so an aggressive
// timeout cuts off real work. The double-retry-of-overloaded-model
// waste that 0.2.22 was trying to fix is already handled by
// retries=1 below: a 503 fast-fails in seconds, and the outer
// fallback chain (Pro → Pro older → Flash → Flash 2.5) moves
// on immediately.
const aiAnalyze = ai;
return {
name: "gemini",
capabilities: {
transcribe: true,
analyze: true,
listModels: true,
},
listAnalysisModels() {
return [...GEMINI_ANALYSIS_MODELS];
},
listTranscriptionModels() {
return [...GEMINI_TRANSCRIPTION_MODELS];
},
// Transcribe a single audio file. The caller handles chunking +
// merging — this is the atomic unit. Returns:
// { text, entries?, usage, cost, finishReason, blockReason }
// `text` is the raw model output (with [MM:SS] markers); the caller
// parses it into entries. `cost` uses the same shape calcCost
// already produces, so existing accounting code is unchanged.
async transcribeAudio({
filePath,
mimeType,
titleHint,
channelHint = "",
descriptionHint = "",
chaptersHint = [],
model,
offsetSeconds = 0,
onProgress = () => {},
signal,
}) {
const upStart = Date.now();
onProgress(
`Uploading audio${offsetSeconds > 0 ? ` (offset ${formatTime(offsetSeconds)})` : ""} to Gemini File API...`
);
const uploaded = await withAbort(
ai.files.upload({
file: filePath,
config: { mimeType },
}),
signal
);
const upTime = ((Date.now() - upStart) / 1000).toFixed(1);
onProgress(`Audio uploaded in ${upTime}s`);
// Wait for the File API to finish ingesting before generation.
let f = uploaded;
const pStart = Date.now();
while (f.state === "PROCESSING") {
if (signal?.aborted) {
throw Object.assign(new Error("aborted"), { name: "AbortError" });
}
const ws = ((Date.now() - pStart) / 1000).toFixed(0);
onProgress(`Waiting for Gemini to process audio... (${ws}s)`);
await new Promise((r) => setTimeout(r, 3000));
f = await withAbort(ai.files.get({ name: f.name }), signal);
}
if (f.state === "FAILED") {
throw new Error("Gemini failed to process audio file.");
}
const pTime = ((Date.now() - pStart) / 1000).toFixed(1);
onProgress(`Audio processed in ${pTime}s. Transcribing with ${model}...`);
const prompt = buildTranscriptionPrompt({
title: titleHint,
channel: channelHint,
description: descriptionHint,
chapters: chaptersHint,
});
// thinkingLevel is a Gemini 3.x param — Gemini 2.5 models use
// a different shape (`thinkingBudget`, integer) and 400 on
// `thinkingLevel`. Pro models reject thinking config entirely
// for the transcribe pipeline. Only send for Gemini 3.x flash
// variants where it's a valid latency/cost knob.
const isGemini3Flash =
model.includes("flash") &&
(model.startsWith("gemini-3-") || model.startsWith("gemini-3.") || model.startsWith("gemini-3."));
const txConfig = isGemini3Flash
? { thinkingConfig: { thinkingLevel: "minimal" } }
: {};
let result;
let finishReason = "UNKNOWN";
let blockReason = "none";
for (let attempt = 0; attempt < EMPTY_RETRIES; attempt++) {
if (signal?.aborted) {
throw Object.assign(new Error("aborted"), { name: "AbortError" });
}
result = await retryGemini(
() =>
withAbort(
ai.models.generateContent({
model,
config: {
...txConfig,
safetySettings: TRANSCRIPTION_SAFETY,
// Transcripts of long audio are output-token-bound.
// Gemini's default is small (commonly 8192) which is
// enough for ~10-15 min of dense speech but truncates
// 30-45 min chunks mid-transcript with no warning.
// Observed (May 2026): a 45-min chunk transcribed by
// gemini-3.1-flash-lite ended at local 31:05, losing
// 14 minutes of speech silently; another chunk lost
// 43 of 45 minutes after the model output 5 segments
// and stopped. Setting this high gives the model room
// to emit the full transcript; models that don't
// support values this large will clamp internally to
// their max. 65,536 is the upper bound for Gemini 3.x
// flash variants per Google's docs.
maxOutputTokens: 65536,
},
contents: [
{
role: "user",
parts: [
{ fileData: { fileUri: f.uri, mimeType } },
{ text: prompt },
],
},
],
}),
signal
),
{
retries: 3,
delayMs: 5000,
label: `Transcription${offsetSeconds > 0 ? ` (chunk@${formatTime(offsetSeconds)})` : ""}`,
log: (msg) => onProgress(msg),
}
);
const text = safeText(result);
if (text) break;
const candidate = result?.candidates?.[0];
finishReason = candidate?.finishReason || "UNKNOWN";
blockReason = result?.promptFeedback?.blockReason || "none";
onProgress(
`⚠ Empty response (attempt ${attempt + 1}/${EMPTY_RETRIES}) — finishReason: ${finishReason}, blockReason: ${blockReason}`
);
if (attempt < EMPTY_RETRIES - 1) {
const waitSec = 10 * (attempt + 1);
onProgress(`Waiting ${waitSec}s before retry...`);
await new Promise((r) => setTimeout(r, waitSec * 1000));
}
}
// Best-effort cleanup of the uploaded file. Failure here is
// harmless — Gemini garbage-collects on its own schedule.
try {
await ai.files.delete({ name: f.name });
} catch {}
const usage = result.usageMetadata || {};
const cost = calcCost(model, usage);
return {
text: safeText(result) || "",
usage,
cost,
finishReason,
blockReason,
// Pass-through for callers that still want the raw SDK response
// (e.g. existing logging code). Will be removed once nothing
// depends on it.
raw: result,
};
},
// Generate text from a prompt (no audio). Used by the topic-analysis
// step today, but generic enough for any text→text model call.
// Returns: { text, usage, cost, finishReason }
async analyzeText({
prompt,
model,
onProgress = () => {},
// Default to 1 attempt (no per-model retry). Analysis-step 503s
// ("model overloaded") almost never clear in 510 seconds —
// they're capacity-shaped, not transient-blip-shaped. Better
// UX: fail fast on a single model and let the outer fallback
// chain in server/index.js walk to the next model (Pro → Pro
// older → Flash → Flash 2.5) immediately. Caller can override
// Bumped 1 → 2 in 0.2.76 alongside the responseMimeType:json
// change. Analyze is by far the cheapest pipeline phase
// (~few seconds per call), so a third total attempt (1 initial
// + 2 retries on caught error) is essentially free in wall time
// but materially reduces "lost window" failures on transient
// 503/429 blips. Callers can override.
retries = 2,
signal,
}) {
const result = await retryGemini(
() =>
withAbort(
aiAnalyze.models.generateContent({
model,
config: {
// JSON mode — Gemini guarantees the response body is
// valid JSON when this is set. Eliminates the entire
// class of "invalid JSON in window response" failures
// that came from the model occasionally wrapping its
// sections array in a prose preamble, a ```json```
// markdown fence, or truncating the closing brace.
// The prompt already asks for JSON; this turns that
// into a hard server-enforced constraint on the
// model\'s decoder. Mirrors recap-relay 0.2.69\'s
// change for the relay-mode analyze path.
responseMimeType: "application/json",
},
contents: [
{
role: "user",
parts: [{ text: prompt }],
},
],
}),
signal
),
{
retries,
delayMs: 5000,
label: "Analysis",
log: (msg) => onProgress(msg),
}
);
const text = safeText(result);
const usage = result.usageMetadata || {};
const cost = calcCost(model, usage);
const finishReason = result?.candidates?.[0]?.finishReason || null;
return {
text: text || "",
usage,
cost,
finishReason,
raw: result,
};
},
};
}
// Transcription prompt — Gemini-specific because it relies on
// timestamp-formatted output we then parse. Other providers may need a
// differently-shaped prompt, so each provider owns its own.
//
// Accepts richer context than just a title: channel name, video
// description, and YouTube chapter markers. These dramatically improve
// speaker-name extraction — most podcast descriptions list host and
// guest by name, channel names are often the host's name, and chapter
// titles sometimes label introductions ("Conversation with John Doe").
// Without this context, the model falls back to "Host"/"Guest".
function buildTranscriptionPrompt({ title, channel, description, chapters } = {}) {
let context = "";
if (title) context += `Video title: "${title}"\n`;
if (channel) context += `Channel: ${channel}\n`;
if (description) {
// Trim to keep prompt size sane on hours-long podcasts whose
// descriptions can include full sponsor lists + show notes.
const desc = description.length > 1500 ? description.slice(0, 1500) + "…" : description;
context += `Video description (use to identify speakers by name):\n${desc}\n`;
}
if (Array.isArray(chapters) && chapters.length > 0) {
const lines = chapters
.slice(0, 30)
.map((c) => {
const start = typeof c.start_time === "number" ? c.start_time : 0;
const mm = Math.floor(start / 60);
const ss = Math.floor(start % 60).toString().padStart(2, "0");
return ` [${mm}:${ss}] ${c.title || ""}`;
})
.join("\n");
context += `Chapter markers (titles often name speakers or topics):\n${lines}\n`;
}
if (context) context += "\n";
return `${context}Transcribe this audio completely and verbatim. Include timestamps at regular intervals (every 15-30 seconds or at natural pauses).
Format each line as:
[MM:SS] The spoken text here...
Rules:
- Transcribe EVERY word spoken, do not skip or summarize anything.
- Use [MM:SS] or [H:MM:SS] timestamp format at the start of each line.
- Start a new timestamped line every 15-30 seconds or at natural speech pauses or speaker changes.
- Include filler words (um, uh, you know) for accuracy.
- Speaker identification: FIRST consult the metadata above — descriptions and chapter titles usually name the host(s) and guest(s) explicitly, and the channel name is often the host's name. Match those names to the voices in the audio (introductions, "I'm Dax", "this is Will", first-person references) and use them as speaker labels. Format as: [MM:SS] Name: text. Only fall back to "Host"/"Guest" if no names appear in the metadata AND nobody is introduced by name in the audio.
Return ONLY the timestamped transcript, nothing else.`;
}
+210
View File
@@ -0,0 +1,210 @@
// Provider registry. Each provider wraps a single LLM/SDK behind a
// uniform interface (see ./gemini.js for the reference shape). The rest
// of the server talks to providers through getProvider() and never
// imports SDKs directly.
//
// Adding a new provider:
// 1. Create ./<name>.js exporting createXxxProvider({ apiKey, ... }).
// 2. Add it to PROVIDER_NAMES + the switch in getProvider().
// 3. Add the matching opts shape to PROVIDER_KEY_FIELDS so
// resolveProviderOpts() can pull the right key/baseURL out of the
// StartOS config.
// 4. Wire its config field into startos/file-models/config.json.ts
// and add a "Set <Provider> Key" StartOS action.
//
// Capabilities (see provider.capabilities) signal what each one can do.
// Some providers analyze but can't transcribe (Claude, OpenAI-compat,
// Ollama); the orchestration layer in server/index.js can mix providers
// across the transcription + analysis pipelines.
import { createGeminiProvider } from "./gemini.js";
import { createAnthropicProvider } from "./anthropic.js";
import { createOpenAIProvider } from "./openai.js";
import { createOpenAICompatibleProvider } from "./openai-compatible.js";
import { createOllamaProvider } from "./ollama.js";
import { createWhisperProvider } from "./whisper.js";
import { createRelayProvider } from "./relay.js";
import { getInstallId } from "../install-id.js";
import { getRawLicenseKey } from "../license.js";
import { getRelayBaseURL, getRelayOperatorKey } from "../relay-default.js";
export const PROVIDER_NAMES = [
"gemini",
"anthropic",
"openai",
"openai-compatible",
"ollama",
"whisper",
"relay",
];
// Map provider name → which fields to read from the StartOS config blob
// when resolving its construction opts. Used by resolveProviderOpts().
export const PROVIDER_KEY_FIELDS = {
gemini: { apiKey: "gemini_api_key" },
anthropic: { apiKey: "anthropic_api_key" },
openai: { apiKey: "openai_api_key" },
"openai-compatible": {
apiKey: "openai_compatible_api_key",
baseURL: "openai_compatible_base_url",
},
ollama: { baseURL: "ollama_base_url" },
whisper: {
apiKey: "whisper_api_key",
baseURL: "whisper_base_url",
},
// Relay is operator-only — base URL is HARDCODED in
// server/relay-default.js, NOT read from StartOS config. The empty
// object is intentional: resolveProviderOpts uses `name in
// PROVIDER_KEY_FIELDS` to recognise the provider, then the
// relay-specific block at the bottom of resolveProviderOpts
// injects baseURL + installId + licenseKey server-side. Without
// this entry the lookup throws "Unknown provider: relay" before
// reaching the injection block.
relay: {},
};
export function getProvider(name, opts = {}) {
switch (name) {
case "gemini":
return createGeminiProvider(opts);
case "anthropic":
return createAnthropicProvider(opts);
case "openai":
return createOpenAIProvider(opts);
case "openai-compatible":
return createOpenAICompatibleProvider(opts);
case "ollama":
return createOllamaProvider(opts);
case "whisper":
return createWhisperProvider(opts);
case "relay":
return createRelayProvider(opts);
default:
throw new Error(
`Unknown provider: ${name}. Available: ${PROVIDER_NAMES.join(", ")}`
);
}
}
// Pull the construction opts for a provider out of the StartOS config
// blob, optionally overridden per-provider by client-side opts the web
// UI passed in the request body.
//
// `config` is the parsed startos-config.json snapshot.
// `clientOpts` is { apiKey?, baseURL? } for THIS provider only —
// typically a value out of req.body.providerOpts[name].
// `req` is the Express request — only used for the relay provider in
// multi-tenant mode, where the relay's install_id + license depend
// on WHICH user is making the call. Pass it through whenever a
// request is in scope. Single-mode + non-relay providers ignore it.
//
// Resolution priority for each field: client opt → config opt.
// Returns { apiKey?, baseURL? } as appropriate for the provider.
export function resolveProviderOpts(name, { config = {}, clientOpts = {}, req = null } = {}) {
const fields = PROVIDER_KEY_FIELDS[name];
if (!fields) {
throw new Error(`Unknown provider: ${name}`);
}
const opts = {};
if (fields.apiKey) {
const fromConfig = config[fields.apiKey] || "";
const fromClient = (clientOpts.apiKey || "").trim();
opts.apiKey = fromClient || fromConfig;
}
if (fields.baseURL) {
const fromConfig = config[fields.baseURL] || "";
const fromClient = (clientOpts.baseURL || "").trim();
opts.baseURL = fromClient || fromConfig;
// Last-resort fallback for Ollama: the canonical StartOS internal
// hostname. Reachable when the optional Ollama dependency is
// installed alongside Recap on the same StartOS server, even if
// the user hasn't run the "Set Ollama Server URL" action.
if (!opts.baseURL && name === "ollama") {
opts.baseURL = "http://ollama.startos:11434";
}
}
// User-defined model list: providers with dynamic catalogs (ollama,
// openai-compatible, whisper) accept a comma- or newline-separated
// list of model names in clientOpts.models. Parse and pass through
// as `defaultModels` so listTranscriptionModels / listAnalysisModels
// return the right thing AND so the orchestration layer's fallback
// chain knows what to walk through if the user's chosen model fails.
if (typeof clientOpts.models === "string" && clientOpts.models.trim()) {
const seen = new Set();
const models = clientOpts.models
.split(/[,\n]/)
.map((s) => s.trim())
.filter((s) => {
if (!s || seen.has(s)) return false;
seen.add(s);
return true;
});
if (models.length > 0) {
opts.defaultModels = models;
}
}
// Relay-specific injections: baseURL (hardcoded constant or env
// override) + install-id + license key. None of these come from
// clientOpts — relay identity + endpoint must not be spoofable from
// a request body.
//
// Identity rules:
// - single mode: always the operator install + operator license
// - multi mode + signed-in user WITH their own keysat_license:
// user's synthetic_install_id + user's license. The relay's
// license-keyed credit ledger (Path 3) routes consumption to
// the right user-pool.
// - multi mode + free / trial / Core user OR signed-in user with
// no license: operator's install + license. Their relay calls
// are paid out of the operator's credit pool (tenant_credits
// gates them locally to control fan-out).
if (name === "relay") {
opts.baseURL = getRelayBaseURL();
const ident = pickRelayIdentity(req);
if (ident.cloud) {
// Core-decoupling cloud identity: authenticate the server with the
// operator key + name the user; no per-user Keysat license.
opts.cloud = true;
opts.userId = ident.userId;
opts.operatorKey = ident.operatorKey;
} else {
opts.installId = ident.installId;
if (ident.licenseKey) opts.licenseKey = ident.licenseKey;
}
}
return opts;
}
// pickRelayIdentity(req) — single source of truth for "which (install_id,
// license) do we present to the relay for THIS request". Centralized so
// the rule doesn't drift across the four resolveProviderOpts callsites.
function pickRelayIdentity(req) {
// Single mode (or no request in scope, e.g. boot-time relay capability
// probe): operator identity, period.
if (!req || req.recapMode !== "multi") {
return { installId: getInstallId(), licenseKey: getRawLicenseKey() || null };
}
// Multi mode + PAID cloud user (core-decoupling): cloud identity —
// authenticate the server with the operator key and name the user by
// their Recaps account id. NO Keysat license; the relay owns the
// tier, keyed by user-id. `req.user.tier` is the Recaps-side cache of
// that relay tier (kept in sync by the operator grant flow). Falls
// back to the operator pool when the operator key isn't configured.
const tier = req.user?.tier;
if (
req.user &&
req.user.id &&
!req.user.is_admin &&
(tier === "pro" || tier === "max")
) {
const operatorKey = getRelayOperatorKey();
if (operatorKey) {
return { cloud: true, userId: req.user.id, operatorKey };
}
}
// Multi mode + everyone else (admin, anon trial, signed-in free user,
// family-share tenant on a self-hosted multi-tenant operator's box):
// pay out of the operator's pool.
return { installId: getInstallId(), licenseKey: getRawLicenseKey() || null };
}
+125
View File
@@ -0,0 +1,125 @@
// Ollama provider — analysis only, raw HTTP to a local Ollama server.
//
// Ollama runs LLMs locally; there is no per-request cost. Default
// baseURL is the conventional `http://localhost:11434`. Users on a
// LAN-hosted Ollama point at it explicitly via the StartOS action.
//
// We don't ship a hardcoded model list — Ollama's catalog is whatever
// the user has `pull`ed locally. listAnalysisModels() can optionally
// query /api/tags at config time, but for v1 we expose a free-text
// model field in the picker UI.
import { retryAPI } from "../util.js";
import { zeroCost } from "./cost.js";
const DEFAULT_BASE_URL = "http://localhost:11434";
export function createOllamaProvider({
baseURL,
timeoutMs = 900_000,
} = {}) {
const base = (baseURL || DEFAULT_BASE_URL).replace(/\/$/, "");
return {
name: "ollama",
capabilities: {
transcribe: false,
analyze: true,
listModels: true,
},
listAnalysisModels() {
return [];
},
listTranscriptionModels() {
return [];
},
async transcribeAudio() {
throw new Error(
"Ollama is wired for analysis only. Use Gemini or OpenAI Whisper for transcription."
);
},
// Lists models the local Ollama server has pulled. Best-effort —
// returns [] on any error so the picker can fall back to the
// free-text input.
async listInstalledModels() {
try {
const res = await fetch(`${base}/api/tags`, {
signal: AbortSignal.timeout(5000),
});
if (!res.ok) return [];
const data = await res.json();
return (data.models || []).map((m) => m.name).filter(Boolean);
} catch {
return [];
}
},
async analyzeText({
prompt,
model,
onProgress = () => {},
retries = 2,
signal,
}) {
const result = await retryAPI(
async () => {
// Combine the per-request timeout with the caller-supplied
// cancel signal so a user-pressed Cancel button aborts the
// fetch immediately instead of waiting for the (long) timeout.
const timeoutSignal = AbortSignal.timeout(timeoutMs);
const combinedSignal = signal
? AbortSignal.any([signal, timeoutSignal])
: timeoutSignal;
const res = await fetch(`${base}/api/generate`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model,
prompt,
stream: false,
}),
signal: combinedSignal,
});
if (!res.ok) {
const errText = await res.text().catch(() => "");
const err = new Error(
`Ollama ${res.status} ${res.statusText}: ${errText.slice(0, 200)}`
);
err.status = res.status;
throw err;
}
return res.json();
},
{
retries,
delayMs: 5000,
label: "Ollama analysis",
log: (msg) => onProgress(msg),
}
);
const text = result.response || "";
// Ollama's /api/generate returns prompt_eval_count + eval_count.
const usage = {
inputTokens: result.prompt_eval_count || 0,
outputTokens: result.eval_count || 0,
thinkingTokens: 0,
};
const cost = zeroCost(usage);
return {
text,
usage,
cost,
finishReason: result.done ? "stop" : null,
raw: result,
};
},
};
}
+110
View File
@@ -0,0 +1,110 @@
// OpenAI-compatible provider — analysis only.
//
// Same wire format as OpenAI's chat.completions endpoint, but pointed
// at a user-supplied baseURL: DeepSeek, Together, Groq, Fireworks, your
// own self-hosted vLLM, etc. The user provides baseURL + apiKey + model
// name; we don't ship a hardcoded model list (each backend's catalog
// differs), and we don't have pricing (varies wildly per backend).
//
// Structurally this is a thin re-export of the OpenAI SDK with the
// pricing table forced to zero — costs are reported as $0.0000 since we
// can't know the backend's rates without per-deploy configuration.
import OpenAI from "openai";
import { retryAPI } from "../util.js";
import { zeroCost } from "./cost.js";
// Default model lists are empty — the picker UI surfaces a free-text
// model field for OpenAI-compatible. listAnalysisModels() returns the
// caller-provided defaults if any were passed at construction time.
const ANALYSIS_MAX_TOKENS = 16000;
export function createOpenAICompatibleProvider({
apiKey,
baseURL,
defaultModels = [],
timeoutMs = 900_000,
} = {}) {
if (!baseURL) {
throw new Error(
"createOpenAICompatibleProvider: baseURL is required (e.g. https://api.deepseek.com/v1)"
);
}
// Some self-hosted backends accept any non-empty key. Default to a
// sentinel so the SDK's auth header stays well-formed.
const client = new OpenAI({
apiKey: apiKey || "no-auth",
baseURL,
timeout: timeoutMs,
});
return {
name: "openai-compatible",
capabilities: {
transcribe: false,
analyze: true,
listModels: defaultModels.length > 0,
},
listAnalysisModels() {
return [...defaultModels];
},
listTranscriptionModels() {
return [];
},
async transcribeAudio() {
throw new Error(
"openai-compatible providers are wired for analysis only. Use Gemini or OpenAI Whisper for transcription."
);
},
async analyzeText({
prompt,
model,
onProgress = () => {},
retries = 2,
signal,
}) {
const result = await retryAPI(
() =>
client.chat.completions.create(
{
model,
max_tokens: ANALYSIS_MAX_TOKENS,
messages: [{ role: "user", content: prompt }],
},
signal ? { signal } : undefined
),
{
retries,
delayMs: 5000,
label: "openai-compatible analysis",
log: (msg) => onProgress(msg),
}
);
const choice = result.choices?.[0];
const text = choice?.message?.content || "";
const usage = {
inputTokens: result.usage?.prompt_tokens || 0,
outputTokens: result.usage?.completion_tokens || 0,
thinkingTokens: 0,
};
// Per-backend pricing varies — report zero. UI can warn that cost
// tracking is not available for this provider.
const cost = zeroCost(usage);
return {
text,
usage,
cost,
finishReason: choice?.finish_reason || null,
raw: result,
};
},
};
}
+201
View File
@@ -0,0 +1,201 @@
// OpenAI provider — analysis (chat.completions) + transcription (Whisper).
//
// Whisper (whisper-1) has a 25 MB per-request file size cap. The
// orchestration layer's audio chunking is currently sized for Gemini's
// much larger cap; long podcasts at high bitrate can push individual
// chunks over Whisper's cap. We surface that as a clear error rather
// than silently truncating — users can mix providers (Whisper for
// short audio, Gemini for long) per-request via the picker.
//
// Pricing values are placeholders — verify against current OpenAI
// pricing before billing-sensitive use.
import { createReadStream, statSync } from "fs";
import OpenAI from "openai";
import { retryAPI, formatTime } from "../util.js";
import { formatCost, ratesFor } from "./cost.js";
// Per-1M-token rates in USD for chat.completions models.
// VERIFY against current OpenAI pricing before relying on these for billing.
export const OPENAI_PRICING = {
"gpt-4o": { input: 2.50, output: 10.00 },
"gpt-4o-mini": { input: 0.15, output: 0.60 },
"gpt-4-turbo": { input: 10.00, output: 30.00 },
"o3-mini": { input: 1.10, output: 4.40 },
// Fallback for unknown / future models.
"default": { input: 2.50, output: 10.00 },
};
// Whisper bills per minute of audio, not per token. The cost record
// reuses the token cost shape, but stores minute-based math in the
// `inputCost` field.
const WHISPER_USD_PER_MINUTE = 0.006;
const WHISPER_MAX_BYTES = 25 * 1024 * 1024; // OpenAI hard limit
export const OPENAI_ANALYSIS_MODELS = [
"gpt-4o",
"gpt-4o-mini",
"gpt-4-turbo",
"o3-mini",
];
export const OPENAI_TRANSCRIPTION_MODELS = ["whisper-1"];
const ANALYSIS_MAX_TOKENS = 16000;
export function createOpenAIProvider({
apiKey,
baseURL,
timeoutMs = 900_000,
} = {}) {
if (!apiKey) {
throw new Error("createOpenAIProvider: apiKey is required");
}
const client = new OpenAI({
apiKey,
baseURL: baseURL || undefined,
timeout: timeoutMs,
});
return {
name: "openai",
capabilities: {
transcribe: true,
analyze: true,
listModels: true,
},
listAnalysisModels() {
return [...OPENAI_ANALYSIS_MODELS];
},
listTranscriptionModels() {
return [...OPENAI_TRANSCRIPTION_MODELS];
},
// Whisper-based transcription. Returns the same [MM:SS] formatted
// text shape Gemini produces, so the orchestration layer's
// parseTimestampedTranscript() works unchanged.
async transcribeAudio({
filePath,
model = "whisper-1",
offsetSeconds = 0,
onProgress = () => {},
signal,
}) {
let bytes = 0;
try {
bytes = statSync(filePath).size;
} catch {}
if (bytes > WHISPER_MAX_BYTES) {
const sizeMB = (bytes / (1024 * 1024)).toFixed(1);
throw new Error(
`OpenAI Whisper file size limit is 25 MB. This chunk is ${sizeMB} MB. Try Gemini for transcription, or split the audio more aggressively.`
);
}
onProgress(
`Uploading audio${offsetSeconds > 0 ? ` (offset ${formatTime(offsetSeconds)})` : ""} to OpenAI Whisper (${model})...`
);
const start = Date.now();
const result = await retryAPI(
() =>
client.audio.transcriptions.create(
{
file: createReadStream(filePath),
model,
response_format: "verbose_json",
timestamp_granularities: ["segment"],
},
signal ? { signal } : undefined
),
{
retries: 3,
delayMs: 5000,
label: `Whisper transcription${offsetSeconds > 0 ? ` (chunk@${formatTime(offsetSeconds)})` : ""}`,
log: (msg) => onProgress(msg),
}
);
const elapsed = ((Date.now() - start) / 1000).toFixed(1);
onProgress(`Whisper transcription complete in ${elapsed}s`);
const segments = Array.isArray(result.segments) ? result.segments : [];
const lines = segments.length
? segments.map((s) => `[${formatTime(s.start || 0)}] ${(s.text || "").trim()}`)
: [`[0:00] ${(result.text || "").trim()}`];
const text = lines.join("\n");
// Whisper bills by audio duration in minutes, not tokens.
const durationSeconds = result.duration || 0;
const minutes = durationSeconds / 60;
const usdCost = minutes * WHISPER_USD_PER_MINUTE;
const cost = {
inputTokens: 0,
outputTokens: 0,
thinkingTokens: 0,
totalTokens: 0,
inputCost: usdCost.toFixed(6),
outputCost: "0.000000",
thinkingCost: "0.000000",
totalCost: usdCost.toFixed(6),
totalCostDisplay: usdCost < 0.01
? `$${(usdCost * 100).toFixed(3)}¢`
: `$${usdCost.toFixed(4)}`,
};
return {
text,
usage: { inputTokens: 0, outputTokens: 0, thinkingTokens: 0, totalTokens: 0 },
cost,
finishReason: null,
blockReason: "none",
raw: result,
};
},
async analyzeText({
prompt,
model,
onProgress = () => {},
retries = 2,
signal,
}) {
const result = await retryAPI(
() =>
client.chat.completions.create(
{
model,
max_tokens: ANALYSIS_MAX_TOKENS,
messages: [{ role: "user", content: prompt }],
},
signal ? { signal } : undefined
),
{
retries,
delayMs: 5000,
label: "OpenAI analysis",
log: (msg) => onProgress(msg),
}
);
const choice = result.choices?.[0];
const text = choice?.message?.content || "";
const usage = {
inputTokens: result.usage?.prompt_tokens || 0,
outputTokens: result.usage?.completion_tokens || 0,
thinkingTokens: 0,
};
const cost = formatCost(ratesFor(OPENAI_PRICING, model), usage);
return {
text,
usage,
cost,
finishReason: choice?.finish_reason || null,
raw: result,
};
},
};
}
File diff suppressed because it is too large Load Diff
+180
View File
@@ -0,0 +1,180 @@
// Whisper provider — transcription via any OpenAI-Audio-Transcription-API-
// compatible endpoint. OpenAI's audio.transcriptions.create wire format
// is the de facto standard; whisper.cpp's HTTP server, faster-whisper-
// server, NVIDIA Parakeet behind speaches, Groq's Whisper API, and most
// other self-hosted implementations honor it. So this provider is
// effectively "OpenAI for transcription with a custom baseURL" —
// distinct from the `openai` provider so users can wire a self-hosted
// transcription engine alongside their cloud OpenAI key (used for GPT
// analysis).
//
// Implementation note: although the wire format matches OpenAI's, this
// provider has its OWN transcribeAudio (rather than reusing the OpenAI
// provider's). Reasons:
// - Log messages should say "Whisper at host:port (model)" not
// "OpenAI Whisper" — Parakeet/whisper.cpp behind a custom URL is
// not "OpenAI" and showing that in logs is misleading.
// - No 25 MB chunk cap. Self-hosted Whisper / Parakeet typically
// handles much larger inputs than the OpenAI cloud API.
// - Zero per-minute cost reporting (self-hosted by definition).
import { createReadStream } from "fs";
import OpenAI from "openai";
import { retryAPI, formatTime } from "../util.js";
const FALLBACK_MODEL = "whisper-1";
export function createWhisperProvider({
apiKey,
baseURL,
defaultModels = [],
timeoutMs = 900_000,
} = {}) {
if (!baseURL) {
throw new Error(
"createWhisperProvider: baseURL is required (e.g. http://localhost:8000/v1)"
);
}
// Self-hosted Whisper servers commonly skip auth — pass a sentinel
// string so the SDK's authorization header is well-formed.
const client = new OpenAI({
apiKey: apiKey || "no-auth",
baseURL,
timeout: timeoutMs,
});
// Pretty-print the host for log messages: strip protocol, ignore /v1
// suffix, trim trailing slash.
const displayHost = baseURL
.replace(/^https?:\/\//, "")
.replace(/\/v\d+\/?$/, "")
.replace(/\/$/, "");
return {
name: "whisper",
capabilities: {
transcribe: true,
analyze: false,
listModels: defaultModels.length > 0,
},
listTranscriptionModels() {
return defaultModels.length > 0 ? [...defaultModels] : [FALLBACK_MODEL];
},
listAnalysisModels() {
return [];
},
async transcribeAudio({
filePath,
model = FALLBACK_MODEL,
offsetSeconds = 0,
onProgress = () => {},
signal,
}) {
// Use the model + host directly in the log — "Whisper" was
// misleading when a user wires up Parakeet (or any non-Whisper
// model) at a custom endpoint.
onProgress(
`Uploading audio${offsetSeconds > 0 ? ` (offset ${formatTime(offsetSeconds)})` : ""} to ${model} at ${displayHost}...`
);
const start = Date.now();
// Try the rich request first (verbose_json + per-segment
// timestamps — needed to render the transcript with timestamps
// and let the analysis step build sections). If the wrapper
// rejects those params (some Whisper-API-compatible servers,
// including some Parakeet wrappers, don't implement them and
// return 500), retry once with the bare-bones request shape.
let result;
let usedFallbackShape = false;
try {
result = await retryAPI(
() =>
client.audio.transcriptions.create(
{
file: createReadStream(filePath),
model,
response_format: "verbose_json",
timestamp_granularities: ["segment"],
},
signal ? { signal } : undefined
),
{
retries: 2,
delayMs: 5000,
label: `${model} transcription${offsetSeconds > 0 ? ` (chunk@${formatTime(offsetSeconds)})` : ""}`,
log: (msg) => onProgress(msg),
}
);
} catch (richErr) {
const richStatus = richErr?.status || 0;
// Only fall back on 4xx / 5xx where the params themselves are
// the likely culprit. Connection / timeout errors get thrown.
if (richStatus >= 400 && richStatus < 600) {
onProgress(
`Rich-request failed (status ${richStatus}); retrying with bare request shape (no verbose_json, no segment timestamps)...`
);
usedFallbackShape = true;
result = await retryAPI(
() =>
client.audio.transcriptions.create(
{
file: createReadStream(filePath),
model,
},
signal ? { signal } : undefined
),
{
retries: 2,
delayMs: 5000,
label: `${model} transcription (fallback)${offsetSeconds > 0 ? ` (chunk@${formatTime(offsetSeconds)})` : ""}`,
log: (msg) => onProgress(msg),
}
);
} else {
throw richErr;
}
}
const elapsed = ((Date.now() - start) / 1000).toFixed(1);
onProgress(
`${model} transcription complete in ${elapsed}s${usedFallbackShape ? " (bare request — no segment timestamps)" : ""}`
);
const segments = Array.isArray(result.segments) ? result.segments : [];
const lines = segments.length
? segments.map((s) => `[${formatTime(s.start || 0)}] ${(s.text || "").trim()}`)
: [`[0:00] ${(result.text || "").trim()}`];
const text = lines.join("\n");
// Self-hosted Whisper / Parakeet are free at the API layer
// (you've already paid for the hardware), so zero cost.
const cost = {
inputTokens: 0,
outputTokens: 0,
thinkingTokens: 0,
totalTokens: 0,
inputCost: "0.000000",
outputCost: "0.000000",
thinkingCost: "0.000000",
totalCost: "0.000000",
totalCostDisplay: "$0.0000",
};
return {
text,
usage: { inputTokens: 0, outputTokens: 0, thinkingTokens: 0, totalTokens: 0 },
cost,
finishReason: null,
blockReason: "none",
raw: result,
};
},
async analyzeText() {
throw new Error(
"Whisper provider is transcription-only. Use a different provider (Gemini / Anthropic / OpenAI / Ollama / OpenAI-compatible) for analysis."
);
},
};
}
+115
View File
@@ -0,0 +1,115 @@
// Recap-side cache of the relay's GET /relay/capabilities response.
// The relay tells us how large an audio file it can comfortably accept
// FOR THIS SPECIFIC INSTALL given the relay's current routing config
// AND this install's tier + Gemini-cap state. The same operator config
// will route a fresh install through Gemini (chunk at 60min/30MB) and
// a cap-exhausted install through hardware (no chunking) — so the
// capabilities answer is per-install, not per-operator.
//
// Recap calls this in three places:
// - on boot (warm the cache with safe defaults)
// - hourly background refresh (catches operator config edits)
// - inline before every relay-backed transcribe (so the chunking
// decision matches the routing decision the relay will actually
// make for this install RIGHT NOW)
//
// When the relay is unreachable, we fall back to Gemini-safe defaults
// so chunking happens defensively for long audio.
import { getRelayBaseURL } from "./relay-default.js";
import { getInstallId } from "./install-id.js";
import { getRawLicenseKey } from "./license.js";
const REFRESH_INTERVAL_MS = 60 * 60 * 1000; // hourly
const FETCH_TIMEOUT_MS = 5000;
// Safe defaults: chunk like we have been. Used until /relay/capabilities
// successfully populates the cache, OR if the relay is unreachable.
const DEFAULTS = Object.freeze({
max_audio_mb: 30,
max_audio_minutes: 60,
preferred_chunk_seconds: 2700,
// Audio-first ("walking mode") TTS availability. The relay advertises
// whether ANY TTS backend (Kokoro on operator hardware, or ElevenLabs)
// can serve a /relay/tts call. The frontend uses has_tts to decide
// whether to show the "Listen" affordance; the prepare route checks it
// before attempting synthesis. Conservative default: off until a fetch
// confirms it.
has_tts: false,
tts_backend: null, // "kokoro" | "elevenlabs" | null
tts_default_voice: null,
reason: "default (relay unreachable or not yet fetched)",
fetched_at: 0,
});
let cached = { ...DEFAULTS };
let refreshTimer = null;
export async function refreshRelayCapabilities() {
const base = getRelayBaseURL();
if (!base) return cached;
const url = `${base.replace(/\/$/, "")}/relay/capabilities`;
// Send install-id + license so the relay can run the per-install
// routing decision. Both are optional from the relay's perspective —
// missing install-id falls back to operator-wide capabilities, which
// is still safer than nothing.
const headers = {};
try {
const installId = getInstallId();
if (installId) headers["X-Recap-Install-Id"] = installId;
} catch {}
try {
const licenseKey = getRawLicenseKey();
if (licenseKey) headers["Authorization"] = `Bearer ${licenseKey}`;
} catch {}
try {
const r = await fetch(url, {
headers,
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
});
if (!r.ok) {
console.warn(`[relay-capabilities] ${url} returned HTTP ${r.status}`);
return cached;
}
const data = await r.json();
cached = {
max_audio_mb:
typeof data.max_audio_mb === "number" ? data.max_audio_mb : DEFAULTS.max_audio_mb,
max_audio_minutes:
typeof data.max_audio_minutes === "number"
? data.max_audio_minutes
: DEFAULTS.max_audio_minutes,
preferred_chunk_seconds:
data.preferred_chunk_seconds === null
? null
: typeof data.preferred_chunk_seconds === "number"
? data.preferred_chunk_seconds
: DEFAULTS.preferred_chunk_seconds,
has_tts: !!data.has_tts,
tts_backend: typeof data.tts_backend === "string" ? data.tts_backend : null,
tts_default_voice:
typeof data.tts_default_voice === "string" ? data.tts_default_voice : null,
reason: typeof data.reason === "string" ? data.reason : null,
fetched_at: Date.now(),
};
console.log(
`[relay-capabilities] refreshed: ${cached.max_audio_mb}MB / ${cached.max_audio_minutes}min / chunk=${cached.preferred_chunk_seconds}s (${cached.reason})`
);
} catch (err) {
console.warn(`[relay-capabilities] fetch failed: ${err?.message || err}`);
}
return cached;
}
export function startRelayCapabilitiesRefresh() {
// Fire-and-forget first refresh on boot; schedule hourly thereafter.
refreshRelayCapabilities().catch(() => {});
if (refreshTimer) clearInterval(refreshTimer);
refreshTimer = setInterval(() => {
refreshRelayCapabilities().catch(() => {});
}, REFRESH_INTERVAL_MS);
}
export function getRelayCapabilities() {
return cached;
}
+37
View File
@@ -0,0 +1,37 @@
// Hardcoded default relay URL. Recap users never see or configure this
// — it's baked into the build so Grant (operator) controls relay
// routing entirely through Recap version updates. Bump this constant
// in a new Recap release to point everyone at a different relay host.
//
// Empty string disables the relay path entirely (the "Relay (comped
// credits)" picker option will fail with "baseURL required" — caught
// by the provider-not-configured guard upstream).
//
// Override at runtime via the RECAP_RELAY_BASE_URL env var for local
// dev testing only — there's no StartOS action exposed for this, so
// production installs always use the hardcoded value.
import { relayOperatorKey } from "./config.js";
const DEFAULT_RELAY_BASE_URL = "https://relay.recaps.cc";
export function getRelayBaseURL() {
const fromEnv = (process.env.RECAP_RELAY_BASE_URL || "").trim();
return fromEnv || DEFAULT_RELAY_BASE_URL;
}
// Shared "operator key" (core-decoupling): the secret that authenticates
// THIS cloud Recaps server to the relay so it can vouch for its signed-in
// users by account-id (X-Recap-User-Id) instead of attaching a per-user
// Keysat license. Set the SAME value in the relay's relay_cloud_operator_key.
// Empty = cloud user-id identity is disabled; paid users fall back to the
// operator pool. Server-side only — never sent to the browser.
//
// Resolution: the RECAP_RELAY_OPERATOR_KEY env var pins it (local dev);
// otherwise the value comes from the StartOS config sidecar via config.js's
// polled `relayOperatorKey` live binding (set by the "Set Relay Operator
// Key" action, picked up within one poll — no restart needed).
export function getRelayOperatorKey() {
const fromEnv = (process.env.RECAP_RELAY_OPERATOR_KEY || "").trim();
if (fromEnv) return fromEnv;
return (relayOperatorKey || "").trim();
}
+117
View File
@@ -0,0 +1,117 @@
// In-memory cache of the most recent relay-reported credit balance + tier,
// keyed by credit-key so each distinct (installId, license) pair has its
// own snapshot. Multi-mode introduces multiple identities in one Recap
// process (operator's pool + each paid user's license), and a single
// global snapshot would let one user's relay ping clobber another's
// cached view.
//
// Credit-key shape mirrors the relay's own keying (Path 3 license-keyed
// credits): `lic:<sha256(licenseKey).slice(0, 16)>` for paid licenses,
// `inst:<installId>` for unlicensed / Core. Stable per identity, doesn't
// require parsing the license payload — we hash the raw LIC1 string,
// which works as a local cache key independent of the relay's exact
// fingerprint formula.
//
// Not persisted to disk — relay is the source of truth. On a fresh boot
// the map is empty until each request triggers its first probe.
import { createHash } from "crypto";
// Map<creditKey, snapshot>. snapshot shape:
// { creditsRemaining, tier, lastUpdated, lastError }
const snapshots = new Map();
// LRU-ish defensive cap so a worst-case (e.g. a buggy probe spinning up
// thousands of distinct identities) can't grow the map unbounded. In
// normal use we expect on the order of (1 operator + N paid users)
// rows, well under this ceiling.
const MAX_SNAPSHOTS = 5000;
const EMPTY_SNAPSHOT = Object.freeze({
creditsRemaining: null,
tier: null,
lastUpdated: null,
lastError: null,
});
// computeCreditKey({ installId, licenseKey, userId }) — derives the stable
// per-identity key used by both the provider (when recording responses)
// and the /api/relay/status handler (when looking up the right snapshot
// for the request). Priority: cloud userId (core-decoupling `user:<id>`)
// → license (hashed raw key) → install_id. Returns null if all three are
// missing (caller should treat as "no cache, never hit" and skip the
// lookup).
export function computeCreditKey({ installId, licenseKey, userId } = {}) {
// Cloud (core-decoupling) identity: keyed by the Recaps account id, to
// mirror the relay's `user:<id>` pool.
const uid = (userId || "").trim();
if (uid) return "user:" + uid;
const lic = (licenseKey || "").trim();
if (lic) {
return "lic:" + createHash("sha256").update(lic).digest("hex").slice(0, 16);
}
const id = (installId || "").trim();
if (id) return "inst:" + id;
return null;
}
function getOrCreate(creditKey) {
let s = snapshots.get(creditKey);
if (!s) {
if (snapshots.size >= MAX_SNAPSHOTS) {
// Evict the oldest entry (Map iteration is insertion-ordered).
const oldest = snapshots.keys().next().value;
if (oldest != null) snapshots.delete(oldest);
}
s = { creditsRemaining: null, tier: null, lastUpdated: null, lastError: null };
snapshots.set(creditKey, s);
}
return s;
}
// Called by the relay provider on every response (including 4xx with the
// standard envelope). `envelope` is the parsed JSON shape:
// { credits_remaining, tier, ... }
// `creditKey` is the per-identity key from computeCreditKey(); required
// in multi-mode, optional in single-mode (falls through to a synthetic
// "operator" bucket — see fallback below).
export function updateRelayState(envelope, creditKey) {
if (!envelope || typeof envelope !== "object") return;
const key = creditKey || "operator";
const s = getOrCreate(key);
if (typeof envelope.credits_remaining === "number") {
s.creditsRemaining = envelope.credits_remaining;
}
if (typeof envelope.tier === "string") {
s.tier = envelope.tier;
}
s.lastUpdated = Date.now();
s.lastError = null;
}
// Record a relay error (network failure, 5xx without envelope, etc.).
// Same keying as updateRelayState — the error sticks to the identity
// that hit it, so an admin's tile doesn't flash "relay unreachable"
// because a tenant's probe failed.
export function recordRelayError(message, creditKey) {
const key = creditKey || "operator";
const s = getOrCreate(key);
s.lastError = (message || "Unknown relay error").slice(0, 300);
s.lastUpdated = Date.now();
}
// getRelayState(creditKey) — fetch the snapshot for an identity. Returns
// the EMPTY_SNAPSHOT (with nulls) if the identity has never recorded
// anything yet. Caller can branch on creditsRemaining === null vs
// numeric to decide whether to probe.
export function getRelayState(creditKey) {
const key = creditKey || "operator";
const s = snapshots.get(key);
return s ? { ...s } : { ...EMPTY_SNAPSHOT };
}
// Test/teardown helper. Clears every cached snapshot. Used by tests + a
// debugging admin endpoint if we ever need one. Not used in production.
export function resetRelayState() {
snapshots.clear();
}
+116
View File
@@ -0,0 +1,116 @@
// SMTP transport for outbound mail (currently: magic-link sign-in
// links). Credentials come from /data/config/startos-config.json,
// which startos/main.ts keeps in sync with the StartOS System SMTP
// config via effects.getSystemSmtp({callback}).
//
// We poll the same JSON the rest of the server reads (via
// getConfigSnapshot()) and rebuild the nodemailer transport whenever
// the credentials change. Rebuilds are cheap — nodemailer's
// createTransport is synchronous and just stashes options. The actual
// SMTP connection is opened per-send (or pooled), not at transport
// creation.
//
// Only used in multi-tenant mode. Single mode never imports this.
import nodemailer from "nodemailer";
import { getConfigSnapshot } from "./config.js";
let cachedTransport = null;
let cachedFingerprint = ""; // last seen credentials as a stable string
let cachedFrom = "";
const POLL_MS = 3000;
// initSmtp(): kicks off the credentials-watch loop. Idempotent in
// effect (later calls are harmless), but you only need it once per
// process lifetime.
export function initSmtp() {
refreshTransport().catch((err) => {
console.warn("[smtp] initial refresh failed:", err);
});
setInterval(() => {
refreshTransport().catch(() => {});
}, POLL_MS);
}
async function refreshTransport() {
const snap = await getConfigSnapshot();
const host = snap.smtp_host || "";
const port = parseInt(snap.smtp_port || 0, 10) || 0;
const security = snap.smtp_security || "tls";
const username = snap.smtp_username || "";
const password = snap.smtp_password || "";
const from = snap.smtp_from || "";
// Fingerprint covers every field that affects the transport. If
// nothing changed, leave the existing transport in place — avoids
// tearing down a pooled connection on every poll tick.
const fingerprint = [host, port, security, username, password, from].join(
"\x1f",
);
if (fingerprint === cachedFingerprint) return;
cachedFingerprint = fingerprint;
cachedFrom = from;
if (!host || !port) {
if (cachedTransport) {
cachedTransport.close?.();
cachedTransport = null;
console.log("[smtp] credentials cleared, transport closed");
}
return;
}
// security mapping:
// - "tls" → implicit TLS (SMTPS, typically port 465). secure=true.
// - "starttls" → cleartext then STARTTLS (typically port 587). secure=false,
// requireTLS=true so we refuse to silently fall back to plain.
const isImplicitTls = security === "tls";
const transportOpts = {
host,
port,
secure: isImplicitTls,
requireTLS: !isImplicitTls, // forces STARTTLS when secure=false
auth: username ? { user: username, pass: password } : undefined,
};
try {
const next = nodemailer.createTransport(transportOpts);
if (cachedTransport) cachedTransport.close?.();
cachedTransport = next;
console.log(
`[smtp] transport built host=${host} port=${port} security=${security} user=${username || "(none)"}`,
);
} catch (err) {
console.warn("[smtp] createTransport failed:", err);
cachedTransport = null;
}
}
// sendMail({to, subject, text, html})
// Returns nodemailer's info object on success. Throws if the
// transport isn't configured — callers should treat that as "SMTP
// not set up yet, ask the operator to configure StartOS System SMTP."
export async function sendMail({ to, subject, text, html }) {
if (!cachedTransport) {
throw new Error("smtp_not_configured");
}
if (!cachedFrom) {
throw new Error("smtp_from_not_set");
}
return await cachedTransport.sendMail({
from: cachedFrom,
to,
subject,
text,
html,
});
}
// True iff the transport is built and ready to send. The auth handlers
// use this to gate magic-link requests with a useful error rather than
// letting nodemailer's "no transport" leak through.
export function isSmtpReady() {
return cachedTransport !== null && cachedFrom !== "";
}
+173
View File
@@ -0,0 +1,173 @@
// Self-serve subscription expiry reminders (multi-mode / cloud only).
//
// The relay owns each user's prepaid-period expiry (subscription source of
// truth); Recaps owns the email address + the recaps.cc SMTP transport. So
// a daily scan here asks the relay who's expiring (or just lapsed), maps
// user_id → users.email, and sends a reminder via the same sendMail() the
// magic-link flow uses. The subscription_reminders table dedups so each
// (user, period, kind) email goes out at most once; a renewal changes the
// expiry instant, which re-arms a fresh set of reminders for the new period.
//
// Self-gating: no-ops unless SMTP is configured, the public URL is set, and
// the relay is reachable. Safe to start unconditionally in multi mode.
import { getDb } from "./db.js";
import { sendMail, isSmtpReady } from "./smtp.js";
import { renderSubscriptionReminderEmail } from "./email-template.js";
import { getRelayExpiringSubscriptions } from "./providers/relay.js";
import { getConfigSnapshot } from "./config.js";
// Reminder thresholds, in days before expiry. Each maps to a distinct
// `kind` so a user gets one heads-up at each crossing (deduped by kind).
const UPCOMING_THRESHOLDS = [
{ days: 7, kind: "upcoming_7d" },
{ days: 1, kind: "upcoming_1d" },
];
const LAPSED_KIND = "lapsed";
// Only email recently-lapsed users; after this they age out of the relay
// window (and we never re-send the same period's lapsed notice anyway).
const LAPSED_WINDOW_DAYS = 3;
const SCAN_INTERVAL_MS = 12 * 60 * 60 * 1000; // twice a day
const BOOT_DELAY_MS = 90 * 1000; // first scan ~90s after boot
let scanning = false;
let scheduled = false;
// Decide which single reminder kind (if any) applies to a subscription
// right now. Pure — exported for testing. `sub` is a relay row:
// { tier, expires_at, expired, days_left }.
export function reminderKindFor(
sub,
{ upcoming = UPCOMING_THRESHOLDS, lapsedWindowDays = LAPSED_WINDOW_DAYS } = {},
) {
if (!sub || typeof sub.days_left !== "number") return null;
const daysLeft = sub.days_left;
if (sub.expired) {
// Recently lapsed → a single lapsed notice (days_left is <= 0).
return daysLeft >= -lapsedWindowDays ? LAPSED_KIND : null;
}
// Upcoming: smallest threshold the sub has crossed wins, so daysLeft=1
// sends 'upcoming_1d' while daysLeft in (1, 7] sends 'upcoming_7d'.
const sorted = [...upcoming].sort((a, b) => a.days - b.days);
for (const t of sorted) {
if (daysLeft <= t.days) return t.kind;
}
return null;
}
function alreadySent(db, userId, periodIso, kind) {
return !!db
.prepare(
"SELECT 1 FROM subscription_reminders WHERE user_id=? AND period_expires_at=? AND kind=?",
)
.get(userId, periodIso, kind);
}
function recordSent(db, userId, periodIso, kind) {
db.prepare(
"INSERT OR IGNORE INTO subscription_reminders (user_id, period_expires_at, kind, sent_at) VALUES (?,?,?,?)",
).run(userId, periodIso, kind, Date.now());
}
function maskEmail(email) {
return String(email).replace(/^(.).*(@.*)$/, "$1***$2");
}
// One scan pass. Returns a small summary object; never throws (logs +
// returns a {skipped} reason instead) so the scheduler stays alive.
export async function runReminderScan({ force = false } = {}) {
if (scanning && !force) return { skipped: "already_running" };
scanning = true;
try {
if (!isSmtpReady()) return { skipped: "smtp_not_ready" };
const snap = await getConfigSnapshot();
const publicUrl = (snap.recap_public_url || "").trim().replace(/\/$/, "");
if (!publicUrl) return { skipped: "public_url_not_set" };
const maxDays = Math.max(1, ...UPCOMING_THRESHOLDS.map((t) => t.days));
const report = await getRelayExpiringSubscriptions({
withinDays: maxDays,
lapsedDays: LAPSED_WINDOW_DAYS,
});
if (!report || !Array.isArray(report.subscriptions)) {
return { skipped: "relay_unavailable" };
}
const db = getDb();
const manageUrl = `${publicUrl}/?renew=1`;
let sent = 0;
let skipped = 0;
for (const sub of report.subscriptions) {
const kind = reminderKindFor(sub);
if (!kind) {
skipped++;
continue;
}
const periodIso = sub.expires_at;
if (!periodIso || alreadySent(db, sub.user_id, periodIso, kind)) {
skipped++;
continue;
}
const u = db
.prepare("SELECT email FROM users WHERE id=?")
.get(sub.user_id);
const email = (u?.email || "").trim();
if (!email) {
// No local user for this id (e.g. another instance's tenant) — skip.
skipped++;
continue;
}
const message = renderSubscriptionReminderEmail({
brandName: "Recaps",
tier: sub.tier,
expiresAt: sub.expires_at,
daysLeft: sub.days_left,
kind,
manageUrl,
});
try {
await sendMail({
to: email,
subject: message.subject,
text: message.text,
html: message.html,
});
recordSent(db, sub.user_id, periodIso, kind);
sent++;
console.log(
`[reminders] sent ${kind} to ${maskEmail(email)} (${sub.tier}, ${sub.days_left}d)`,
);
} catch (err) {
// Don't record on failure → retried next scan.
console.warn(
`[reminders] sendMail failed for ${sub.user_id}: ${err?.message || err}`,
);
skipped++;
}
}
if (sent) {
console.log(`[reminders] scan complete: ${sent} sent, ${skipped} skipped`);
}
return { sent, skipped };
} catch (err) {
console.warn(`[reminders] scan error: ${err?.message || err}`);
return { skipped: "error", error: err?.message || String(err) };
} finally {
scanning = false;
}
}
// Start the daily-ish scan loop. Idempotent. Self-gates inside the scan,
// so it's safe to call whenever multi mode boots.
export function startReminderScheduler() {
if (scheduled) return;
scheduled = true;
setTimeout(() => {
runReminderScan().catch(() => {});
}, BOOT_DELAY_MS);
setInterval(() => {
runReminderScan().catch(() => {});
}, SCAN_INTERVAL_MS);
console.log("[reminders] expiry-reminder scheduler started");
}
+285
View File
@@ -0,0 +1,285 @@
// Subscription storage + discovery helpers, keyed by scope.
//
// Today the subscription feature is operator-only in multi mode (gated in
// license-middleware), so the only active scope is "owner". But the storage,
// dedup, check-enumeration, and migration here are all scope-parameterized,
// so flipping the gate to per-tenant (see docs/per-tenant-subscriptions-plan.md)
// is a matter of passing each user's scope instead of "owner" — no storage
// rework. Step 4 of that plan (the background processor acting as each
// owning user) is the only remaining piece and needs on-device testing.
//
// Each scope's state lives under scopeDir(scope) = history/<scope>/:
// subscriptions.json — the user's channel/podcast subscriptions
// auto-queue.json — discovered videos awaiting approval / processing
// skip-list.json — videoIds the user declined (never re-offer)
// seen-list.json — videoIds already offered (don't re-surface)
import fs from "fs/promises";
import path from "path";
import { getScopeHistoryDir, getHistoryDir, ROOT_SIDECARS } from "./history.js";
// ── Per-scope file paths ─────────────────────────────────────────────────
function subsPath(scope) {
return path.join(getScopeHistoryDir(scope), "subscriptions.json");
}
function skipPath(scope) {
return path.join(getScopeHistoryDir(scope), "skip-list.json");
}
function seenPath(scope) {
return path.join(getScopeHistoryDir(scope), "seen-list.json");
}
function autoQueuePath(scope) {
return path.join(getScopeHistoryDir(scope), "auto-queue.json");
}
async function ensureScopeDir(scope) {
await fs.mkdir(getScopeHistoryDir(scope), { recursive: true }).catch(() => {});
}
// Serialize read-modify-write on a given file path so two concurrent
// handlers can't each load the same snapshot, mutate, and have the second
// write clobber the first. Keyed by absolute path → naturally per-scope.
const _fileLocks = new Map();
function withFileLock(key, fn) {
const prev = _fileLocks.get(key) || Promise.resolve();
const next = prev.then(fn, fn); // run fn whether prev resolved or rejected
_fileLocks.set(
key,
next.catch(() => {}),
);
return next;
}
// ── Subscriptions ────────────────────────────────────────────────────────
export async function loadSubscriptions(scope) {
try {
return (
JSON.parse(await fs.readFile(subsPath(scope), "utf-8")).subscriptions || []
);
} catch {
return [];
}
}
export async function saveSubscriptions(scope, subs) {
return withFileLock(subsPath(scope), async () => {
await ensureScopeDir(scope);
await fs.writeFile(
subsPath(scope),
JSON.stringify({ subscriptions: subs }, null, 2),
);
});
}
// ── Skip list (declined videos — never re-add) ───────────────────────────
export async function loadSkipList(scope) {
try {
return new Set(
JSON.parse(await fs.readFile(skipPath(scope), "utf-8")).videoIds || [],
);
} catch {
return new Set();
}
}
export async function addToSkipList(scope, videoId) {
return withFileLock(skipPath(scope), async () => {
const ids = await loadSkipList(scope);
ids.add(videoId);
await ensureScopeDir(scope);
await fs.writeFile(skipPath(scope), JSON.stringify({ videoIds: [...ids] }));
});
}
// ── Seen list (already offered — don't re-surface) ───────────────────────
export async function loadSeenList(scope) {
try {
return new Set(
JSON.parse(await fs.readFile(seenPath(scope), "utf-8")).videoIds || [],
);
} catch {
return new Set();
}
}
export async function addToSeenList(scope, videoIds) {
return withFileLock(seenPath(scope), async () => {
const seen = await loadSeenList(scope);
for (const id of videoIds) seen.add(id);
await ensureScopeDir(scope);
await fs.writeFile(seenPath(scope), JSON.stringify({ videoIds: [...seen] }));
});
}
// ── Auto-queue ───────────────────────────────────────────────────────────
// Read-only load. For mutations use mutateAutoQueue so the read-modify-write
// is atomic per scope (the old in-memory global array gave this implicitly).
export async function loadAutoQueue(scope) {
try {
return JSON.parse(await fs.readFile(autoQueuePath(scope), "utf-8")).items || [];
} catch {
return [];
}
}
export async function saveAutoQueue(scope, items) {
return withFileLock(autoQueuePath(scope), async () => {
await ensureScopeDir(scope);
await fs.writeFile(
autoQueuePath(scope),
JSON.stringify({ items }, null, 2),
);
});
}
// Atomic read-modify-write. `fn(items)` may mutate `items` in place and/or
// return a replacement array. Returns the saved array. Use for every
// status change / add / remove so concurrent handlers don't lose updates.
export async function mutateAutoQueue(scope, fn) {
return withFileLock(autoQueuePath(scope), async () => {
let items = [];
try {
items =
JSON.parse(await fs.readFile(autoQueuePath(scope), "utf-8")).items || [];
} catch {}
const result = await fn(items);
const toSave = Array.isArray(result) ? result : items;
await ensureScopeDir(scope);
await fs.writeFile(
autoQueuePath(scope),
JSON.stringify({ items: toSave }, null, 2),
);
return toSave;
});
}
// ── Dedup ────────────────────────────────────────────────────────────────
// All videoIds already summarized in a scope's library.
//
// CRITICAL: summaries live under scopeDir(scope) = history/<scope>/, NOT the
// top-level history dir. Scanning the top level (the historical bug) found
// zero processed videos, so the subscription check never deduped against the
// library and re-queued already-summarized videos every run.
export async function getProcessedVideoIds(scope = "owner") {
const ids = new Set();
const dir = getScopeHistoryDir(scope);
try {
const files = await fs.readdir(dir);
for (const file of files.filter(
(f) => f.endsWith(".json") && !ROOT_SIDECARS.has(f),
)) {
try {
const raw = await fs.readFile(path.join(dir, file), "utf-8");
const data = JSON.parse(raw);
if (data.videoId) ids.add(data.videoId);
} catch {}
}
} catch {}
return ids;
}
// Pure dedup predicate: is this discovered video already accounted for?
// Known = in the library (processed), already queued, declined (skip), or
// offered before (seen). Used identically by the podcast + YouTube branches.
export function isKnownVideo(
id,
{ processedIds, queuedIds, skippedIds, seenIds } = {},
) {
return !!(
(processedIds && processedIds.has(id)) ||
(queuedIds && queuedIds.has(id)) ||
(skippedIds && skippedIds.has(id)) ||
(seenIds && seenIds.has(id))
);
}
// ── Scope enumeration (for the periodic check loop) ──────────────────────
// Scopes that have at least one subscription. Always includes "owner" (the
// operator). Behind the operator-only gate this returns just ["owner"]; when
// per-tenant subscriptions ship it picks up each tenant scope automatically.
export async function listSubscriptionScopes() {
const root = getHistoryDir();
const scopes = new Set(["owner"]);
try {
const entries = await fs.readdir(root, { withFileTypes: true });
for (const e of entries) {
if (!e.isDirectory() || e.name === "owner") continue;
try {
const subs =
JSON.parse(
await fs.readFile(
path.join(root, e.name, "subscriptions.json"),
"utf-8",
),
).subscriptions || [];
if (subs.length > 0) scopes.add(e.name);
} catch {}
}
} catch {}
return [...scopes];
}
// Scopes that have a non-empty auto-queue. The background processor walks
// these to find approved items across all owners (a scope can have queued
// items even after its subscriptions were deleted, so this is a superset of
// listSubscriptionScopes for processing purposes).
export async function listAutoQueueScopes() {
const root = getHistoryDir();
const scopes = new Set(["owner"]);
try {
const entries = await fs.readdir(root, { withFileTypes: true });
for (const e of entries) {
if (!e.isDirectory() || e.name === "owner") continue;
try {
const items =
JSON.parse(
await fs.readFile(
path.join(root, e.name, "auto-queue.json"),
"utf-8",
),
).items || [];
if (items.length > 0) scopes.add(e.name);
} catch {}
}
} catch {}
return [...scopes];
}
// ── Migration: history-root globals → owner scope (one-time, idempotent) ──
// The pre-0.2.147 layout kept subscription state at the history root (one
// install-wide store). Move it under the operator's own scope so the
// storage is uniformly per-scope. Only moves a file if the source exists
// and the destination doesn't (never clobbers).
const SUB_FILES = [
"subscriptions.json",
"auto-queue.json",
"skip-list.json",
"seen-list.json",
];
export async function migrateGlobalSubscriptionsToOwner() {
const root = getHistoryDir();
const ownerDir = getScopeHistoryDir("owner");
let moved = 0;
for (const f of SUB_FILES) {
const from = path.join(root, f);
const to = path.join(ownerDir, f);
try {
await fs.access(from);
} catch {
continue; // source missing → nothing to move
}
try {
await fs.access(to);
continue; // dest already exists → don't clobber
} catch {}
try {
await fs.mkdir(ownerDir, { recursive: true });
await fs.rename(from, to);
moved++;
} catch {
// rename can fail across devices — fall back to copy + unlink.
try {
await fs.mkdir(ownerDir, { recursive: true });
await fs.copyFile(from, to);
await fs.unlink(from);
moved++;
} catch {}
}
}
return moved;
}
+293
View File
@@ -0,0 +1,293 @@
// Tenant (multi-user) auth middleware.
//
// Two modes:
// - single (the original self-hosted experience): no-op. Sets
// req.userId = "owner" so downstream code can scope everything
// uniformly to a single synthetic user.
// - multi (cloud / family-share): validates the recap_session cookie
// against the sessions table, looks up the user, attaches both to
// req. Non-public paths without a valid session get a 401.
//
// This is layered AFTER the admin-auth middleware (which gates /api/*
// behind the operator's password in single mode). The two are
// orthogonal:
// - admin-auth: "is the OPERATOR allowed in?" (single mode only)
// - tenant-auth: "which user is this request from?" (multi mode only)
// In multi mode the admin-auth gate is disabled — tenant accounts
// authenticate themselves directly.
import { getDb } from "./db.js";
import {
TRIAL_COOKIE,
lookupTrial,
hasTrialBudget,
} from "./anon-trial.js";
const SESSION_COOKIE = "recap_session";
// Endpoints that must remain reachable WITHOUT a session — the auth
// flow itself, health check, static assets, the BTCPay webhook. The
// frontend's static files are served outside the /api/* tree so they
// bypass this middleware entirely; this list only matters for /api/*
// and /auth/* paths.
const PUBLIC_PATH_PREFIXES = [
"/auth/", // /auth/request-link, /auth/verify, /auth/signout
"/api/health",
"/api/auth/", // future client-facing auth shims (CSRF token issue, etc.)
"/api/btcpay/webhook", // BTCPay needs to reach this without a session
"/api/digest/unsubscribe", // one-click unsubscribe from a digest email (no session)
"/api/network-mode", // returns lan-vs-local; safe to expose
"/api/relay/status", // public relay capabilities — pre-trial visibility
"/api/account/whoami", // returns state — anonymous visitors must call this
// License-status family — anonymous visitors must call these on page
// load to render the right header/badge. Multi-mode handlers branch
// on req.user/req.trial to return per-user views.
"/api/license-status",
"/api/install-id",
// Credit-purchase family — accepts both signed-in users AND anon
// trial cookies (the buy handler routes credits to the right local
// balance based on which identity made the call). Each handler
// validates buyer presence inline, so leaving these "public" is
// safe — it just defers the auth check from the middleware to the
// handler where buyer-type-specific logic lives anyway.
"/api/credits/",
// License purchase + poll — the 3-tier signup modal lets anon
// visitors buy a Pro/Max license at the same moment they create
// their account. /policies is a passthrough to Keysat's public
// /v1/products/.../policies (no auth needed there). /purchase
// accepts an anon buyer_email and records a pending_signups row
// so the poll-settle handler can create the user + attach the
// license + send a magic-link email. /poll just reads invoice
// status from Keysat — same trust model as the credit flow.
//
// Note: /api/license/activate and /api/license/deactivate are
// NOT in this prefix (they're operator-only single-mode endpoints
// that write /data/license.txt; tenants shouldn't reach them).
"/api/license/policies",
"/api/license/purchase",
"/api/license/poll/",
];
// Paths where an unauthenticated visitor is allowed to obtain (or use)
// an anonymous-trial cookie. Restricting trials to the actual
// "use the product" endpoint keeps bots that scrape the homepage from
// minting trial rows just by visiting /.
const TRIAL_ELIGIBLE_PATHS = new Set(["/api/process"]);
function isTrialEligiblePath(reqPath, method) {
if (method !== "POST") return false;
return TRIAL_ELIGIBLE_PATHS.has(reqPath);
}
function isPublicPath(reqPath) {
for (const p of PUBLIC_PATH_PREFIXES) {
if (reqPath === p || reqPath.startsWith(p)) return true;
}
return false;
}
// We only gate /api/* paths. Static assets (/, /auth.html, /assets/*)
// and the auth-flow routes (/auth/*) are ALWAYS reachable so anonymous
// visitors can see the landing page and the sign-in form. The handlers
// behind /api/* are where actual access decisions need to happen.
function isGatedPath(reqPath) {
return reqPath.startsWith("/api/");
}
// Lightweight cookie parser — same dep (`cookie` v1) the rest of the
// project uses. Keeps us from pulling in cookie-parser middleware just
// for one header.
import * as cookie from "cookie";
function parseCookies(req) {
const header = req.headers?.cookie;
if (!header) return {};
try {
return cookie.parse(header);
} catch {
return {};
}
}
// Factory — returns the middleware closure. `mode` is the RECAP_MODE
// env value ("single" | "multi"). Captured at boot so we don't branch
// on a hot path on every request.
export function buildTenantAuthMiddleware({ mode }) {
if (mode !== "multi") {
// Single-mode shim: stamp every request with the synthetic owner
// userId so user-scoped handlers (history reads, library inserts,
// etc.) keep working without per-call branching.
return function singleModeAuth(req, _res, next) {
req.userId = "owner";
req.user = null;
req.recapMode = "single";
next();
};
}
// Multi-mode: validate session on every request. Cache one prepared
// statement per query in module scope (better-sqlite3 idiom — `.prepare`
// returns a reusable handle).
let sessionLookupStmt = null;
let userLookupStmt = null;
let sessionTouchStmt = null;
function stmts() {
const db = getDb();
if (!sessionLookupStmt) {
sessionLookupStmt = db.prepare(
"SELECT * FROM sessions WHERE id = ? AND expires_at > ?",
);
userLookupStmt = db.prepare("SELECT * FROM users WHERE id = ?");
sessionTouchStmt = db.prepare(
"UPDATE sessions SET last_used_at = ? WHERE id = ?",
);
}
return { sessionLookupStmt, userLookupStmt, sessionTouchStmt };
}
// Touch debounce — every authenticated request would otherwise issue
// an UPDATE for last_used_at. We coalesce per-session: at most one
// touch per LAST_USED_DEBOUNCE_MS window.
const LAST_USED_DEBOUNCE_MS = 60_000;
const lastTouchedAt = new Map();
function maybeTouch(sessionId) {
const now = Date.now();
const prev = lastTouchedAt.get(sessionId) || 0;
if (now - prev < LAST_USED_DEBOUNCE_MS) return;
lastTouchedAt.set(sessionId, now);
try {
stmts().sessionTouchStmt.run(now, sessionId);
} catch {
// Touch is best-effort; a write contention here shouldn't fail
// the request.
}
}
return function multiModeAuth(req, res, next) {
req.recapMode = "multi";
// Static assets and the auth-flow pages aren't gated at all — they
// need to be reachable for any visitor to see the landing page or
// sign-in form. Just pass through without attaching anything.
if (!isGatedPath(req.path)) {
return next();
}
const cookies = parseCookies(req);
const sessionId = cookies[SESSION_COOKIE];
if (!sessionId) {
// Look up the trial cookie FIRST (regardless of path) so handlers
// like /api/account/whoami can see req.trial even on public paths.
// We separate "trial is present" from "trial gates access" —
// attachment is unconditional, gating is path-dependent below.
let trial = null;
const trialCookieId = cookies[TRIAL_COOKIE];
if (trialCookieId) {
try {
trial = lookupTrial(trialCookieId);
} catch (err) {
console.warn("[tenant-auth] trial lookup failed:", err);
}
if (trial) {
req.trial = trial;
}
}
// Public paths (auth flow, health, whoami, etc.) pass through.
// Handlers should treat req.userId === undefined as "anonymous"
// and inspect req.trial separately to render appropriate UI.
if (isPublicPath(req.path)) {
req.user = null;
return next();
}
// Second lane: anonymous trial cookie with remaining budget.
// Attach a synthetic userId so /api/process and other gated
// endpoints can run. If the cookie exists but is exhausted,
// fall through to 401 — the UI's "sign up for more" nudge
// takes it from here.
if (trial && hasTrialBudget(trial)) {
req.userId = `anon:${trial.cookie_id}`;
req.user = null;
return next();
}
// No session, no usable trial. The /api/process handler is the
// one place where a fresh trial cookie CAN still be minted on
// first POST — let it through with no user attached so it can
// call issueIfEligible() and either issue + proceed or 401.
if (isTrialEligiblePath(req.path, req.method)) {
req.userId = undefined; // signal: "pre-trial, mint if eligible"
req.user = null;
return next();
}
return res.status(401).json({ error: "auth_required" });
}
let session;
try {
session = stmts().sessionLookupStmt.get(sessionId, Date.now());
} catch (err) {
console.warn("[tenant-auth] session lookup failed:", err);
if (isPublicPath(req.path)) return next();
return res.status(500).json({ error: "auth_lookup_failed" });
}
if (!session) {
// Stale or expired cookie. Clear it so the browser stops sending
// a dead token on every request.
res.clearCookie?.(SESSION_COOKIE);
if (isPublicPath(req.path)) return next();
return res.status(401).json({ error: "session_expired" });
}
let user;
try {
user = stmts().userLookupStmt.get(session.user_id);
} catch (err) {
console.warn("[tenant-auth] user lookup failed:", err);
if (isPublicPath(req.path)) return next();
return res.status(500).json({ error: "user_lookup_failed" });
}
if (!user) {
// User row was deleted but the session row survived — shouldn't
// happen under ON DELETE CASCADE, but defend in depth.
res.clearCookie?.(SESSION_COOKIE);
if (isPublicPath(req.path)) return next();
return res.status(401).json({ error: "user_gone" });
}
req.userId = user.id;
req.user = user;
req.session = session;
maybeTouch(sessionId);
next();
};
}
// Convenience: a guard middleware to chain after auth, for endpoints
// that MUST have a user (where the single-mode synthetic "owner"
// won't do, e.g. /api/account/*). Single-mode falls through because
// req.userId === "owner" is truthy.
export function requireUser(req, res, next) {
if (!req.userId) {
return res.status(401).json({ error: "auth_required" });
}
next();
}
// Operator-only guard: passes through in single mode (the operator
// IS the only user), in multi mode requires req.user.is_admin === 1.
// Used to gate the full settings panel (provider keys, prompts, etc.).
export function requireOperator(req, res, next) {
if (req.recapMode !== "multi") return next();
if (!req.user || !req.user.is_admin) {
return res.status(403).json({ error: "operator_only" });
}
next();
}
+255
View File
@@ -0,0 +1,255 @@
// Per-tenant credit ledger — operations on the tenant_credits SQLite
// table. Two buckets per user:
// purchased_balance — permanent (a la carte purchases + admin grants
// + anon-trial carry-over on signup). Never
// wiped or refilled.
// replenish_balance — refillable (initial signup grant + periodic
// anniversary refill to tenant_default_credits).
// Leftovers at the end of a period are FORFEIT.
//
// Spend order: replenish first, then purchased — refillable bucket
// is "use it or lose it" so it makes sense to burn first.
//
// Multi-mode only. Single-mode doesn't use this table at all.
import { getDb } from "./db.js";
import { getConfigSnapshot } from "./config.js";
const DAY_MS = 24 * 60 * 60 * 1000;
// addMonthClamped(date) — calendar-month add for monthly replenishment.
// Mirrors the relay's same-named helper: Jan 31 → Feb 28/29 (clamped),
// Feb 28 → Mar 28 (preserve day-of-month). Returns a Date.
function addMonthClamped(date) {
const d = new Date(date.getTime());
const year = d.getUTCFullYear();
const month = d.getUTCMonth();
const day = d.getUTCDate();
const lastDayOfTargetMonth = new Date(
Date.UTC(year, month + 2, 0),
).getUTCDate();
const targetDay = Math.min(day, lastDayOfTargetMonth);
return new Date(
Date.UTC(
year,
month + 1,
targetDay,
d.getUTCHours(),
d.getUTCMinutes(),
d.getUTCSeconds(),
d.getUTCMilliseconds(),
),
);
}
// Resolve the next anniversary boundary after `last` for the configured
// period. Returns null if period is "off" (no replenishment).
function nextReplenishAt(lastEpochMs, period) {
if (period === "off" || !lastEpochMs) return null;
if (period === "daily") return lastEpochMs + DAY_MS;
if (period === "weekly") return lastEpochMs + 7 * DAY_MS;
if (period === "monthly") {
return addMonthClamped(new Date(lastEpochMs)).getTime();
}
return null;
}
// Read the current operator config relevant to credits. Cached per-
// request by getConfigSnapshot (which polls config.js's snapshot).
async function readCreditConfig() {
const snap = await getConfigSnapshot();
const period =
snap?.tenant_credit_replenish_period &&
["off", "daily", "weekly", "monthly"].includes(
snap.tenant_credit_replenish_period,
)
? snap.tenant_credit_replenish_period
: "off";
const defaultCredits = Math.max(
0,
parseInt(snap?.tenant_default_credits ?? 5, 10) || 0,
);
return { period, defaultCredits };
}
// Internal: ensure a tenant_credits row exists for this user. New users
// (just signed up but no row yet — shouldn't happen post-auth-routes-fix
// but defensive) get a row with replenish_balance = current default.
function ensureRow(userId, defaultCredits) {
const db = getDb();
const existing = db
.prepare("SELECT * FROM tenant_credits WHERE user_id = ?")
.get(userId);
if (existing) return existing;
const now = Date.now();
db.prepare(
`INSERT INTO tenant_credits
(user_id, purchased_balance, replenish_balance, last_replenish_at,
lifetime_granted, lifetime_consumed)
VALUES (?, 0, ?, ?, ?, 0)`,
).run(userId, defaultCredits, now, defaultCredits);
return db
.prepare("SELECT * FROM tenant_credits WHERE user_id = ?")
.get(userId);
}
// Apply periodic refill if due. Returns the (possibly updated) row.
// Anniversary semantics: if last_replenish_at + period_ms <= now, the
// replenish bucket is RESET to defaultCredits (any leftover is
// forfeit), and last_replenish_at is advanced. Idempotent if called
// multiple times in the same period (no-op).
//
// "Multi-period catch-up" rule: if a user has been idle for several
// periods, only ONE refill is applied (we don't stack refills from
// missed periods). They effectively lost the credits for the missed
// days — same as a per-day allowance in any other SaaS.
function maybeReplenish(row, period, defaultCredits) {
if (period === "off") return row;
const due = nextReplenishAt(row.last_replenish_at, period);
if (due === null) return row;
const now = Date.now();
if (now < due) return row;
const db = getDb();
db.prepare(
`UPDATE tenant_credits
SET replenish_balance = ?, last_replenish_at = ?
WHERE user_id = ?`,
).run(defaultCredits, now, row.user_id);
return {
...row,
replenish_balance: defaultCredits,
last_replenish_at: now,
};
}
// ── Public API ──────────────────────────────────────────────────────────
// getOrInit(userId) — fetch the tenant_credits row, lazily refilling if
// the period boundary has passed. Returns the canonical shape for
// callers that just want to display + compute totals.
export async function getOrInit(userId) {
if (!userId) return null;
const { period, defaultCredits } = await readCreditConfig();
let row = ensureRow(userId, defaultCredits);
row = maybeReplenish(row, period, defaultCredits);
return {
user_id: row.user_id,
purchased: row.purchased_balance,
replenish: row.replenish_balance,
total: row.purchased_balance + row.replenish_balance,
last_replenish_at: row.last_replenish_at,
lifetime_granted: row.lifetime_granted,
lifetime_consumed: row.lifetime_consumed,
period, // surfaces config in returned shape for UI hints
};
}
// gateAndDebit(userId) — atomic: refill if due, check total > 0, debit
// one credit (replenish first, then purchased). Returns
// { ok: true, total, source: "replenish"|"purchased" } on success,
// { ok: false, reason: "no_credits", total: 0 } if nothing available.
export async function gateAndDebit(userId) {
if (!userId) return { ok: false, reason: "no_user_id" };
const state = await getOrInit(userId);
if (!state) return { ok: false, reason: "no_user_id" };
if (state.total <= 0) {
return { ok: false, reason: "no_credits", total: 0 };
}
const db = getDb();
const tx = db.transaction(() => {
if (state.replenish > 0) {
db.prepare(
`UPDATE tenant_credits
SET replenish_balance = replenish_balance - 1,
lifetime_consumed = lifetime_consumed + 1
WHERE user_id = ?`,
).run(userId);
return "replenish";
}
db.prepare(
`UPDATE tenant_credits
SET purchased_balance = purchased_balance - 1,
lifetime_consumed = lifetime_consumed + 1
WHERE user_id = ?`,
).run(userId);
return "purchased";
});
const source = tx();
// Re-read for the new total. Cheap — same row, no replenish-check
// needed since we just touched it.
const fresh = db
.prepare(
"SELECT purchased_balance, replenish_balance FROM tenant_credits WHERE user_id = ?",
)
.get(userId);
return {
ok: true,
total: (fresh.purchased_balance || 0) + (fresh.replenish_balance || 0),
source,
};
}
// addPurchased(userId, amount) — increment the permanent bucket. Used
// by admin grants AND a la carte purchase apply AND anon-trial
// carry-over on signup. Increments lifetime_granted too.
//
// We don't replenish-check here — adding to the permanent bucket
// shouldn't trigger a refill side-effect. The refill happens lazily
// on the next getOrInit() call.
export function addPurchased(userId, amount) {
if (!userId || !Number.isFinite(amount) || amount <= 0) return null;
const db = getDb();
const existing = db
.prepare("SELECT user_id FROM tenant_credits WHERE user_id = ?")
.get(userId);
if (existing) {
db.prepare(
`UPDATE tenant_credits
SET purchased_balance = purchased_balance + ?,
lifetime_granted = lifetime_granted + ?
WHERE user_id = ?`,
).run(amount, amount, userId);
} else {
// Row didn't exist — initialize WITHOUT a replenishable seed
// (this user hasn't been through the signup flow on this Recap;
// probably an admin granting credits before they sign in).
const now = Date.now();
db.prepare(
`INSERT INTO tenant_credits
(user_id, purchased_balance, replenish_balance, last_replenish_at,
lifetime_granted, lifetime_consumed)
VALUES (?, ?, 0, ?, ?, 0)`,
).run(userId, amount, now, amount);
}
return db
.prepare("SELECT * FROM tenant_credits WHERE user_id = ?")
.get(userId);
}
// seedSignup(userId, amount?) — initialize a tenant_credits row at
// signup. Seeds replenish_balance with the configured default
// (overridable for testing), sets last_replenish_at = now so the
// first refill boundary is computed correctly.
export async function seedSignup(userId, amountOverride) {
if (!userId) return null;
const { defaultCredits } = await readCreditConfig();
const amount =
typeof amountOverride === "number" && amountOverride >= 0
? amountOverride
: defaultCredits;
const db = getDb();
const existing = db
.prepare("SELECT user_id FROM tenant_credits WHERE user_id = ?")
.get(userId);
if (existing) return existing; // don't re-seed
const now = Date.now();
db.prepare(
`INSERT INTO tenant_credits
(user_id, purchased_balance, replenish_balance, last_replenish_at,
lifetime_granted, lifetime_consumed)
VALUES (?, 0, ?, ?, ?, 0)`,
).run(userId, amount, now, amount);
return db
.prepare("SELECT * FROM tenant_credits WHERE user_id = ?")
.get(userId);
}
+37
View File
@@ -0,0 +1,37 @@
// Tests for server/anon-trial.js — focused on getClientIp, which underpins
// the per-IP trial cap. The DB-backed minting paths need a multi-mode SQLite
// handle and are exercised by integration tests; here we lock down that the
// client IP is taken from Express's trust-proxy-resolved req.ip, never from a
// raw client-supplied X-Forwarded-For header.
import { test, describe } from "node:test";
import { strict as assert } from "node:assert";
import { getClientIp } from "../anon-trial.js";
describe("getClientIp", () => {
test("uses req.ip (Express's trust-proxy-resolved client address)", () => {
assert.equal(getClientIp({ ip: "203.0.113.7" }), "203.0.113.7");
});
test("strips the IPv4-mapped IPv6 prefix", () => {
assert.equal(getClientIp({ ip: "::ffff:203.0.113.7" }), "203.0.113.7");
});
test("falls back to the socket address when req.ip is absent", () => {
assert.equal(
getClientIp({ socket: { remoteAddress: "::ffff:198.51.100.9" } }),
"198.51.100.9",
);
});
test("does NOT trust a raw client-supplied X-Forwarded-For header", () => {
// Express, not getClientIp, decides the client IP from trust proxy. A
// header Express hasn't blessed must be ignored — so with no req.ip we
// fall through to the socket address, never the spoofed header value.
const spoofed = {
headers: { "x-forwarded-for": "1.2.3.4" },
socket: { remoteAddress: "203.0.113.7" },
};
assert.equal(getClientIp(spoofed), "203.0.113.7");
});
});
+88
View File
@@ -0,0 +1,88 @@
// Tests for server/audio.js — focused on the SSRF guard around
// downloadPodcastAudio (a fully user-controlled outbound fetch). The
// ffprobe/ffmpeg helpers need real media + binaries and aren't covered here.
import { test, describe } from "node:test";
import { strict as assert } from "node:assert";
import { isBlockedAddress, downloadPodcastAudio } from "../audio.js";
const SINK = "/tmp/recap-audio-test-should-not-be-written";
describe("isBlockedAddress", () => {
test("blocks IPv4 loopback / private / link-local / reserved", () => {
for (const ip of [
"127.0.0.1", "127.1.2.3",
"10.0.0.1", "172.16.0.1", "172.31.255.255", "192.168.1.1",
"169.254.169.254", // cloud metadata
"100.64.0.1", // CGNAT
"0.0.0.0",
"224.0.0.1", "255.255.255.255",
]) {
assert.equal(isBlockedAddress(ip), true, `${ip} should be blocked`);
}
});
test("allows ordinary public IPv4 (incl. 172.x boundaries)", () => {
for (const ip of ["8.8.8.8", "1.1.1.1", "93.184.216.34", "172.15.0.1", "172.32.0.1"]) {
assert.equal(isBlockedAddress(ip), false, `${ip} should be allowed`);
}
});
test("blocks IPv6 loopback / ULA / link-local / multicast + IPv4-mapped privates", () => {
for (const ip of [
"::1", "::", "fc00::1", "fd12:3456::1", "fe80::1", "ff02::1",
"::ffff:127.0.0.1", "::ffff:169.254.169.254",
]) {
assert.equal(isBlockedAddress(ip), true, `${ip} should be blocked`);
}
});
test("blocks hex-encoded embedded-IPv4 IPv6 forms (mapped/SIIT/NAT64/6to4)", () => {
for (const ip of [
"::ffff:7f00:1", // IPv4-mapped 127.0.0.1, hex form
"::ffff:0:7f00:1", // SIIT 127.0.0.1
"64:ff9b::7f00:1", // NAT64 well-known prefix of 127.0.0.1
"2002:7f00:1::", // 6to4 of 127.0.0.1
]) {
assert.equal(isBlockedAddress(ip), true, `${ip} should be blocked`);
}
});
test("allows ordinary public IPv6", () => {
assert.equal(isBlockedAddress("2606:4700:4700::1111"), false);
assert.equal(isBlockedAddress("::ffff:8.8.8.8"), false);
});
test("blocks junk / empty / non-strings", () => {
assert.equal(isBlockedAddress(""), true);
assert.equal(isBlockedAddress(null), true);
assert.equal(isBlockedAddress("not-an-ip"), true);
});
});
describe("downloadPodcastAudio SSRF guard", () => {
test("rejects non-HTTP(S) schemes", async () => {
await assert.rejects(downloadPodcastAudio("file:///etc/passwd", SINK), /non-HTTP/);
await assert.rejects(downloadPodcastAudio("ftp://example.com/x", SINK), /non-HTTP/);
});
test("rejects internal / private destinations before connecting", async () => {
// 127.0.0.1:1 would refuse instantly if we connected; we must reject at
// the DNS-guard step instead (proving the guard fires before connect).
await assert.rejects(
downloadPodcastAudio("http://127.0.0.1:1/x", SINK),
/disallowed address/,
);
await assert.rejects(
downloadPodcastAudio("http://169.254.169.254/latest/meta-data/", SINK),
/disallowed address/,
);
});
test("rejects a malformed URL", async () => {
await assert.rejects(
downloadPodcastAudio("not a url at all", SINK),
/invalid podcast audio URL/,
);
});
});
+248
View File
@@ -0,0 +1,248 @@
// Pure / injectable-logic tests for Daily Digest episode synthesis. The
// relay round-trip and FS cache write-back aren't exercised here (a fake
// provider stands in, save:false skips disk); this nails prompt shaping,
// the operator-string scrub backstop, and the get-or-generate cache gate.
import { test, describe } from "node:test";
import assert from "node:assert/strict";
import {
buildOverviewPrompt,
scrubOperatorStrings,
synthesizeEpisodeOverview,
getOrCreateEpisodeOverview,
selectDigestEpisodes,
scopeForUser,
nextDigestWatermark,
} from "../daily-digest.js";
import { renderDigestEmail } from "../email-template.js";
const record = (over = {}) => ({
id: "1700000000000-abc",
title: "How Markets Work",
type: "podcast",
chunks: [
{ title: "Supply & demand", summary: "Prices clear where the two curves meet." },
{ title: "Information", summary: "Asymmetry distorts outcomes." },
],
...over,
});
// A provider stub recording calls, returning a fixed analyze result.
const fakeProvider = (text, sink = {}) => ({
async analyzeText(args) {
sink.calls = (sink.calls || 0) + 1;
sink.lastArgs = args;
return { text };
},
});
describe("buildOverviewPrompt", () => {
test("includes title, type, and each topic's title + summary", () => {
const p = buildOverviewPrompt(record());
assert.match(p, /"How Markets Work"/);
assert.match(p, /podcast episode/);
assert.match(p, /- Supply & demand: Prices clear/);
assert.match(p, /- Information: Asymmetry distorts/);
assert.match(p, /12 paragraph/);
});
test("a topic with no summary still contributes its title", () => {
const p = buildOverviewPrompt(
record({ chunks: [{ title: "Loose ends" }] }),
);
assert.match(p, /- Loose ends/);
assert.doesNotMatch(p, /Loose ends:/); // no trailing colon when summary absent
});
test("null-safe: missing fields fall back", () => {
const p = buildOverviewPrompt({});
assert.match(p, /"Untitled"/);
assert.match(p, /recording/); // unknown type
});
});
describe("scrubOperatorStrings", () => {
test("removes operator/infra tokens", () => {
const out = scrubOperatorStrings(
"Routed via Spark Control and the vLLM box on Parakeet.",
);
assert.doesNotMatch(out, /spark control/i);
assert.doesNotMatch(out, /vllm/i);
assert.doesNotMatch(out, /parakeet/i);
});
test("strips LAN hosts and private IPs, keeps public/content data", () => {
assert.doesNotMatch(
scrubOperatorStrings("see http://immense-voyage.local/admin"),
/\.local/,
);
assert.doesNotMatch(scrubOperatorStrings("host 192.168.1.42 here"), /192\.168/);
// A public dotted quad in content is the user's data, not a leak.
assert.match(scrubOperatorStrings("DNS is 8.8.8.8"), /8\.8\.8\.8/);
});
test("leaves ordinary prose intact", () => {
const clean = "The episode covers supply, demand, and information costs.";
assert.equal(scrubOperatorStrings(clean), clean);
});
test("null-safe", () => {
assert.equal(scrubOperatorStrings(null), "");
assert.equal(scrubOperatorStrings(""), "");
});
});
describe("synthesizeEpisodeOverview", () => {
test("scrubs the model result and passes a stable per-episode jobId", async () => {
const sink = {};
const out = await synthesizeEpisodeOverview(record(), {
provider: fakeProvider("A clear overview from Spark Control.", sink),
});
assert.doesNotMatch(out, /spark control/i);
assert.equal(sink.calls, 1);
assert.equal(sink.lastArgs.jobId, "digest-1700000000000-abc");
});
test("throws when there are no topics", async () => {
await assert.rejects(
() => synthesizeEpisodeOverview(record({ chunks: [] }), { provider: fakeProvider("x") }),
/no topic summaries/,
);
});
test("throws when the model returns nothing usable", async () => {
await assert.rejects(
() => synthesizeEpisodeOverview(record(), { provider: fakeProvider(" ") }),
/empty synthesis result/,
);
});
});
describe("getOrCreateEpisodeOverview", () => {
test("cache hit returns stored overview without calling the provider", async () => {
const sink = {};
const res = await getOrCreateEpisodeOverview({
record: record({ digestOverview: "Already done." }),
provider: fakeProvider("fresh", sink),
save: false,
});
assert.equal(res.cached, true);
assert.equal(res.overview, "Already done.");
assert.equal(sink.calls || 0, 0);
});
test("cache miss synthesizes (save:false skips disk)", async () => {
const sink = {};
const res = await getOrCreateEpisodeOverview({
record: record(),
provider: fakeProvider("A fresh overview.", sink),
save: false,
});
assert.equal(res.cached, false);
assert.equal(res.overview, "A fresh overview.");
assert.equal(sink.calls, 1);
});
});
describe("selectDigestEpisodes", () => {
const at = (iso, id) => ({ id, title: id, type: "youtube", url: "", createdAt: iso });
const sessions = [
at("2026-06-14T00:00:00.000Z", "old"),
at("2026-06-15T09:00:00.000Z", "a"),
at("2026-06-15T10:00:00.000Z", "b"),
];
const watermark = new Date("2026-06-15T08:00:00.000Z").getTime();
test("keeps only recaps created after the watermark, oldest first", () => {
const { episodes, total, overflow } = selectDigestEpisodes(sessions, watermark);
assert.deepEqual(episodes.map((e) => e.id), ["a", "b"]);
assert.equal(total, 2);
assert.equal(overflow, 0);
});
test("caps the list and reports the overflow", () => {
const many = Array.from({ length: 13 }, (_, i) =>
at(`2026-06-15T1${i % 10}:00:00.000Z`, `e${i}`),
);
const { episodes, overflow, total } = selectDigestEpisodes(many, 0, 10);
assert.equal(episodes.length, 10);
assert.equal(overflow, 3);
assert.equal(total, 13);
});
test("empty / malformed-date inputs", () => {
assert.deepEqual(selectDigestEpisodes([], watermark).episodes, []);
assert.deepEqual(selectDigestEpisodes(null, watermark).episodes, []);
assert.deepEqual(
selectDigestEpisodes([at("not-a-date", "x")], 0).episodes,
[],
);
});
test("null watermark treats everything as fresh", () => {
assert.equal(selectDigestEpisodes(sessions, null).total, 3);
});
});
describe("nextDigestWatermark", () => {
const t = (iso) => new Date(iso).getTime();
const e1 = "2026-06-15T09:00:00.000Z";
const e2 = "2026-06-15T10:00:00.000Z";
const e3 = "2026-06-15T11:00:00.000Z";
test("all sent → newest sent createdAt", () => {
assert.equal(nextDigestWatermark([e1, e2, e3], []), t(e3));
});
test("never advances past the oldest failure (so it's retried)", () => {
// sent e1 & e3, e2 failed → watermark just before e2, NOT now/e3.
assert.equal(nextDigestWatermark([e1, e3], [e2]), t(e2) - 1);
});
test("failures newer than everything sent don't pull the watermark back", () => {
assert.equal(nextDigestWatermark([e1, e2], [e3]), t(e2));
});
test("nothing sent → null (caller must not advance)", () => {
assert.equal(nextDigestWatermark([], [e1]), null);
assert.equal(nextDigestWatermark(null, null), null);
});
});
describe("scopeForUser", () => {
test("admin keeps the owner scope; everyone else is their id", () => {
assert.equal(scopeForUser({ id: "u1", is_admin: 1 }), "owner");
assert.equal(scopeForUser({ id: "u1", is_admin: 0 }), "u1");
});
});
describe("renderDigestEmail", () => {
const episodes = [
{ title: "First", type: "podcast", url: "https://x/1", overview: "Ov one." },
{ title: "Second", type: "youtube", url: "", overview: "Ov two." },
];
test("subject reflects count; body carries overviews + unsubscribe link", () => {
const m = renderDigestEmail({
episodes,
overflowCount: 3,
manageUrl: "https://recaps.cc/",
unsubscribeUrl: "https://recaps.cc/api/digest/unsubscribe?token=tok",
});
assert.match(m.subject, /2 new recaps/);
assert.match(m.html, /Ov one\./);
assert.match(m.html, /unsubscribe\?token=tok/);
assert.match(m.html, /3 more/);
assert.match(m.text, /Unsubscribe: https:\/\/recaps\.cc/);
});
test("singular subject for one recap", () => {
const m = renderDigestEmail({
episodes: [episodes[0]],
manageUrl: "https://recaps.cc/",
unsubscribeUrl: "https://recaps.cc/u",
});
assert.match(m.subject, /1 new recap\b/);
});
});
+4 -3
View File
@@ -4,10 +4,11 @@ import { PRICING, calcCost, buildAnalysisPrompt } from "../gemini-helpers.js";
describe("PRICING table", () => {
test("includes all current production model slugs", () => {
assert.ok(PRICING["gemini-3-flash-preview"]);
assert.ok(PRICING["gemini-3-pro-preview"]);
assert.ok(PRICING["gemini-3.1-pro-preview"]);
assert.ok(PRICING["gemini-2.5-pro"]);
assert.ok(PRICING["gemini-3-flash-preview"]);
assert.ok(PRICING["gemini-2.5-flash"]);
assert.ok(PRICING["gemini-3.1-flash-lite"]);
});
test("has a 'default' fallback row", () => {
@@ -66,7 +67,7 @@ describe("calcCost", () => {
});
test("formats >$0.01 totals as $X.XXXX", () => {
const cost = calcCost("gemini-3-pro-preview", {
const cost = calcCost("gemini-3.1-pro-preview", {
promptTokenCount: 1_000_000,
candidatesTokenCount: 0,
thoughtsTokenCount: 0,
+42 -11
View File
@@ -43,8 +43,14 @@ describe("initHistory + getHistoryDir", () => {
});
describe("saveToHistory", () => {
// saveToHistory is scope-aware: it takes a `scope` first and writes under
// history/<scope>/. Tests use the "owner" scope (single-mode / operator).
const SCOPE = "owner";
const scopeFile = (id) => path.join(historyDir, SCOPE, `${id}.json`);
test("returns an id and writes a file with the expected shape", async () => {
const id = await history.saveToHistory(
SCOPE,
"videoId123",
"https://youtu.be/videoId123",
"My title",
@@ -56,7 +62,7 @@ describe("saveToHistory", () => {
);
assert.match(id, /^\d+-videoId123$/);
const raw = await fs.readFile(path.join(historyDir, `${id}.json`), "utf-8");
const raw = await fs.readFile(scopeFile(id), "utf-8");
const record = JSON.parse(raw);
assert.equal(record.id, id);
assert.equal(record.videoId, "videoId123");
@@ -74,6 +80,7 @@ describe("saveToHistory", () => {
test("falls back to 'Untitled' when title is empty", async () => {
const id = await history.saveToHistory(
SCOPE,
"noTitleX",
"url",
"",
@@ -83,20 +90,20 @@ describe("saveToHistory", () => {
"",
"youtube"
);
const raw = await fs.readFile(path.join(historyDir, `${id}.json`), "utf-8");
const raw = await fs.readFile(scopeFile(id), "utf-8");
const record = JSON.parse(raw);
assert.equal(record.title, "Untitled");
});
test("defaults type to 'youtube' when not specified", async () => {
const id = await history.saveToHistory("vid", "url", "t", [], [], [], "", null);
const raw = await fs.readFile(path.join(historyDir, `${id}.json`), "utf-8");
const id = await history.saveToHistory(SCOPE, "vid", "url", "t", [], [], [], "", null);
const raw = await fs.readFile(scopeFile(id), "utf-8");
assert.equal(JSON.parse(raw).type, "youtube");
});
test("encodes long podcast guids into a base64-truncated id suffix", async () => {
const longGuid = "https://example.com/podcasts/feed.xml#episode-uuid-very-long-string";
const id = await history.saveToHistory(longGuid, longGuid, "ep", [], [], [], "", "podcast");
const id = await history.saveToHistory(SCOPE, longGuid, longGuid, "ep", [], [], [], "", "podcast");
// suffix should be 16 base64 chars, not the raw URL
assert.ok(!id.includes("https"));
assert.match(id, /^\d+-[A-Za-z0-9_-]{16}$/);
@@ -104,9 +111,10 @@ describe("saveToHistory", () => {
});
describe("loadMeta + saveMeta", () => {
// loadMeta/saveMeta are scope-aware (history/<scope>/_meta.json).
test("loadMeta returns default empty shape when file missing", async () => {
// Use a fresh sub-history to ensure no prior _meta.json
const meta = await history.loadMeta();
// A scope that has never been written → default shape.
const meta = await history.loadMeta("never-written-scope");
assert.ok(Array.isArray(meta.folders));
assert.ok(Array.isArray(meta.uncategorized));
});
@@ -116,15 +124,38 @@ describe("loadMeta + saveMeta", () => {
folders: [{ id: "f1", name: "Bitcoin podcasts", collapsed: false, items: ["s1", "s2"] }],
uncategorized: ["s3"],
};
await history.saveMeta(original);
const loaded = await history.loadMeta();
await history.saveMeta("owner", original);
const loaded = await history.loadMeta("owner");
assert.deepEqual(loaded, original);
});
test("loadMeta returns default when _meta.json is corrupt", async () => {
await fs.writeFile(path.join(historyDir, "_meta.json"), "{ this is not json");
const loaded = await history.loadMeta();
const dir = path.join(historyDir, "corrupt-scope");
await fs.mkdir(dir, { recursive: true });
await fs.writeFile(path.join(dir, "_meta.json"), "{ this is not json");
const loaded = await history.loadMeta("corrupt-scope");
assert.deepEqual(loaded.folders, []);
assert.deepEqual(loaded.uncategorized, []);
});
});
describe("safeFilename", () => {
// Exported so callers writing user content to disk (e.g. library import)
// can share the one guard instead of rolling their own.
test("returns valid ids unchanged", () => {
for (const id of ["abc123", "a_b-c", "VIDEOid123", "0"]) {
assert.equal(history.safeFilename(id), id);
}
});
test("rejects traversal, separators, and other unsafe chars", () => {
for (const bad of ["../../evil", "..", "a/b", "a\\b", "a.json", "foo bar", "", "a:b"]) {
assert.throws(() => history.safeFilename(bad), /invalid_session_id/);
}
});
test("rejects non-strings", () => {
assert.throws(() => history.safeFilename(123), /invalid_session_id/);
assert.throws(() => history.safeFilename(null), /invalid_session_id/);
});
});
+79
View File
@@ -0,0 +1,79 @@
// Tests for the ephemeral sessions the subscription background processor
// mints to run /api/process AS each item's owner (per-tenant subscriptions,
// step 4). The mechanism reuses the real sessions table, so verifying the
// row it writes is valid + non-expired (and gets cleaned up) is enough to
// trust the existing cookie → tenant-auth → req.user chain.
import { test, describe, before, after } from "node:test";
import { strict as assert } from "node:assert";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import { initDb, getDb, closeDb } from "../db.js";
import {
mintInternalSession,
deleteInternalSession,
adminUserId,
} from "../auth-routes.js";
let dataDir;
function makeUser({ id, email, isAdmin = 0 }) {
getDb()
.prepare(
`INSERT INTO users (id, email, created_at, synthetic_install_id, is_admin)
VALUES (?, ?, ?, ?, ?)`,
)
.run(id, email, Date.now(), `inst-${id}`, isAdmin);
}
before(async () => {
dataDir = mkdtempSync(path.join(tmpdir(), "recap-sess-"));
await initDb({ dataDir });
});
after(() => {
closeDb();
rmSync(dataDir, { recursive: true, force: true });
});
describe("mintInternalSession / deleteInternalSession", () => {
test("creates a valid, non-expired session row for the user", () => {
makeUser({ id: "u-tenant", email: "tenant@example.com" });
const token = mintInternalSession("u-tenant");
assert.ok(typeof token === "string" && token.length > 20);
// Looks up exactly like tenant-auth does: by id, must be unexpired.
const row = getDb()
.prepare("SELECT * FROM sessions WHERE id = ? AND expires_at > ?")
.get(token, Date.now());
assert.ok(row, "session row exists and is not expired");
assert.equal(row.user_id, "u-tenant");
assert.ok(row.expires_at > Date.now(), "expires in the future");
});
test("deleteInternalSession removes the row (no lingering identity)", () => {
makeUser({ id: "u-temp", email: "temp@example.com" });
const token = mintInternalSession("u-temp");
assert.ok(getDb().prepare("SELECT 1 FROM sessions WHERE id = ?").get(token));
deleteInternalSession(token);
assert.equal(
getDb().prepare("SELECT 1 FROM sessions WHERE id = ?").get(token),
undefined,
);
});
test("deleteInternalSession tolerates null / unknown tokens", () => {
// Should not throw.
deleteInternalSession(null);
deleteInternalSession("does-not-exist");
});
});
describe("adminUserId", () => {
test("returns the operator (is_admin = 1) user id", () => {
makeUser({ id: "u-admin", email: "admin@example.com", isAdmin: 1 });
assert.equal(adminUserId(), "u-admin");
});
});
@@ -0,0 +1,51 @@
// Pure-logic tests for the expiry-reminder cadence: which reminder kind
// (if any) applies to a relay subscription row right now. The send/dedup
// path hits SQLite + SMTP and isn't unit-tested here; this nails the
// decision that drives it.
import { test, describe } from "node:test";
import assert from "node:assert/strict";
import { reminderKindFor } from "../subscription-reminders.js";
const sub = (days_left, expired = false) => ({
tier: "pro",
expires_at: "2026-07-01T00:00:00.000Z",
expired,
days_left,
});
describe("reminderKindFor", () => {
test("upcoming: smallest crossed threshold wins", () => {
assert.equal(reminderKindFor(sub(8)), null); // beyond 7d
assert.equal(reminderKindFor(sub(7)), "upcoming_7d");
assert.equal(reminderKindFor(sub(5)), "upcoming_7d");
assert.equal(reminderKindFor(sub(2)), "upcoming_7d");
assert.equal(reminderKindFor(sub(1)), "upcoming_1d");
assert.equal(reminderKindFor(sub(0)), "upcoming_1d"); // expires today, not yet lapsed
});
test("lapsed: only within the lapsed window", () => {
assert.equal(reminderKindFor(sub(0, true)), "lapsed");
assert.equal(reminderKindFor(sub(-1, true)), "lapsed");
assert.equal(reminderKindFor(sub(-3, true)), "lapsed");
assert.equal(reminderKindFor(sub(-4, true)), null); // aged out (window=3)
});
test("null-safe / malformed input", () => {
assert.equal(reminderKindFor(null), null);
assert.equal(reminderKindFor({}), null);
assert.equal(reminderKindFor({ expired: false }), null); // no days_left
assert.equal(reminderKindFor({ days_left: "soon" }), null);
});
test("respects custom thresholds", () => {
const upcoming = [
{ days: 14, kind: "upcoming_14d" },
{ days: 3, kind: "upcoming_3d" },
];
assert.equal(reminderKindFor(sub(14), { upcoming }), "upcoming_14d");
assert.equal(reminderKindFor(sub(3), { upcoming }), "upcoming_3d");
assert.equal(reminderKindFor(sub(15), { upcoming }), null);
});
});
+343
View File
@@ -0,0 +1,343 @@
import { test, describe } from "node:test";
import { strict as assert } from "node:assert";
import { mkdtempSync, mkdirSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import { readFileSync, existsSync } from "node:fs";
import { initHistory, getScopeHistoryDir, getHistoryDir } from "../history.js";
import {
getProcessedVideoIds,
isKnownVideo,
loadSubscriptions,
saveSubscriptions,
loadSkipList,
addToSkipList,
loadSeenList,
addToSeenList,
loadAutoQueue,
saveAutoQueue,
mutateAutoQueue,
listSubscriptionScopes,
migrateGlobalSubscriptionsToOwner,
} from "../subscriptions.js";
// ── isKnownVideo (pure dedup predicate) ──────────────────────────────────
describe("isKnownVideo", () => {
const sets = {
processedIds: new Set(["P1"]),
queuedIds: new Set(["Q1"]),
skippedIds: new Set(["S1"]),
seenIds: new Set(["N1"]),
};
test("true when the id is in the library (processed)", () => {
assert.equal(isKnownVideo("P1", sets), true);
});
test("true when the id is already queued", () => {
assert.equal(isKnownVideo("Q1", sets), true);
});
test("true when the id was declined (skip list)", () => {
assert.equal(isKnownVideo("S1", sets), true);
});
test("true when the id was offered before (seen list)", () => {
assert.equal(isKnownVideo("N1", sets), true);
});
test("false for a genuinely new id", () => {
assert.equal(isKnownVideo("NEW", sets), false);
});
test("returns a real boolean, not a Set/undefined", () => {
assert.equal(typeof isKnownVideo("NEW", sets), "boolean");
assert.equal(typeof isKnownVideo("P1", sets), "boolean");
});
test("tolerates missing/partial set bags", () => {
assert.equal(isKnownVideo("X", {}), false);
assert.equal(isKnownVideo("X", undefined), false);
assert.equal(isKnownVideo("X", { processedIds: new Set(["X"]) }), true);
});
});
// ── getProcessedVideoIds (scope-aware library scan) ──────────────────────
// This is the function whose wrong-directory bug caused the auto-queue to
// re-offer already-summarized videos: it must scan history/<scope>/, NOT
// the top-level history dir, and must not leak ids across scopes.
describe("getProcessedVideoIds", () => {
function freshDataDir() {
const dataDir = mkdtempSync(path.join(tmpdir(), "recap-subs-"));
return dataDir;
}
test("scans the scope's library dir and collects videoIds", async () => {
const dataDir = freshDataDir();
await initHistory({ dataDir, mode: "multi" });
const ownerDir = getScopeHistoryDir("owner");
mkdirSync(ownerDir, { recursive: true });
writeFileSync(path.join(ownerDir, "a.json"), JSON.stringify({ videoId: "VID_A" }));
writeFileSync(path.join(ownerDir, "b.json"), JSON.stringify({ videoId: "VID_B" }));
const ids = await getProcessedVideoIds("owner");
assert.equal(ids.size, 2);
assert.ok(ids.has("VID_A"));
assert.ok(ids.has("VID_B"));
});
test("does NOT leak ids across scopes (per-tenant isolation)", async () => {
const dataDir = freshDataDir();
await initHistory({ dataDir, mode: "multi" });
const ownerDir = getScopeHistoryDir("owner");
const tenantDir = getScopeHistoryDir("tenant123");
mkdirSync(ownerDir, { recursive: true });
mkdirSync(tenantDir, { recursive: true });
writeFileSync(path.join(ownerDir, "a.json"), JSON.stringify({ videoId: "OWNER_VID" }));
writeFileSync(path.join(tenantDir, "c.json"), JSON.stringify({ videoId: "TENANT_VID" }));
const ownerIds = await getProcessedVideoIds("owner");
assert.ok(ownerIds.has("OWNER_VID"));
assert.equal(ownerIds.has("TENANT_VID"), false);
});
test("ignores _meta.json and non-JSON / malformed files", async () => {
const dataDir = freshDataDir();
await initHistory({ dataDir, mode: "multi" });
const ownerDir = getScopeHistoryDir("owner");
mkdirSync(ownerDir, { recursive: true });
writeFileSync(path.join(ownerDir, "good.json"), JSON.stringify({ videoId: "GOOD" }));
writeFileSync(path.join(ownerDir, "_meta.json"), JSON.stringify({ folders: [], uncategorized: [] }));
writeFileSync(path.join(ownerDir, "notes.txt"), "videoId: NOPE");
writeFileSync(path.join(ownerDir, "broken.json"), "{ not valid json");
writeFileSync(path.join(ownerDir, "nofield.json"), JSON.stringify({ title: "no videoId here" }));
const ids = await getProcessedVideoIds("owner");
assert.deepEqual([...ids], ["GOOD"]);
});
test("returns an empty set for a scope with no library yet", async () => {
const dataDir = freshDataDir();
await initHistory({ dataDir, mode: "multi" });
const ids = await getProcessedVideoIds("never-summarized");
assert.equal(ids.size, 0);
});
test("defaults to the owner scope when none is passed", async () => {
const dataDir = freshDataDir();
await initHistory({ dataDir, mode: "single" });
const ownerDir = getScopeHistoryDir("owner");
mkdirSync(ownerDir, { recursive: true });
writeFileSync(path.join(ownerDir, "a.json"), JSON.stringify({ videoId: "DEFAULT_OWNER" }));
const ids = await getProcessedVideoIds(); // no arg → "owner"
assert.ok(ids.has("DEFAULT_OWNER"));
});
// Regression: after the 0.2.147 migration the subscription sidecar files
// live inside the scope dir. They must NOT be treated as session records
// (that produced the phantom "Invalid Date · undefined topics" library
// entry). Verify the ROOT_SIDECARS filter skips them even if one happens
// to carry a top-level videoId.
test("skips subscription sidecar files when scanning the library", async () => {
const dataDir = freshDataDir();
await initHistory({ dataDir, mode: "multi" });
const ownerDir = getScopeHistoryDir("owner");
mkdirSync(ownerDir, { recursive: true });
writeFileSync(path.join(ownerDir, "real.json"), JSON.stringify({ videoId: "REAL" }));
for (const f of ["subscriptions.json", "auto-queue.json", "skip-list.json", "seen-list.json"]) {
// Even with a (bogus) top-level videoId, a sidecar must be ignored.
writeFileSync(path.join(ownerDir, f), JSON.stringify({ videoId: "SIDECAR_" + f, items: [] }));
}
const ids = await getProcessedVideoIds("owner");
assert.deepEqual([...ids], ["REAL"]);
});
});
// ── Per-scope storage (subscriptions / skip / seen / auto-queue) ─────────
describe("per-scope storage", () => {
function fresh() {
return mkdtempSync(path.join(tmpdir(), "recap-store-"));
}
test("subscriptions round-trip and stay isolated per scope", async () => {
await initHistory({ dataDir: fresh(), mode: "multi" });
assert.deepEqual(await loadSubscriptions("owner"), []); // empty default
await saveSubscriptions("owner", [{ id: "s1", url: "u1" }]);
await saveSubscriptions("tenantA", [{ id: "s2", url: "u2" }]);
assert.equal((await loadSubscriptions("owner")).length, 1);
assert.equal((await loadSubscriptions("owner"))[0].id, "s1");
assert.equal((await loadSubscriptions("tenantA"))[0].id, "s2");
});
test("skip + seen lists are per-scope Sets", async () => {
await initHistory({ dataDir: fresh(), mode: "multi" });
await addToSkipList("owner", "VID1");
await addToSkipList("owner", "VID2");
await addToSeenList("owner", ["S1", "S2"]);
const skip = await loadSkipList("owner");
const seen = await loadSeenList("owner");
assert.ok(skip.has("VID1") && skip.has("VID2"));
assert.ok(seen.has("S1") && seen.has("S2"));
assert.equal((await loadSkipList("tenantA")).size, 0); // isolated
});
test("auto-queue save/load round-trips", async () => {
await initHistory({ dataDir: fresh(), mode: "multi" });
await saveAutoQueue("owner", [{ id: "a", status: "pending" }]);
const q = await loadAutoQueue("owner");
assert.equal(q.length, 1);
assert.equal(q[0].status, "pending");
});
test("mutateAutoQueue serializes concurrent read-modify-writes (no lost updates)", async () => {
await initHistory({ dataDir: fresh(), mode: "multi" });
await saveAutoQueue("owner", []);
// Fire 20 concurrent appends. Under a naive load→mutate→save these would
// clobber each other; mutateAutoQueue must serialize them.
await Promise.all(
Array.from({ length: 20 }, (_, i) =>
mutateAutoQueue("owner", (items) => {
items.push({ id: `item-${i}` });
}),
),
);
const q = await loadAutoQueue("owner");
assert.equal(q.length, 20, "all 20 concurrent appends survived");
assert.equal(new Set(q.map((x) => x.id)).size, 20, "no duplicates/drops");
});
test("mutateAutoQueue can replace the array (filter/remove)", async () => {
await initHistory({ dataDir: fresh(), mode: "multi" });
await saveAutoQueue("owner", [{ id: "keep" }, { id: "drop" }]);
await mutateAutoQueue("owner", (items) => items.filter((x) => x.id !== "drop"));
const q = await loadAutoQueue("owner");
assert.deepEqual(q.map((x) => x.id), ["keep"]);
});
});
// ── Auto-queue endpoint mutation patterns ────────────────────────────────
// The /api/auto-queue/* handlers run these exact callbacks through
// mutateAutoQueue. Verify the logic directly (the HTTP layer just adds the
// license gate + scope resolution, covered elsewhere).
describe("auto-queue endpoint mutation logic", () => {
async function seed(items) {
await initHistory({ dataDir: mkdtempSync(path.join(tmpdir(), "recap-ep-")), mode: "multi" });
await saveAutoQueue("owner", items);
}
test("approve: pending → approved; rejects non-pending", async () => {
await seed([{ id: "x", status: "pending" }, { id: "y", status: "completed" }]);
// approve x
let item = null, badStatus = null;
await mutateAutoQueue("owner", (items) => {
const it = items.find((q) => q.id === "x");
if (!it) return;
if (it.status !== "pending") { badStatus = it.status; return; }
it.status = "approved";
item = { ...it };
});
assert.equal(badStatus, null);
assert.equal(item.status, "approved");
assert.equal((await loadAutoQueue("owner")).find((q) => q.id === "x").status, "approved");
// approving y (completed) is rejected
badStatus = null;
await mutateAutoQueue("owner", (items) => {
const it = items.find((q) => q.id === "y");
if (it && it.status !== "pending") badStatus = it.status;
});
assert.equal(badStatus, "completed");
});
test("skip: captures videoId then removes the item", async () => {
await seed([{ id: "x", videoId: "VID_X", status: "pending" }, { id: "z", status: "pending" }]);
let videoId = null;
await mutateAutoQueue("owner", (items) => {
const it = items.find((q) => q.id === "x");
if (it && it.videoId) videoId = it.videoId;
return items.filter((q) => q.id !== "x");
});
assert.equal(videoId, "VID_X");
const q = await loadAutoQueue("owner");
assert.deepEqual(q.map((i) => i.id), ["z"]);
});
test("clear-finished: drops completed + failed, keeps active", async () => {
await seed([
{ id: "a", status: "pending" },
{ id: "b", status: "completed" },
{ id: "c", status: "failed" },
{ id: "d", status: "approved" },
]);
let removed = 0;
await mutateAutoQueue("owner", (items) => {
const before = items.length;
const kept = items.filter((q) => !["completed", "failed"].includes(q.status));
removed = before - kept.length;
return kept;
});
assert.equal(removed, 2);
assert.deepEqual((await loadAutoQueue("owner")).map((i) => i.id).sort(), ["a", "d"]);
});
test("delete-by-subscription: removes that sub's items only", async () => {
await seed([
{ id: "a", subscriptionId: "sub1", status: "pending" },
{ id: "b", subscriptionId: "sub2", status: "pending" },
{ id: "c", subscriptionId: "sub1", status: "approved" },
]);
await mutateAutoQueue("owner", (items) => items.filter((q) => q.subscriptionId !== "sub1"));
assert.deepEqual((await loadAutoQueue("owner")).map((i) => i.id), ["b"]);
});
});
// ── Scope enumeration ────────────────────────────────────────────────────
describe("listSubscriptionScopes", () => {
test("always includes owner; adds scopes with non-empty subscriptions", async () => {
await initHistory({ dataDir: mkdtempSync(path.join(tmpdir(), "recap-enum-")), mode: "multi" });
await saveSubscriptions("tenantA", [{ id: "s", url: "u" }]);
await saveSubscriptions("tenantB", []); // empty → excluded
const scopes = await listSubscriptionScopes();
assert.ok(scopes.includes("owner"));
assert.ok(scopes.includes("tenantA"));
assert.ok(!scopes.includes("tenantB"));
});
test("returns just owner when nobody has subscriptions", async () => {
await initHistory({ dataDir: mkdtempSync(path.join(tmpdir(), "recap-enum2-")), mode: "multi" });
const scopes = await listSubscriptionScopes();
assert.deepEqual(scopes, ["owner"]);
});
});
// ── Migration: history-root globals → owner scope ────────────────────────
describe("migrateGlobalSubscriptionsToOwner", () => {
test("moves root files into owner/, idempotently, without clobbering", async () => {
const dataDir = mkdtempSync(path.join(tmpdir(), "recap-mig-"));
await initHistory({ dataDir, mode: "multi" });
const root = getHistoryDir();
mkdirSync(root, { recursive: true });
// Legacy global files at the history root.
writeFileSync(path.join(root, "subscriptions.json"), JSON.stringify({ subscriptions: [{ id: "s1" }] }));
writeFileSync(path.join(root, "auto-queue.json"), JSON.stringify({ items: [{ id: "q1" }] }));
const moved = await migrateGlobalSubscriptionsToOwner();
assert.equal(moved, 2);
// Now readable via the owner scope.
assert.equal((await loadSubscriptions("owner"))[0].id, "s1");
assert.equal((await loadAutoQueue("owner"))[0].id, "q1");
// Source files gone.
assert.equal(existsSync(path.join(root, "subscriptions.json")), false);
// Re-running is a no-op (nothing left to move).
assert.equal(await migrateGlobalSubscriptionsToOwner(), 0);
});
test("does not clobber an existing owner-scope file", async () => {
const dataDir = mkdtempSync(path.join(tmpdir(), "recap-mig2-"));
await initHistory({ dataDir, mode: "multi" });
const root = getHistoryDir();
const ownerDir = getScopeHistoryDir("owner");
mkdirSync(ownerDir, { recursive: true });
// Owner already has subscriptions; a stale root file must NOT overwrite.
writeFileSync(path.join(ownerDir, "subscriptions.json"), JSON.stringify({ subscriptions: [{ id: "KEEP" }] }));
writeFileSync(path.join(root, "subscriptions.json"), JSON.stringify({ subscriptions: [{ id: "STALE" }] }));
await migrateGlobalSubscriptionsToOwner();
assert.equal((await loadSubscriptions("owner"))[0].id, "KEEP");
});
});
+16 -1
View File
@@ -22,6 +22,17 @@ describe("extractVideoId", () => {
assert.equal(extractVideoId("https://www.youtube.com/embed/dQw4w9WgXcQ"), "dQw4w9WgXcQ");
});
test("extracts from /live/ URL (with ?si tracking param)", () => {
assert.equal(
extractVideoId("https://www.youtube.com/live/QEq1Fa-Br0U?si=CqlsUBpyTs_ksqi3"),
"QEq1Fa-Br0U"
);
});
test("extracts from /shorts/ URL", () => {
assert.equal(extractVideoId("https://www.youtube.com/shorts/dQw4w9WgXcQ"), "dQw4w9WgXcQ");
});
test("accepts a bare 11-character id", () => {
assert.equal(extractVideoId("dQw4w9WgXcQ"), "dQw4w9WgXcQ");
});
@@ -207,7 +218,11 @@ describe("retryGemini", () => {
},
{ retries: 2, delayMs: 1, label: "test", log: (msg) => logs.push(msg) }
).catch(() => {});
assert.equal(logs.length, 1);
// retries: 2 → the loop logs twice: the "failed, retrying in …s" notice
// before attempt 2's wait, then "Retrying… (attempt 2/2)" at the top of
// attempt 2. (The test previously expected 1, written before the
// top-of-attempt retry line existed.)
assert.equal(logs.length, 2);
assert.match(logs[0], /test/);
assert.match(logs[0], /retrying/);
});
+258
View File
@@ -0,0 +1,258 @@
// Audio-first ("walking mode") TTS routes. Turns a saved recap's per-topic
// summaries into spoken MP3 clips (via the relay's /relay/tts → Kokoro),
// caches them next to the session JSON, and serves them to the player.
//
// Endpoints (all scope-isolated to the requesting user's library):
// GET /api/tts/availability — { has_tts, allowed, default_voice }
// POST /api/tts/generate/:id/:index — synthesize + cache ONE topic clip
// (idempotent, retried, deduped by job
// id). The player calls this on demand
// as it reaches each topic + prefetches
// the next, so clips are generated when
// needed and retried until they succeed
// rather than skipped.
// GET /api/tts/status/:id — { total, ready:[idx...], done }
// GET /api/tts/audio/:id/:index — serve a cached topic clip (mp3)
//
// Access policy (the "Max gate"):
// - single mode: the operator owns the box AND the TTS hardware, so no
// tier gate — TTS is available whenever the relay advertises has_tts.
// - multi mode admin: the operator; allowed.
// - multi-tenant cloud users: any paid subscription (Pro or Max). The
// operator can tighten this to Max-only here if shared TTS hardware
// throughput becomes a constraint.
//
// Billing: all of a recap's topics share ONE relay job id (`tts:<id>`), so
// the relay charges at most 1 credit to voice an entire recap.
import fs from "fs/promises";
import path from "path";
import {
scopeForRequest,
sessionAudioDir,
loadSession,
patchSession,
} from "./history.js";
import { getProvider, resolveProviderOpts } from "./providers/index.js";
import { getRelayCapabilities } from "./relay-capabilities.js";
const CLIP_FORMAT = "mp3";
const CLIP_EXT = "mp3";
// Whether THIS request's user may generate TTS. See the policy note above.
export function userHasTtsAccess(req) {
// Single mode (or no request context): operator owns the hardware.
if (!req || req.recapMode !== "multi") return true;
// Multi-mode admin = the operator.
if (req.user && req.user.is_admin) return true;
// Multi-tenant cloud user: Pro or Max. Core-decoupling — the tier is the
// relay-owned subscription tier, cached on the Recaps account
// (req.user.tier), kept in sync by the operator grant flow.
const tier = req.user?.tier;
return tier === "pro" || tier === "max";
}
// The text we speak for a topic: its title as a lead-in, then the summary,
// so an eyes-free listener hears what the topic is before its recap.
export function chunkSpeechText(chunk) {
const title = (chunk?.title || "").trim();
const summary = (chunk?.summary || "").trim();
if (title && summary) return `${title}. ${summary}`;
return summary || title || "";
}
function clipFileName(index) {
return `topic-${index}.${CLIP_EXT}`;
}
// Server-side retries per clip on a transient (5xx/network) relay failure,
// on top of any retry the relay itself does.
const GEN_RETRIES = 2;
// Generate + cache ONE topic clip. Idempotent: returns {cached:true} if the
// file already exists. Retries transient failures; a 4xx (e.g. bad voice) or
// empty summary is permanent (no retry). Returns
// { ok, cached?, empty?, error?, voice? }.
async function generateClip({ scope, id, index, chunk, provider, jobId, voice }) {
const dir = sessionAudioDir(scope, id);
const file = path.join(dir, clipFileName(index));
try {
await fs.access(file);
return { ok: true, cached: true };
} catch {}
const text = chunkSpeechText(chunk);
if (!text) return { ok: false, empty: true, error: "empty_summary" };
await fs.mkdir(dir, { recursive: true }).catch(() => {});
let lastErr = null;
for (let attempt = 1; attempt <= GEN_RETRIES + 1; attempt++) {
try {
const r = await provider.tts({ text, voice, format: CLIP_FORMAT, jobId });
await fs.writeFile(file, r.audio);
return { ok: true, voice: r.voice, backend: r.backend };
} catch (err) {
lastErr = err;
const status = err?.status || 0;
console.warn(
`[tts] clip ${index} attempt ${attempt}/${GEN_RETRIES + 1} failed (${status || "net"}): ${err?.message || err}`,
);
if (status >= 400 && status < 500) break; // client error → permanent
if (attempt <= GEN_RETRIES) await new Promise((r2) => setTimeout(r2, 600));
}
}
return { ok: false, error: (lastErr?.message || "tts_failed").slice(0, 200) };
}
function resolveScope(req, res) {
try {
return scopeForRequest(req);
} catch {
res.status(401).json({ error: "no_scope" });
return null;
}
}
export function setupTtsRoutes(app) {
// Lightweight probe for the frontend: should it show the "Listen"
// affordance, and what's the default voice?
app.get("/api/tts/availability", (req, res) => {
const caps = getRelayCapabilities();
res.json({
has_tts: !!caps.has_tts,
tts_backend: caps.tts_backend || null,
default_voice: caps.tts_default_voice || null,
allowed: userHasTtsAccess(req) && !!caps.has_tts,
});
});
// Generate (or return cached) the audio for ONE topic. The player calls
// this on demand as it reaches each topic — and prefetches the next — so a
// clip is generated when needed and RETRIED until it succeeds, rather than
// skipped. Idempotent + deduped by the shared job id (≤1 credit/recap).
//
// Responses:
// 200 { ok:true, index, cached } — clip is ready to play
// 200 { ok:false, empty:true } — topic has no summary text (permanent;
// client should not retry)
// 502 { ok:false, error } — transient failure; client retries
app.post("/api/tts/generate/:id/:index", async (req, res) => {
const scope = resolveScope(req, res);
if (!scope) return;
if (!userHasTtsAccess(req)) {
return res.status(403).json({
error: "tts_requires_subscription",
message: "Audio recaps are available to Pro and Max subscribers.",
});
}
const caps = getRelayCapabilities();
if (!caps.has_tts) {
return res.status(503).json({
error: "tts_unavailable",
message: "Text-to-speech isn't available on this relay right now.",
});
}
const id = req.params.id;
const index = parseInt(req.params.index, 10);
const session = await loadSession(scope, id);
if (!session) return res.status(404).json({ error: "session_not_found" });
const chunks = Array.isArray(session.chunks) ? session.chunks : [];
if (!Number.isInteger(index) || index < 0 || index >= chunks.length) {
return res.status(400).json({ error: "bad_index" });
}
let provider;
try {
provider = getProvider("relay", resolveProviderOpts("relay", { req }));
} catch (err) {
return res.status(503).json({
error: "relay_unavailable",
message: err?.message || "Relay is not configured.",
});
}
const voice =
typeof req.query.voice === "string" && req.query.voice.trim()
? req.query.voice.trim()
: undefined;
const result = await generateClip({
scope,
id,
index,
chunk: chunks[index],
provider,
jobId: `tts:${id}`, // one credit for the whole recap
voice,
});
if (result.ok) {
patchSession(scope, id, {
summaryAudio: {
ready: true,
total: chunks.length,
voice: result.voice || caps.tts_default_voice || null,
format: CLIP_FORMAT,
updatedAt: new Date().toISOString(),
},
}).catch(() => {});
return res.json({ ok: true, index, cached: !!result.cached, voice: result.voice || null });
}
if (result.empty) {
return res.json({ ok: false, index, empty: true, error: "empty_summary" });
}
return res.status(502).json({ ok: false, index, error: result.error || "tts_failed" });
});
// Which topics are already synthesized for a recap.
app.get("/api/tts/status/:id", async (req, res) => {
const scope = resolveScope(req, res);
if (!scope) return;
const session = await loadSession(scope, req.params.id);
if (!session) return res.status(404).json({ error: "session_not_found" });
const total = Array.isArray(session.chunks) ? session.chunks.length : 0;
const dir = sessionAudioDir(scope, req.params.id);
let files = [];
try {
files = await fs.readdir(dir);
} catch {}
const ready = files
.map((f) => {
const m = new RegExp(`^topic-(\\d+)\\.${CLIP_EXT}$`).exec(f);
return m ? Number(m[1]) : null;
})
.filter((n) => n !== null)
.sort((a, b) => a - b);
const caps = getRelayCapabilities();
res.json({
total,
ready,
done: total > 0 && ready.length >= total,
allowed: userHasTtsAccess(req) && !!caps.has_tts,
voice: session.summaryAudio?.voice || caps.tts_default_voice || null,
});
});
// Serve one cached topic clip. sendFile handles Range requests (so the
// <audio> element can seek) and 404s cleanly when the clip isn't ready.
app.get("/api/tts/audio/:id/:index", async (req, res) => {
const scope = resolveScope(req, res);
if (!scope) return;
const idx = parseInt(req.params.index, 10);
if (!Number.isInteger(idx) || idx < 0) {
return res.status(400).json({ error: "bad_index" });
}
const file = path.join(sessionAudioDir(scope, req.params.id), clipFileName(idx));
res.sendFile(
file,
{
headers: {
"Content-Type": "audio/mpeg",
// Content is immutable for a given (session, topic) — safe to
// cache hard, which also primes the Phase 4 offline service worker.
"Cache-Control": "private, max-age=31536000, immutable",
},
},
(err) => {
if (err && !res.headersSent) {
res.status(404).json({ error: "clip_not_ready" });
}
}
);
});
}
+475
View File
@@ -0,0 +1,475 @@
// Resolves "share URLs" from Apple Podcasts and Spotify into something
// Recap's existing podcast pipeline can swallow. Most users share these
// links rather than the underlying RSS feed (which they rarely know
// exists), so transparent resolution turns the most common podcast
// share path into "paste link, hit summarize, done".
//
// Apple Podcasts URLs resolve directly via the public iTunes Lookup API:
// the episode result includes `episodeUrl` (the audio enclosure) and
// the show's `feedUrl`. No API key required, no auth.
//
// Spotify URLs are harder: Spotify-hosted audio is DRM-wrapped and not
// served via a public stream URL. We use the unauthenticated oEmbed
// endpoint to get the episode + show titles, then ask PodcastIndex to
// find the same episode in its RSS-indexed catalog. Spotify Originals
// (Joe Rogan, Anchor exclusives, …) have no RSS counterpart and fail
// the lookup — we surface a clear error in that case so the user
// understands and can paste the RSS link manually.
//
// Returns a normalized shape that maps cleanly onto Recap's existing
// podcast pipeline:
// {
// source: "apple" | "spotify",
// audioUrl: string, // direct audio URL (.mp3/.m4a) — feeds the existing podcast path
// episodeId: string, // stable GUID used by history dedup
// title: string,
// podcastTitle: string,
// uploadDate: string, // "YYYYMMDD"
// durationSec: number?, // null when unknown
// feedUrl: string?, // for context; not required downstream
// }
//
// Throws `URLResolveError` with a `.code` field for things the UI may
// want to format specifically:
// - "spotify_no_rss" → episode is Spotify-exclusive
// - "episode_not_found" → looked up but couldn't match
// - "apple_lookup_failed"
// - "podcastindex_unconfigured"
// - "podcastindex_not_implemented" → caller didn't pass keys
import crypto from "crypto";
import { fetchUrl } from "./util.js";
export class URLResolveError extends Error {
constructor(code, message) {
super(message);
this.name = "URLResolveError";
this.code = code;
}
}
const APPLE_EPISODE_URL_RE =
/^https?:\/\/(?:www\.)?podcasts\.apple\.com\/[^/]+\/podcast\/[^/]+\/id(\d+)(?:\?|.*[?&])i=(\d+)/i;
// Detection only — no I/O.
export function isApplePodcastUrl(url) {
if (!url) return false;
return /^https?:\/\/(?:www\.)?podcasts\.apple\.com\//i.test(url);
}
export function isSpotifyUrl(url) {
if (!url) return false;
return /^https?:\/\/(?:open|play)\.spotify\.com\/(?:episode|show)\//i.test(url);
}
// Fountain (https://fountain.fm) is a Bitcoin-Lightning podcast app
// that hosts a Podcasting 2.0-native catalog. Episode pages are at
// /episode/<short-id>; the underlying media is served from
// feeds.fountain.fm and exposed via standard Open Graph tags
// (og:audio, og:image, og:title) on the public episode HTML — no API
// key required to resolve. Show pages (/show/<id>) aren't supported
// for now; users should paste a specific episode link.
export function isFountainUrl(url) {
if (!url) return false;
return /^https?:\/\/(?:www\.)?fountain\.fm\/episode\//i.test(url);
}
// True if the URL is one of the "share link" forms we know how to turn
// into a podcast audio URL. Callers should only invoke the network-
// touching resolvers when this returns true.
export function isResolvableShareUrl(url) {
return isApplePodcastUrl(url) || isSpotifyUrl(url) || isFountainUrl(url);
}
// ── Apple Podcasts ─────────────────────────────────────────────────────
// Strategy: parse the podcast ID + episode track ID out of the URL,
// hit iTunes Lookup, find the matching episode by trackId. Apple
// returns the episode's actual audio enclosure URL — same URL the
// Apple Podcasts app streams from — so the existing podcast download
// pipeline (audio.downloadPodcastAudio) can swallow it unchanged.
export async function resolveApplePodcastUrl(url) {
const m = url.match(APPLE_EPISODE_URL_RE);
if (!m) {
throw new URLResolveError(
"apple_lookup_failed",
"Apple Podcasts URL is missing podcast ID or episode ID (?i= param)"
);
}
const podcastId = m[1];
const episodeTrackId = m[2];
// The lookup endpoint returns the show metadata as result[0] and the
// most-recent N episodes as result[1..]. Apple silently caps at 200
// even if you ask for more.
const lookupUrl = `https://itunes.apple.com/lookup?id=${encodeURIComponent(
podcastId
)}&entity=podcastEpisode&limit=200`;
let parsed;
try {
const raw = await fetchUrl(lookupUrl);
parsed = JSON.parse(raw);
} catch (err) {
throw new URLResolveError(
"apple_lookup_failed",
`iTunes lookup failed: ${err?.message || err}`
);
}
const results = Array.isArray(parsed?.results) ? parsed.results : [];
const show = results.find((r) => r.wrapperType === "track" || r.kind === "podcast") || {};
const episode = results.find(
(r) =>
r.wrapperType === "podcastEpisode" &&
String(r.trackId) === String(episodeTrackId)
);
if (!episode || !episode.episodeUrl) {
throw new URLResolveError(
"episode_not_found",
`Apple returned ${results.length} results for podcast ${podcastId} but episode ${episodeTrackId} was not among them. The episode may be older than Apple's 200-episode lookup cap.`
);
}
// releaseDate is ISO 8601; collapse to YYYYMMDD to match the rest of
// the pipeline's date convention.
let uploadDate = "";
if (episode.releaseDate) {
try {
const d = new Date(episode.releaseDate);
if (!isNaN(d.getTime())) {
uploadDate = d.toISOString().slice(0, 10).replace(/-/g, "");
}
} catch {}
}
const durationSec =
typeof episode.trackTimeMillis === "number"
? Math.round(episode.trackTimeMillis / 1000)
: null;
return {
source: "apple",
audioUrl: episode.episodeUrl,
episodeId: episode.episodeGuid || `apple-${episodeTrackId}`,
title: episode.trackName || show.collectionName || "Untitled episode",
podcastTitle: show.collectionName || episode.collectionName || "Unknown podcast",
uploadDate,
durationSec,
feedUrl: show.feedUrl || null,
};
}
// ── Spotify ────────────────────────────────────────────────────────────
// Strategy: oEmbed for title/show, then PodcastIndex search to map
// title+show → RSS feed → episode → audio enclosure. Spotify-exclusive
// content has no RSS counterpart and fails the lookup with a clear
// `spotify_no_rss` error.
//
// PodcastIndex auth (https://podcastindex.org/api/dev) requires:
// - `User-Agent` header (PodcastIndex blocks anonymous UAs)
// - `X-Auth-Key`: API key (free, signup at api.podcastindex.org)
// - `X-Auth-Date`: unix timestamp (current time)
// - `Authorization`: sha1(apiKey + apiSecret + apiDate)
export async function resolveSpotifyUrl(url, { podcastIndexKey, podcastIndexSecret } = {}) {
if (!podcastIndexKey || !podcastIndexSecret) {
throw new URLResolveError(
"podcastindex_unconfigured",
'Spotify needs both a free PodcastIndex API Key AND API Secret. Sign up at api.podcastindex.org — your account page shows both credentials side-by-side. Paste them in Recaps → Settings → API Keys → PodcastIndex. (Apple Podcasts and Fountain links work without any API key — try those for the same episode if it\'s also distributed there.)'
);
}
// oEmbed gives us episode title + show name with no auth.
let episodeTitle = "";
let showName = "";
try {
const oemRaw = await fetchUrl(
`https://open.spotify.com/oembed?url=${encodeURIComponent(url)}`
);
const oem = JSON.parse(oemRaw);
episodeTitle = (oem?.title || "").trim();
// oEmbed's "title" includes the show name in some Spotify variants
// (e.g. "<episode> · <show>"). Split if we see the delimiter.
const sep = episodeTitle.lastIndexOf(" · ");
if (sep > 0) {
showName = episodeTitle.slice(sep + 3).trim();
episodeTitle = episodeTitle.slice(0, sep).trim();
}
} catch (err) {
throw new URLResolveError(
"episode_not_found",
`Could not fetch Spotify episode metadata: ${err?.message || err}`
);
}
if (!episodeTitle) {
throw new URLResolveError(
"episode_not_found",
"Spotify oEmbed returned no episode title"
);
}
// Authoritative PodcastIndex search uses byperson/bypath/byterm. The
// episode-search endpoint accepts a free-text query and returns
// candidate episodes across the index.
const q = encodeURIComponent(
showName ? `${episodeTitle} ${showName}` : episodeTitle
);
const searchUrl = `https://api.podcastindex.org/api/1.0/search/byterm?q=${q}&max=5`;
let candidate = null;
try {
const headers = buildPodcastIndexHeaders({ podcastIndexKey, podcastIndexSecret });
const r = await fetch(searchUrl, { headers });
const data = await r.json();
const feeds = Array.isArray(data?.feeds) ? data.feeds : [];
// Best-match heuristic: prefer a feed whose title fuzzy-matches the
// show name, fall back to the first result.
const norm = (s) => (s || "").toLowerCase().replace(/[^a-z0-9]/g, "");
const showKey = norm(showName);
candidate =
(showKey && feeds.find((f) => norm(f.title).includes(showKey))) ||
feeds[0] ||
null;
} catch (err) {
throw new URLResolveError(
"episode_not_found",
`PodcastIndex feed search failed: ${err?.message || err}`
);
}
if (!candidate || !candidate.id) {
throw new URLResolveError(
"spotify_no_rss",
`This Spotify episode "${episodeTitle}" doesn't appear in PodcastIndex. It may be a Spotify exclusive (Spotify Originals, Anchor-only shows). Paste the show's RSS feed URL instead, or use the YouTube version if available.`
);
}
// Pull the episode list for the matched feed and find the closest
// title match.
let episodes = [];
try {
const headers = buildPodcastIndexHeaders({ podcastIndexKey, podcastIndexSecret });
const r = await fetch(
`https://api.podcastindex.org/api/1.0/episodes/byfeedid?id=${candidate.id}&max=200`,
{ headers }
);
const data = await r.json();
episodes = Array.isArray(data?.items) ? data.items : [];
} catch (err) {
throw new URLResolveError(
"episode_not_found",
`PodcastIndex episode lookup failed: ${err?.message || err}`
);
}
const norm = (s) => (s || "").toLowerCase().replace(/[^a-z0-9]/g, "");
const targetKey = norm(episodeTitle);
const episode =
episodes.find((e) => norm(e.title) === targetKey) ||
episodes.find((e) => norm(e.title).includes(targetKey)) ||
episodes.find((e) => targetKey.includes(norm(e.title))) ||
null;
if (!episode || !episode.enclosureUrl) {
throw new URLResolveError(
"episode_not_found",
`Matched "${candidate.title}" in PodcastIndex but couldn't find an episode titled "${episodeTitle}". Episode may be too new for PodcastIndex's snapshot, or only available on Spotify.`
);
}
let uploadDate = "";
if (episode.datePublished) {
try {
const d = new Date(episode.datePublished * 1000);
if (!isNaN(d.getTime())) {
uploadDate = d.toISOString().slice(0, 10).replace(/-/g, "");
}
} catch {}
}
return {
source: "spotify",
audioUrl: episode.enclosureUrl,
episodeId: episode.guid || `spotify-${episode.id}`,
title: episode.title || episodeTitle,
podcastTitle: candidate.title || showName || "Unknown podcast",
uploadDate,
durationSec:
typeof episode.duration === "number" ? episode.duration : null,
feedUrl: candidate.url || null,
};
}
// ── Fountain ───────────────────────────────────────────────────────────
// Strategy: fetch the episode HTML, parse Open Graph + JSON-LD tags
// for the audio URL, title, podcast name, and upload date. Fountain
// serves the actual MP3 enclosure URL on og:audio so we don't need
// any PodcastIndex lookup or API key. Same shape as the Apple
// resolver returns so the downstream podcast pipeline doesn't have
// to branch.
//
// The og:title format is "Show • Episode title • Watch on Fountain".
// We split on " • " to separate show + episode; the trailing "Watch
// on Fountain" branding gets dropped.
//
// Fountain URLs encode the episode in a short opaque id at the URL
// path; we use that as our episodeId for history dedup.
export async function resolveFountainUrl(url) {
const m = url.match(/\/episode\/([A-Za-z0-9_-]+)/);
if (!m) {
throw new URLResolveError(
"fountain_lookup_failed",
"Fountain URL is missing the /episode/<id> path",
);
}
const shortId = m[1];
// Use global fetch directly (Node 18+) so we can send a UA header.
// fetchUrl() in util.js doesn't take options; we don't want to
// expand its signature just for this one caller. Fountain's SSR
// response includes the og:audio tag we need for ANY UA, but mimic
// a modern Safari to stay on the well-tested response path.
let html;
try {
const res = await fetch(url, {
headers: {
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 14_0) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15",
},
// Aggressive timeout — Fountain's page is small (~120KB) and
// we shouldn't hold up the summarize pipeline if their server
// hangs. AbortSignal.timeout is Node 18+, same baseline.
signal: AbortSignal.timeout(8000),
});
if (!res.ok) {
throw new URLResolveError(
"fountain_lookup_failed",
`Fountain returned HTTP ${res.status}`,
);
}
html = await res.text();
} catch (err) {
if (err instanceof URLResolveError) throw err;
throw new URLResolveError(
"fountain_lookup_failed",
`Couldn't reach Fountain: ${err?.message || err}`,
);
}
if (!html || typeof html !== "string") {
throw new URLResolveError(
"fountain_lookup_failed",
"Empty response from Fountain",
);
}
// Extract a meta tag's content by either property= or name=.
// Fountain uses property= for OG and name= for Twitter — we match
// both since users might paste links Twitter has re-fetched.
function metaContent(key) {
const re = new RegExp(
`<meta\\s+(?:property|name)="${key}"\\s+content="([^"]+)"`,
"i",
);
const found = html.match(re);
return found ? decodeHtmlEntities(found[1]) : null;
}
const audioUrl = metaContent("og:audio");
if (!audioUrl) {
throw new URLResolveError(
"fountain_lookup_failed",
"Fountain episode has no og:audio tag — the page may have changed format or the episode is video-only.",
);
}
const ogTitleRaw = metaContent("og:title") || "";
// og:title format: "Show • Episode • Watch on Fountain". Strip the
// trailing brand and split.
const titlePieces = ogTitleRaw
.replace(/\s*•\s*Watch on Fountain\s*$/i, "")
.split(/\s*•\s*/);
const podcastTitle = titlePieces[0] || "Podcast";
const episodeTitle = titlePieces.slice(1).join(" • ") || podcastTitle;
// JSON-LD on the page carries an ISO uploadDate. We don't parse
// the full JSON; a targeted regex is enough.
const uploadDateMatch = html.match(/"uploadDate":"([^"]+)"/);
const uploadDateRaw = uploadDateMatch ? uploadDateMatch[1] : "";
const uploadDate = isoToYYYYMMDD(uploadDateRaw);
// ISO-8601 duration (e.g. "PT2H7M27S") → seconds. Optional —
// returns null if absent.
const durationMatch = html.match(/"duration":"(PT[0-9HMS]+)"/);
const durationSec = durationMatch
? iso8601DurationToSeconds(durationMatch[1])
: null;
return {
source: "fountain",
audioUrl,
episodeId: `fountain:${shortId}`,
title: episodeTitle,
podcastTitle,
uploadDate,
durationSec,
feedUrl: null, // Fountain doesn't always expose the source RSS URL
};
}
// "2026-05-07T20:53:13.003Z" → "20260507". Returns empty string on
// unparseable input so the downstream pipeline treats it as unknown.
function isoToYYYYMMDD(iso) {
if (!iso) return "";
const d = new Date(iso);
if (Number.isNaN(d.getTime())) return "";
const y = d.getUTCFullYear();
const m = String(d.getUTCMonth() + 1).padStart(2, "0");
const day = String(d.getUTCDate()).padStart(2, "0");
return `${y}${m}${day}`;
}
// "PT2H7M27S" → 7647 seconds. Handles the subset of ISO-8601 that
// podcasts actually use (no fractional, no days).
function iso8601DurationToSeconds(s) {
if (typeof s !== "string") return null;
const m = s.match(/^PT(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?$/);
if (!m) return null;
const h = parseInt(m[1] || "0", 10);
const min = parseInt(m[2] || "0", 10);
const sec = parseInt(m[3] || "0", 10);
return h * 3600 + min * 60 + sec;
}
function decodeHtmlEntities(s) {
return String(s)
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'");
}
function buildPodcastIndexHeaders({ podcastIndexKey, podcastIndexSecret }) {
const date = Math.floor(Date.now() / 1000).toString();
const sig = crypto
.createHash("sha1")
.update(podcastIndexKey + podcastIndexSecret + date)
.digest("hex");
return {
"User-Agent": "Recap/1.0 (+https://github.com/keysat-xyz/recap)",
"X-Auth-Key": podcastIndexKey,
"X-Auth-Date": date,
Authorization: sig,
};
}
// Single entry point: takes any URL and a config object, returns
// either the normalized resolved shape (for apple/spotify) or null
// (for URLs we don't recognize as share links — caller passes those
// through to the existing youtube / rss path unchanged).
export async function resolveShareUrl(url, opts = {}) {
if (isApplePodcastUrl(url)) {
return resolveApplePodcastUrl(url);
}
if (isSpotifyUrl(url)) {
return resolveSpotifyUrl(url, opts);
}
if (isFountainUrl(url)) {
return resolveFountainUrl(url);
}
return null;
}
+115 -9
View File
@@ -14,12 +14,12 @@ export function sendEvent(res, event, data) {
}
// ── YouTube video-id extraction ─────────────────────────────────────────────
// Accepts watch URLs, youtu.be, /embed/, /v/, or a bare 11-char id.
// Returns null when no id can be extracted.
// Accepts watch URLs, youtu.be, /embed/, /v/, /live/, /shorts/, or a bare
// 11-char id. Returns null when no id can be extracted.
export function extractVideoId(url) {
if (!url) return null;
const patterns = [
/(?:youtube\.com\/watch\?v=|youtu\.be\/|youtube\.com\/embed\/|youtube\.com\/v\/)([a-zA-Z0-9_-]{11})/,
/(?:youtube\.com\/watch\?v=|youtu\.be\/|youtube\.com\/embed\/|youtube\.com\/v\/|youtube\.com\/live\/|youtube\.com\/shorts\/)([a-zA-Z0-9_-]{11})/,
/^([a-zA-Z0-9_-]{11})$/,
];
for (const p of patterns) {
@@ -113,18 +113,50 @@ export function fetchUrl(url) {
});
}
// ── Retry helper for transient Gemini API errors ────────────────────────────
// Retries on 503/429 and on common transient network errors. Linear backoff
// (delayMs * attempt). The optional `log` callback receives a one-line
// status message per retry — useful for streaming progress to a UI.
export async function retryGemini(fn, { retries = 3, delayMs = 3000, label = "Gemini call", log: logFn } = {}) {
// ── Retry helper for transient API errors ──────────────────────────────────
// Retries on 503/429/529 and on common transient network errors. Linear
// backoff (delayMs * attempt). The optional `log` callback receives a
// one-line status message per retry — useful for streaming progress to a
// UI. Provider-neutral: error shapes from @google/genai, @anthropic-ai/sdk,
// openai, and raw fetch all expose `.status` (or message text) we can match.
export async function retryAPI(fn, { retries = 3, delayMs = 3000, label = "API call", log: logFn } = {}) {
let lastErr;
for (let attempt = 1; attempt <= retries; attempt++) {
// Surface every attempt — including the first — so the user
// sees what's happening when a retry is in flight rather than
// a frozen-looking activity log between "failed, retrying in
// 5s" and the final outcome.
if (attempt > 1 && logFn) {
logFn(`Retrying ${label}... (attempt ${attempt}/${retries})`);
}
try {
return await fn();
} catch (err) {
// User-cancelled requests must not be retried — re-throw so the
// outer handler can treat it as a clean cancellation rather than
// letting the retry loop log noise and burn time.
if (err?.name === "AbortError" || /aborted|operation was aborted/i.test(err?.message || "")) {
throw err;
}
lastErr = err;
const msg = err?.message || String(err);
const status = err?.status || err?.httpStatusCode || 0;
const isRetryable = status === 503 || status === 429 || /overloaded|unavailable|capacity|high demand|rate limit|fetch failed|ECONNRESET|ETIMEDOUT|socket hang up|network/i.test(msg);
const isRetryable = status === 503 || status === 429 || status === 529 || /overloaded|unavailable|capacity|high demand|rate limit|fetch failed|ECONNRESET|ETIMEDOUT|socket hang up|network/i.test(msg);
// Dump every detail we can pry out of the error so generic
// messages like "500 status code (no body)" become debuggable
// server-side. Anthropic/OpenAI SDK errors expose .response,
// .body, .headers, .cause; Node stream errors expose .code.
const richDetail = {
status,
code: err?.code,
type: err?.type,
body: err?.body || err?.response?.body || err?.error,
cause: err?.cause?.message || err?.cause?.code || err?.cause,
};
console.error(
`[retryAPI] ${label} failed (attempt ${attempt}/${retries}, status=${status || "n/a"}): ${msg}`,
JSON.stringify(richDetail, (_k, v) => (typeof v === "bigint" ? v.toString() : v))
);
if (isRetryable && attempt < retries) {
const waitSec = (delayMs * attempt / 1000).toFixed(0);
if (logFn) logFn(`${label} failed (${status || "error"}), retrying in ${waitSec}s... (attempt ${attempt}/${retries})`);
@@ -134,4 +166,78 @@ export async function retryGemini(fn, { retries = 3, delayMs = 3000, label = "Ge
}
}
}
throw lastErr;
}
// Back-compat alias: pre-existing call sites used `retryGemini`. Keep
// the name working so this rename is non-breaking.
export const retryGemini = retryAPI;
// Split a plain-text transcript into synthetic sentence-based entries
// with interpolated timestamps. Used when a transcription provider
// returns just text (no per-segment timing) — e.g. NVIDIA Parakeet
// behind an OpenAI-compatible wrapper. Without this, the entire
// transcript lands in one entry at [0:00] and the analyzer can only
// produce a single section spanning the whole audio.
//
// Strategy:
// 1. Split on sentence terminators (. ! ?). Keep the punctuation.
// 2. If no terminators (very rare in real speech), fall back to
// 30-word chunks.
// 3. Distribute timestamps proportionally by character count —
// sentence N starts at (cum_chars_so_far / total_chars) *
// audio_duration. Not perfectly accurate, but good enough to
// let the analyzer carve out coherent topic sections.
export function synthesizeEntriesFromText(text, totalDurationSeconds) {
const t = (text || "").trim();
if (!t || !totalDurationSeconds || totalDurationSeconds <= 0) {
return [{ offset: 0, text: t, duration: totalDurationSeconds || 0 }];
}
// Sentence split — keep the terminator on each sentence.
const sentenceMatches = t.match(/[^.!?\n]+[.!?]+|[^.!?\n]+$/g) || [];
let chunks = sentenceMatches.map((s) => s.trim()).filter(Boolean);
// If we couldn't find sentence boundaries (unpunctuated transcript),
// fall back to fixed-size word chunks.
if (chunks.length <= 1) {
const words = t.split(/\s+/).filter(Boolean);
if (words.length <= 1) {
return [{ offset: 0, text: t, duration: totalDurationSeconds }];
}
const wordsPerChunk = 30;
chunks = [];
for (let i = 0; i < words.length; i += wordsPerChunk) {
chunks.push(words.slice(i, i + wordsPerChunk).join(" "));
}
}
// Coalesce extremely short sentences (single words like "Yeah." or
// "Right.") into the previous chunk so we don't end up with hundreds
// of useless 5-char entries.
const COALESCE_MIN_CHARS = 40;
const coalesced = [];
for (const c of chunks) {
if (coalesced.length > 0 && coalesced[coalesced.length - 1].length < COALESCE_MIN_CHARS) {
coalesced[coalesced.length - 1] = `${coalesced[coalesced.length - 1]} ${c}`.trim();
} else {
coalesced.push(c);
}
}
// Distribute timestamps proportionally by character length.
const totalChars = coalesced.reduce((sum, c) => sum + c.length, 0) || 1;
const entries = [];
let cumChars = 0;
for (const c of coalesced) {
const startRatio = cumChars / totalChars;
cumChars += c.length;
const endRatio = cumChars / totalChars;
entries.push({
offset: startRatio * totalDurationSeconds,
text: c,
duration: Math.max(0.1, (endRatio - startRatio) * totalDurationSeconds),
});
}
return entries;
}
+64
View File
@@ -0,0 +1,64 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// Multi-tenant mode (a.k.a. cloud mode) turns the self-hosted Recaps
// into a multi-user app served over the web. Adds email + magic-link
// auth, per-user libraries, per-user keysat licenses, and BTCPay-based
// subscriptions. The .s9pk defaults to single-mode (one operator, no
// auth) so installing Recaps doesn't surprise anyone — multi is the
// deliberate opt-in for operators who want to host the app for others.
//
// Prerequisites before enabling:
// 1. StartOS System SMTP must be configured + tested. Magic-link
// emails fail otherwise.
// 2. "Set Recaps public URL" action must be run with the ClearNet URL
// where your Recaps will live (e.g. https://recap.example.com),
// so the sign-in emails contain a working link.
// 3. A keysat issuer the relay trusts must be reachable, since each
// cloud user gets a keysat-minted license at signup.
//
// Switching back to single mode is non-destructive — multi-tenant data
// (the user DB at /data/recap.db) stays on disk and is restored if you
// flip back to multi later. The operator-owner's library lives under
// /data/history/owner/ in either mode.
const inputSpec = InputSpec.of({
mode: Value.select({
name: 'Mode',
description:
'Single = original self-hosted experience (one operator, no auth). Multi = cloud mode (email/magic-link auth, multi-user, BTCPay subscriptions). Switching takes effect on next service restart.',
default: 'single',
values: {
single: 'Single (self-hosted, no accounts)',
multi: 'Multi (cloud, email auth + subscriptions)',
},
}),
})
export const enableMultiTenantMode = sdk.Action.withInput(
'enable-multi-tenant-mode',
async ({ effects }) => ({
name: 'Enable Multi-Tenant Mode',
description:
'Switch Recaps between single-user (self-hosted) and multi-tenant (cloud) modes. Multi mode adds email-based sign-in, per-user libraries, and BTCPay subscriptions. Configure SMTP and the public URL before enabling.',
warning:
'Switching to multi mode requires StartOS SMTP to be configured and the Recaps public URL set. The service restarts on save.',
allowedStatuses: 'any',
group: 'Multi-Tenant',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return { mode: (config?.recap_mode as 'single' | 'multi') || 'single' }
},
async ({ effects, input }) => {
await configFile.merge(effects, { recap_mode: input.mode })
return null
},
)
+35
View File
@@ -1,7 +1,42 @@
import { sdk } from '../sdk'
import { setApiKey } from './setApiKey'
import { setLicense } from './setLicense'
import { setAdminPassword } from './setAdminPassword'
import { setAnthropicApiKey } from './setAnthropicApiKey'
import { setOpenAIApiKey } from './setOpenAIApiKey'
import { setOpenAICompatible } from './setOpenAICompatible'
import { setOllamaUrl } from './setOllamaUrl'
import { setWhisperEndpoint } from './setWhisperEndpoint'
import { setPodcastIndex } from './setPodcastIndex'
import { enableMultiTenantMode } from './enableMultiTenantMode'
import { setRecapPublicUrl } from './setRecapPublicUrl'
import { setTenantDefaultCredits } from './setTenantDefaultCredits'
import { setTrialCreditsPerVisitor } from './setTrialCreditsPerVisitor'
import { setTrialsPerIpPerDay } from './setTrialsPerIpPerDay'
import { setReplenishPeriod } from './setReplenishPeriod'
import { setRelayOperatorKey } from './setRelayOperatorKey'
// NOTE: setRelayUrl was removed in 0.2.34. The relay base URL is now
// hardcoded in server/relay-default.js and updated via Recap version
// releases — end users should never see or configure it.
//
// Multi-tenant cloud-mode actions (added 0.2.77) — these only matter
// when recap_mode === 'multi'. In single mode they're inert: the
// fields they manipulate exist in the config but nothing reads them.
export const actions = sdk.Actions.of()
.addAction(setApiKey)
.addAction(setAnthropicApiKey)
.addAction(setOpenAIApiKey)
.addAction(setOpenAICompatible)
.addAction(setOllamaUrl)
.addAction(setWhisperEndpoint)
.addAction(setPodcastIndex)
.addAction(setLicense)
.addAction(setAdminPassword)
.addAction(enableMultiTenantMode)
.addAction(setRecapPublicUrl)
.addAction(setTenantDefaultCredits)
.addAction(setTrialCreditsPerVisitor)
.addAction(setTrialsPerIpPerDay)
.addAction(setReplenishPeriod)
.addAction(setRelayOperatorKey)
+108
View File
@@ -0,0 +1,108 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
import { randomBytes, scryptSync } from 'crypto'
const { InputSpec, Value } = sdk
const SCRYPT_KEYLEN = 64
const inputSpec = InputSpec.of({
recap_admin_username: Value.text({
name: 'Admin Username',
description:
'Username required at the login screen. Defaults to "admin".',
required: true,
default: 'admin',
minLength: 1,
maxLength: 64,
}),
recap_admin_password: Value.text({
name: 'Admin Password',
description:
'Password required at the login screen. Must be at least 8 characters. Leave blank to disable the login gate.',
required: false,
default: null,
masked: true,
minLength: 0,
maxLength: 256,
}),
recap_admin_password_confirm: Value.text({
name: 'Confirm Password',
description: 'Re-enter the password to confirm.',
required: false,
default: null,
masked: true,
minLength: 0,
maxLength: 256,
}),
})
export const setAdminPassword = sdk.Action.withInput(
'set-admin-password',
async ({ effects }) => ({
name: 'Set Admin Password',
description:
'Set a username and password that gate the Recaps web UI. Anyone visiting the site (LAN or clearnet) must log in before reaching the activation screen. Leave the password blank to disable the gate.',
warning: null,
allowedStatuses: 'any',
group: 'Setup',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return {
recap_admin_username: config?.recap_admin_username || 'admin',
recap_admin_password: undefined,
recap_admin_password_confirm: undefined,
}
},
async ({ effects, input }) => {
const username = (input.recap_admin_username || '').trim()
const password = input.recap_admin_password || ''
const confirm = input.recap_admin_password_confirm || ''
if (!username) {
throw new Error('Username is required.')
}
if (password === '' && confirm === '') {
// Disable the gate: clear hash + salt, keep username for next time.
await configFile.merge(effects, {
recap_admin_username: username,
recap_admin_password_hash: '',
recap_admin_password_salt: '',
})
return null
}
if (password !== confirm) {
throw new Error('Password and confirmation do not match.')
}
if (password.length < 8) {
throw new Error('Password must be at least 8 characters.')
}
const salt = randomBytes(16).toString('hex')
const hash = scryptSync(password, salt, SCRYPT_KEYLEN).toString('hex')
const existing = await configFile.read().once()
const sessionSecret =
existing?.recap_admin_session_secret && existing.recap_admin_session_secret.length > 0
? existing.recap_admin_session_secret
: randomBytes(32).toString('hex')
await configFile.merge(effects, {
recap_admin_username: username,
recap_admin_password_hash: hash,
recap_admin_password_salt: salt,
recap_admin_session_secret: sessionSecret,
})
return null
},
)
+45
View File
@@ -0,0 +1,45 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
const inputSpec = InputSpec.of({
anthropic_api_key: Value.text({
name: 'Anthropic API Key',
description:
'Your Anthropic (Claude) API key. Get one at console.anthropic.com. Required to use Claude models for topic analysis.',
required: true,
default: null,
masked: true,
minLength: 1,
maxLength: 256,
}),
})
export const setAnthropicApiKey = sdk.Action.withInput(
'set-anthropic-api-key',
async ({ effects }) => ({
name: 'Set Anthropic API Key',
description:
'Configure your Anthropic (Claude) API key for topic analysis. Claude does not transcribe audio — pair it with Gemini or OpenAI Whisper for transcription.',
warning: null,
allowedStatuses: 'any',
group: 'AI Providers',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return { anthropic_api_key: config?.anthropic_api_key || undefined }
},
async ({ effects, input }) => {
await configFile.merge(effects, {
anthropic_api_key: (input.anthropic_api_key || '').trim(),
})
return null
},
)
+1 -1
View File
@@ -25,7 +25,7 @@ export const setApiKey = sdk.Action.withInput(
'Configure your Google Gemini API key for transcription and analysis',
warning: null,
allowedStatuses: 'any',
group: null,
group: 'AI Providers',
visibility: 'enabled',
}),
+5 -5
View File
@@ -5,9 +5,9 @@ const { InputSpec, Value } = sdk
const inputSpec = InputSpec.of({
recap_license_key: Value.text({
name: 'Recap License Key',
name: 'Recaps License Key',
description:
'Paste your Recap license key here. Keys start with "LIC1-..." — get one from your Recap seller. (Keys are also accepted via the web UI activation screen.)',
'Paste your Recaps license key here. Keys start with "LIC1-..." — get one from your Recaps seller. (Keys are also accepted via the web UI activation screen.)',
required: true,
default: null,
masked: true,
@@ -26,12 +26,12 @@ export const setLicense = sdk.Action.withInput(
'set-license',
async ({ effects }) => ({
name: 'Set Recap License',
name: 'Set Recaps License',
description:
'Activate a Recap license to unlock paid features (saved library, channel & podcast subscriptions, auto-queue).',
'Activate a Recaps license to unlock paid features (channel & podcast subscriptions, auto-queue, and a monthly allotment of relay credits).',
warning: null,
allowedStatuses: 'any',
group: null,
group: 'Setup',
visibility: 'enabled',
}),
+81
View File
@@ -0,0 +1,81 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// Standard Ollama port. Hardcoded because Ollama upstream uses 11434
// universally — its StartOS package preserves this. If a future
// release changes the port we can swap to a runtime
// sdk.serviceInterface.get(...) lookup against ollama's exposed
// interface, but for now hardcode + override-on-mismatch is simpler
// and avoids a guess at the interface ID.
const OLLAMA_DEFAULT_PORT = 11434
const inputSpec = InputSpec.of({
ollama_base_url: Value.text({
name: 'Ollama Base URL',
description:
'URL of your Ollama server. If you have the Ollama StartOS package installed on this server, this field is pre-populated automatically. Override only if you want to point at a different Ollama instance (e.g. on another machine: http://192.168.1.10:11434).',
required: false,
default: 'http://localhost:11434',
minLength: 0,
maxLength: 256,
patterns: [
{
regex: '^(https?://.+)?$',
description: 'Must be empty or start with http:// or https://',
},
],
}),
})
// Best-effort detection of an Ollama instance running on this same
// StartOS server. StartOS exposes every package on its own internal
// `<package-id>.startos` hostname, reachable from any other package's
// container without explicit networking config (per the Service
// Packaging docs). Returns the URL when ollama is installed, null
// otherwise.
async function detectStartOsOllamaUrl(effects: any): Promise<string | null> {
try {
const check = await sdk.checkDependencies(effects, ['ollama'])
if (!check.installedSatisfied('ollama')) return null
return `http://ollama.startos:${OLLAMA_DEFAULT_PORT}`
} catch {
return null
}
}
export const setOllamaUrl = sdk.Action.withInput(
'set-ollama-url',
async ({ effects }) => ({
name: 'Set Ollama Server URL',
description:
'Configure where to reach a local Ollama server for topic analysis. No API key required (Ollama runs locally). Does not transcribe audio. Auto-pre-populates if the Ollama StartOS package is installed on this server.',
warning: null,
allowedStatuses: 'any',
group: 'AI Providers',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
// If the user has already set a value, respect it — don't
// overwrite a manual override on every action open.
if (config?.ollama_base_url) {
return { ollama_base_url: config.ollama_base_url }
}
const auto = await detectStartOsOllamaUrl(effects)
if (auto) return { ollama_base_url: auto }
return { ollama_base_url: 'http://localhost:11434' }
},
async ({ effects, input }) => {
await configFile.merge(effects, {
ollama_base_url: (input.ollama_base_url || '').trim(),
})
return null
},
)
+45
View File
@@ -0,0 +1,45 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
const inputSpec = InputSpec.of({
openai_api_key: Value.text({
name: 'OpenAI API Key',
description:
'Your OpenAI API key. Get one at platform.openai.com. Used for both topic analysis (GPT models) and audio transcription (Whisper).',
required: true,
default: null,
masked: true,
minLength: 1,
maxLength: 256,
}),
})
export const setOpenAIApiKey = sdk.Action.withInput(
'set-openai-api-key',
async ({ effects }) => ({
name: 'Set OpenAI API Key',
description:
'Configure your OpenAI API key. Enables GPT models for topic analysis and Whisper for audio transcription.',
warning: null,
allowedStatuses: 'any',
group: 'AI Providers',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return { openai_api_key: config?.openai_api_key || undefined }
},
async ({ effects, input }) => {
await configFile.merge(effects, {
openai_api_key: (input.openai_api_key || '').trim(),
})
return null
},
)
+64
View File
@@ -0,0 +1,64 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
const inputSpec = InputSpec.of({
openai_compatible_base_url: Value.text({
name: 'Base URL',
description:
'OpenAI-compatible API endpoint. Examples: https://api.deepseek.com/v1, https://api.together.xyz/v1, https://api.groq.com/openai/v1. Must include the /v1 (or equivalent) path segment.',
required: true,
default: null,
minLength: 1,
maxLength: 512,
patterns: [
{
regex: '^https?://.+',
description: 'Must start with http:// or https://',
},
],
}),
openai_compatible_api_key: Value.text({
name: 'API Key',
description:
'API key for the OpenAI-compatible backend. Some self-hosted backends accept any non-empty value — leave blank for those.',
required: false,
default: null,
masked: true,
minLength: 0,
maxLength: 256,
}),
})
export const setOpenAICompatible = sdk.Action.withInput(
'set-openai-compatible',
async ({ effects }) => ({
name: 'Set OpenAI-Compatible Backend',
description:
'Point Recaps at any OpenAI-compatible chat-completions API: DeepSeek, Together, Groq, Fireworks, self-hosted vLLM, etc. Used for topic analysis only — does not transcribe audio.',
warning: null,
allowedStatuses: 'any',
group: 'AI Providers',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return {
openai_compatible_base_url: config?.openai_compatible_base_url || undefined,
openai_compatible_api_key: config?.openai_compatible_api_key || undefined,
}
},
async ({ effects, input }) => {
await configFile.merge(effects, {
openai_compatible_base_url: (input.openai_compatible_base_url || '').trim(),
openai_compatible_api_key: (input.openai_compatible_api_key || '').trim(),
})
return null
},
)
+59
View File
@@ -0,0 +1,59 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
const inputSpec = InputSpec.of({
podcastindex_api_key: Value.text({
name: 'PodcastIndex API Key',
description:
'First of the two credentials shown on your PodcastIndex account page after free signup at api.podcastindex.org. Both Key AND Secret are required for Spotify link resolution — paste the SECRET in the field below. Apple Podcasts and Fountain links work without any PodcastIndex auth.',
required: false,
default: null,
masked: true,
minLength: 0,
maxLength: 256,
}),
podcastindex_api_secret: Value.text({
name: 'PodcastIndex API Secret',
description:
'Second of the two credentials shown on your PodcastIndex account page (right next to the API Key — sometimes labeled "API Secret" or "auth secret"). REQUIRED alongside the API Key for the Spotify lookup to work — leaving this blank is the most common reason Spotify URLs fail with "PodcastIndex unconfigured."',
required: false,
default: null,
masked: true,
minLength: 0,
maxLength: 256,
}),
})
export const setPodcastIndex = sdk.Action.withInput(
'set-podcastindex',
async ({ effects }) => ({
name: 'Set PodcastIndex Credentials',
description:
'Configure PodcastIndex API credentials so Recaps can resolve Spotify episode links. Optional — Apple Podcasts links work without this. Sign up free at api.podcastindex.org.',
warning: null,
allowedStatuses: 'any',
group: 'External Services',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return {
podcastindex_api_key: config?.podcastindex_api_key || undefined,
podcastindex_api_secret: config?.podcastindex_api_secret || undefined,
}
},
async ({ effects, input }) => {
await configFile.merge(effects, {
podcastindex_api_key: input.podcastindex_api_key || '',
podcastindex_api_secret: input.podcastindex_api_secret || '',
})
return null
},
)
+59
View File
@@ -0,0 +1,59 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// The Recaps public URL is the ClearNet URL where users access the app
// — typically a domain you've pointed at Start Tunnel. Magic-link
// sign-in emails interpolate this URL into the verification link:
//
// Click here to sign in: https://recap.example.com/auth/verify?token=...
//
// Without this set, magic-link emails contain a link to localhost or
// to the StartOS internal hostname, neither of which work for a user
// reading the email on a different device.
//
// Only meaningful when recap_mode === 'multi'. In single mode the
// value is ignored — there's no magic-link flow.
const inputSpec = InputSpec.of({
public_url: Value.text({
name: 'Public URL',
description:
'Full URL where users reach your Recaps (e.g. https://recapapp.xyz). Include the https:// prefix. Used to build sign-in links in magic-link emails. Must be reachable from the public internet for users to receive a working link.',
required: true,
default: null,
placeholder: 'https://recapapp.xyz',
masked: false,
minLength: 8,
maxLength: 256,
}),
})
export const setRecapPublicUrl = sdk.Action.withInput(
'set-recap-public-url',
async ({ effects }) => ({
name: 'Set Recaps Public URL',
description:
'Set the ClearNet URL where users will access your Recaps. Used in magic-link sign-in emails. Required for multi-tenant mode.',
warning: null,
allowedStatuses: 'any',
group: 'Multi-Tenant',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return { public_url: config?.recap_public_url || undefined }
},
async ({ effects, input }) => {
// Strip trailing slash for canonical form — the auth-link builder
// concatenates "/auth/verify?token=..." without checking.
const url = (input.public_url || '').trim().replace(/\/$/, '')
await configFile.merge(effects, { recap_public_url: url })
return null
},
)
+59
View File
@@ -0,0 +1,59 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// The "operator key" is the shared secret that authenticates THIS Recaps
// server to the Recap Relay for the core-decoupling cloud path. With it
// set, the server vouches for its signed-in Pro/Max users by their Recaps
// account-id (X-Recap-User-Id) — the relay owns their subscription tier,
// keyed by that id, with NO per-user Keysat license involved.
//
// It MUST exactly match the value set on the relay (its
// relay_cloud_operator_key, via the relay's own "Set Cloud Operator Key"
// action). If the two don't match, the relay rejects the cloud calls and
// paid users silently fall back to the operator's shared relay pool.
//
// Only meaningful when recap_mode === 'multi' (the cloud deployment). In
// single mode there are no per-user accounts to vouch for, so the value
// is ignored.
const inputSpec = InputSpec.of({
operator_key: Value.text({
name: 'Relay Operator Key',
description:
'Shared secret that authenticates this Recaps server to the Recap Relay. Must EXACTLY match the relay\'s "Cloud Operator Key". Generate a long random string (e.g. `openssl rand -hex 32`) and set the same value on both. Server-side only — never shown to users.',
required: true,
default: null,
placeholder: 'paste the same key set on the relay',
masked: true,
minLength: 16,
maxLength: 256,
}),
})
export const setRelayOperatorKey = sdk.Action.withInput(
'set-relay-operator-key',
async ({ effects }) => ({
name: 'Set Relay Operator Key',
description:
'Set the shared operator key that lets this Recaps server vouch for its Pro/Max users to the Recap Relay by account-id (core-decoupling). Must match the key set on the relay. Multi-tenant mode only.',
warning: null,
allowedStatuses: 'any',
group: 'Multi-Tenant',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return { operator_key: config?.recap_relay_operator_key || undefined }
},
async ({ effects, input }) => {
const key = (input.operator_key || '').trim()
await configFile.merge(effects, { recap_relay_operator_key: key })
return null
},
)
+73
View File
@@ -0,0 +1,73 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// How often to refresh tenants' replenishable credit bucket. The
// bucket size itself is set via "Set Default Tenant Credits" — this
// action just controls WHEN it gets refilled.
//
// Refresh semantics: replenish_balance is RESET to
// tenant_default_credits at each anniversary boundary. Any leftover
// from the previous period is forfeit (use-it-or-lose-it). Purchased
// credits + admin grants live in a SEPARATE bucket
// (tenant_credits.purchased_balance) that's never wiped by refills.
//
// Spend order is replenish first, then purchased — so a tenant
// burns through the refillable bucket each period before touching
// their permanent balance.
//
// Anniversary alignment: refills are anchored to each tenant's
// individual last_replenish_at timestamp (set when they signed up,
// or when the operator first switched this action on). A user who
// signed up at 3:17pm gets their daily refresh at 3:17pm each day,
// not at calendar midnight.
const inputSpec = InputSpec.of({
period: Value.select({
name: 'Replenishment period',
description:
'How often each tenant\'s replenishable credit bucket gets refilled to the configured default. Set to "off" for a one-time signup grant (Grant\'s use case — tenants are paying customers and don\'t get free daily refills). Set to daily/weekly/monthly for a free-tier-with-daily-allowance model.',
default: 'off',
values: {
off: "Off (one-time signup grant only)",
daily: 'Daily (every 24 hours)',
weekly: 'Weekly (every 7 days)',
monthly: 'Monthly (calendar month)',
},
}),
})
export const setReplenishPeriod = sdk.Action.withInput(
'set-replenish-period',
async ({ effects }) => ({
name: 'Set Tenant Credit Replenishment',
description:
"How often a tenant's free credit bucket refills (uses Set Default Tenant Credits as the refill amount). Off = no replenishment; their initial signup grant is one-time. Daily/Weekly/Monthly = anniversary-aligned refill of the replenishable bucket. Purchased credits never expire.",
warning: null,
allowedStatuses: 'any',
group: 'Multi-Tenant',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return {
period:
(config?.tenant_credit_replenish_period as
| 'off'
| 'daily'
| 'weekly'
| 'monthly') || 'off',
}
},
async ({ effects, input }) => {
await configFile.merge(effects, {
tenant_credit_replenish_period: input.period,
})
return null
},
)
@@ -0,0 +1,56 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// When a self-hosted Recaps is running in multi-tenant mode, the
// operator can invite family members / guests to sign up on their
// domain. New tenants start with this many credits drawn from the
// operator's relay credit pool (i.e., the operator pays for them).
//
// Default 5 is generous enough to try, tight enough that running a
// public sign-up doesn't immediately drain the operator. Set to 0 to
// disable auto-allocation (operator must explicitly grant credits to
// each tenant before they can summarize).
//
// Doesn't apply to Grant's canonical cloud Recaps — cloud users have
// their own keysat licenses + relay-side credit pools, so the relay's
// per-tier quotas handle their initial credit allowance directly.
const inputSpec = InputSpec.of({
credits: Value.number({
name: 'Default credits per new tenant',
description:
'When a new user signs up on this multi-tenant Recaps, they start with this many credits. Charged against your relay credit pool when they summarize. Set to 0 to require manual approval before any tenant can summarize.',
required: true,
default: 5,
integer: true,
min: 0,
max: 1000,
}),
})
export const setTenantDefaultCredits = sdk.Action.withInput(
'set-tenant-default-credits',
async ({ effects }) => ({
name: 'Set Default Tenant Credits',
description:
"How many credits new sign-ups get for free on your multi-tenant Recaps. Charged against your relay credit pool. Default: 5.",
warning: null,
allowedStatuses: 'any',
group: 'Multi-Tenant',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return { credits: config?.tenant_default_credits ?? 5 }
},
async ({ effects, input }) => {
await configFile.merge(effects, { tenant_default_credits: input.credits })
return null
},
)
@@ -0,0 +1,60 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// Anonymous-trial allowance per first-time visitor. Multi-tenant mode
// only. When a visitor lands on the public Recaps URL without a session
// cookie and submits a YouTube URL, the server issues them a
// recap_anon_trial cookie with this many free summaries — no email,
// no signup, no friction. After they're spent, the UI nudges them to
// create an account for more credits.
//
// Trial summaries draw from the OPERATOR's relay credit pool, so this
// number times the visitor volume sets the floor on your sample-cost
// exposure. Tune downward if you see abuse, upward if you want a more
// generous activation funnel. Set to 0 to disable trials entirely
// (visitors immediately hit the sign-up gate).
//
// Defaults to 1 — enough to demo the value prop, tight enough that
// scripted-signup abuse doesn't drain the pool fast.
const inputSpec = InputSpec.of({
credits: Value.number({
name: 'Trial credits per anonymous visitor',
description:
'How many free summaries an unauthenticated visitor gets before being asked to sign up. Charged against your relay credit pool. Set to 0 to disable trials (immediate sign-up gate).',
required: true,
default: 1,
integer: true,
min: 0,
max: 5,
}),
})
export const setTrialCreditsPerVisitor = sdk.Action.withInput(
'set-trial-credits-per-visitor',
async ({ effects }) => ({
name: 'Set Trial Credits per Visitor',
description:
"How many free summaries anonymous visitors get on your multi-tenant Recaps before being prompted to sign up. Default: 1. Charged against your relay credit pool.",
warning: null,
allowedStatuses: 'any',
group: 'Multi-Tenant',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return { credits: config?.trial_credits_per_visitor ?? 1 }
},
async ({ effects, input }) => {
await configFile.merge(effects, {
trial_credits_per_visitor: input.credits,
})
return null
},
)
+79
View File
@@ -0,0 +1,79 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
// Lifetime cap on how many distinct anonymous trial cookies one IP
// address can mint, FOR THE LIFETIME OF THE INSTALL — not a rolling
// daily window. Was previously per-day; switched in 0.2.84 so a user
// who clears cookies can't simply wait 24h and replay the trial.
//
// Anti-abuse model:
// - Each minted cookie carries trial_credits_per_visitor credits
// - Each IP can mint at most `trials_per_ip_lifetime` cookies, ever
// - Combined effect: an IP's total free credits =
// trial_credits_per_visitor × trials_per_ip_lifetime
// - Once spent, the visitor must sign up (which grants
// tenant_default_credits) or pay for more
//
// IP rotation via VPN / proxy pool defeats this, same as before. The
// goal isn't to be unbypassable — it's to raise the floor for casual
// scripted abuse and give the operator forensic data (IP + UA logged
// on every trial) to manually ban any sophisticated abuser.
//
// Defaults to 5 — generous enough that a family on one NAT all get
// trials, tight enough that 50 trials/IP from one address looks like
// scripted abuse in the admin dashboard.
//
// Legacy field name `trials_per_ip_per_day` is preserved on the
// config schema as a read-only alias so installs upgrading from
// 0.2.770.2.83 don't lose their existing setting.
const inputSpec = InputSpec.of({
limit: Value.number({
name: 'Max trial cookies per IP (lifetime)',
description:
'How many anonymous trial cookies can be issued from a single IP, FOR THE LIFE OF THIS INSTALL. Not a rolling daily window — once the IP hits this cap, no more trial cookies from that address ever. Higher = friendlier to shared networks (offices, families). Lower = tighter against scripted abuse + cookie-clearing replay.',
required: true,
default: 5,
integer: true,
min: 1,
max: 50,
}),
})
export const setTrialsPerIpPerDay = sdk.Action.withInput(
'set-trials-per-ip-per-day',
async ({ effects }) => ({
name: 'Set Trial Cookies per IP (Lifetime)',
description:
'Anti-abuse cap on how many trial cookies a single IP can mint over the life of this install. Default: 5. Was per-day in 0.2.770.2.83 and is now lifetime — see release notes for 0.2.84.',
warning: null,
allowedStatuses: 'any',
group: 'Multi-Tenant',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
// Prefer the new field; fall back to the legacy per_day key for
// operators whose StartOS-managed config still has the old name.
const current =
config?.trials_per_ip_lifetime ??
config?.trials_per_ip_per_day ??
5
return { limit: current }
},
async ({ effects, input }) => {
// Write to BOTH keys so any code path still reading the legacy name
// gets a sane value too. anon-trial.js prefers the new key.
await configFile.merge(effects, {
trials_per_ip_lifetime: input.limit,
trials_per_ip_per_day: input.limit,
})
return null
},
)
+64
View File
@@ -0,0 +1,64 @@
import { sdk } from '../sdk'
import { configFile } from '../file-models/config.json'
const { InputSpec, Value } = sdk
const inputSpec = InputSpec.of({
whisper_base_url: Value.text({
name: 'Whisper Base URL',
description:
"URL of your Whisper-compatible transcription server. Example: http://whisper.startos:8000 for a local StartOS package, or http://192.168.1.10:9000 for whisper.cpp running on another machine on your LAN. The endpoint must implement OpenAI's /v1/audio/transcriptions wire format.",
required: true,
default: null,
minLength: 1,
maxLength: 512,
patterns: [
{
regex: '^https?://.+',
description: 'Must start with http:// or https://',
},
],
}),
whisper_api_key: Value.text({
name: 'API Key (optional)',
description:
'API key for the Whisper backend. Most self-hosted Whisper servers (whisper.cpp HTTP server, faster-whisper-server) accept any value or none at all — leave blank for those. Cloud Whisper providers (Groq, etc.) require a real key.',
required: false,
default: null,
masked: true,
minLength: 0,
maxLength: 256,
}),
})
export const setWhisperEndpoint = sdk.Action.withInput(
'set-whisper-endpoint',
async ({ effects }) => ({
name: 'Set Whisper Endpoint',
description:
'Point Recaps at a self-hosted or third-party Whisper transcription server (whisper.cpp, faster-whisper-server, Groq, etc.). Free alternative to OpenAI Whisper API or Gemini multimodal transcription.',
warning: null,
allowedStatuses: 'any',
group: 'AI Providers',
visibility: 'enabled',
}),
inputSpec,
async ({ effects }) => {
const config = await configFile.read().once()
return {
whisper_base_url: config?.whisper_base_url || undefined,
whisper_api_key: config?.whisper_api_key || undefined,
}
},
async ({ effects, input }) => {
await configFile.merge(effects, {
whisper_base_url: (input.whisper_base_url || '').trim(),
whisper_api_key: (input.whisper_api_key || '').trim(),
})
return null
},
)
+7 -1
View File
@@ -1,6 +1,12 @@
import { sdk } from './sdk'
// Recap has no dependencies on other StartOS services.
// Recap declares Ollama as an OPTIONAL dependency in the manifest.
// We do not return it here because we don't want to enforce a runtime
// requirement on it — Recap runs fine using cloud providers
// (Gemini/Anthropic/OpenAI) when Ollama is not installed. The optional
// declaration in the manifest is what surfaces it as a suggested
// install on the Marketplace; this empty result keeps it from blocking
// startup.
export const setDependencies = sdk.setupDependencies(async ({ effects }) => {
return {}
})
+94
View File
@@ -11,6 +11,100 @@ export const configFile = FileHelper.json(
},
z.object({
gemini_api_key: z.string().default(''),
anthropic_api_key: z.string().default(''),
openai_api_key: z.string().default(''),
openai_compatible_base_url: z.string().default(''),
openai_compatible_api_key: z.string().default(''),
ollama_base_url: z.string().default(''),
whisper_base_url: z.string().default(''),
whisper_api_key: z.string().default(''),
// NOTE: relay_base_url was removed in 0.2.34. The relay endpoint
// is hardcoded in server/relay-default.js and updated via Recap
// version releases — never exposed to end users.
recap_license_key: z.string().default(''),
recap_admin_username: z.string().default(''),
recap_admin_password_hash: z.string().default(''),
recap_admin_password_salt: z.string().default(''),
recap_admin_session_secret: z.string().default(''),
// PodcastIndex credentials — used to resolve Spotify share URLs to
// their underlying RSS audio enclosure. Free tier signup at
// api.podcastindex.org. Apple Podcasts URLs resolve without auth.
podcastindex_api_key: z.string().default(''),
podcastindex_api_secret: z.string().default(''),
// ── Multi-tenant cloud-mode fields (added 0.2.77) ──
// recap_mode: "single" → existing self-hosted, one operator, no auth
// "multi" → cloud mode with email + magic-link auth,
// per-user library, BTCPay subscriptions
// The .s9pk defaults to single. Operators flip to multi via the
// "Enable multi-tenant mode" StartOS action.
recap_mode: z.enum(['single', 'multi']).default('single'),
// Public URL used in magic-link sign-in emails. Must be the
// ClearNet URL pointing at the Recap UI (typically a domain
// routed through Start Tunnel). Without this set, magic-link
// emails won't have a working sign-in link.
recap_public_url: z.string().default(''),
// Shared "operator key" for the core-decoupling cloud path. The secret
// that authenticates THIS Recap server to the Recap Relay so it can
// vouch for its signed-in users by account-id (X-Recap-User-Id) instead
// of attaching a per-user Keysat license. Must EXACTLY match the relay's
// own relay_cloud_operator_key. Empty = the cloud user-id path is
// disabled and paid users fall back to the operator's relay pool.
// Set via the "Set Relay Operator Key" StartOS action. Server-side
// only — never sent to the browser. Picked up live by the config poll
// (no restart needed), same as the provider API keys.
recap_relay_operator_key: z.string().default(''),
// Per-tenant default credit allowance — only used when running in
// multi mode on a self-hosted operator's StartOS server. New
// tenants (family members who sign up on the operator's domain)
// start with this many credits. The operator's relay-credit pool
// is debited as their tenants summarize. Doesn't apply to the
// canonical cloud deployment because cloud users have their own
// keysat licenses + relay-side credit pools.
tenant_default_credits: z.number().int().nonnegative().default(5),
// How often a tenant's "replenishable" credit bucket is refilled
// to tenant_default_credits.
// "off" — no refill; the signup-grant is one-time
// "daily" — anniversary-aligned 24h
// "weekly" — anniversary-aligned 7d
// "monthly" — anniversary-aligned calendar month
// Anniversary-aligned = anchored to each tenant's last_replenish_at
// (set when they signed up or when the operator first turned the
// period on). Purchased credits + admin grants are persisted in
// tenant_credits.purchased_balance and NEVER affected by refills —
// only the replenish_balance bucket gets reset to the configured N.
tenant_credit_replenish_period: z
.enum(["off", "daily", "weekly", "monthly"])
.default("off"),
// Anonymous-trial knobs (multi-mode only). Visitors who land on
// recapapp.xyz without an account get N free summaries gated by a
// browser cookie before being prompted to sign up. Set
// trial_credits_per_visitor=0 to disable trials (return to an
// auth-wall landing). trials_per_ip_per_day caps how many distinct
// trial cookies one IP can mint in 24h — anti-script-abuse floor.
trial_credits_per_visitor: z.number().int().nonnegative().default(1),
// Lifetime cap on the number of distinct trial cookies one IP can
// mint. Was previously rolling-24h; switched to lifetime so a user
// who clears cookies can't replay the trial every day. The legacy
// field name `trials_per_ip_per_day` is preserved below as a
// read-only alias so installs that already have it set don't lose
// their value during the rename — anon-trial.js prefers
// _lifetime if set, falls back to _per_day if not.
trials_per_ip_lifetime: z.number().int().positive().default(5),
// Legacy alias — read for backward compatibility with v0.2.770.2.83
// installs that wrote the per-day variant. Will be removed once
// all known installs have re-saved their config under the new key.
trials_per_ip_per_day: z.number().int().positive().default(5),
// ── SMTP — synced from StartOS System SMTP via the SDK ──
// main.ts subscribes to effects.getSystemSmtp() and writes these
// fields here whenever the operator changes the system SMTP. The
// server reads them through the same file-poll as everything else
// and (re)builds its nodemailer transport. NEVER edit these
// fields directly — they get overwritten on the next sync.
smtp_host: z.string().default(''),
smtp_port: z.number().int().default(0),
smtp_security: z.enum(['starttls', 'tls']).default('tls'),
smtp_username: z.string().default(''),
smtp_password: z.string().default(''),
smtp_from: z.string().default(''),
}),
)
+67
View File
@@ -1,10 +1,74 @@
import { i18n } from './i18n'
import { sdk } from './sdk'
import { uiPort } from './utils'
import { configFile } from './file-models/config.json'
// ── System SMTP → config.json sync ───────────────────────────────────────
// The StartOS SDK exposes shared SMTP credentials via effects.getSystemSmtp.
// Passing a callback subscribes us — StartOS re-invokes the callback when
// the operator changes their System → SMTP settings. We mirror the values
// into our own config.json so the server (which polls the JSON via
// server/config.js) can pick them up without needing direct SDK access.
//
// `password` can be null in the StartOS payload — coerce to empty string
// so the Zod schema (z.string()) accepts it. The server treats "" the
// same as "auth disabled".
//
// Only meaningful in multi-tenant cloud mode (recap_mode === 'multi'),
// where the server sends magic-link sign-in emails. In single mode the
// fields exist in config.json but nothing reads them.
async function syncSystemSmtpToConfig(effects: any) {
try {
const smtp = await effects.getSystemSmtp({
callback: () => {
// Re-fire ourselves on change. Effects callbacks fire on every
// mutation of the underlying value, so this stays in sync.
syncSystemSmtpToConfig(effects).catch((err) => {
console.warn('[smtp] sync callback failed:', err)
})
},
})
if (!smtp) {
// No System SMTP configured — clear stale values so the server
// doesn't try to send mail through a transport we no longer have
// credentials for.
await configFile.merge(effects, {
smtp_host: '',
smtp_port: 0,
smtp_security: 'tls',
smtp_username: '',
smtp_password: '',
smtp_from: '',
})
return
}
await configFile.merge(effects, {
smtp_host: smtp.host || '',
smtp_port: smtp.port || 0,
smtp_security: smtp.security || 'tls',
smtp_username: smtp.username || '',
smtp_password: smtp.password || '',
smtp_from: smtp.from || '',
})
} catch (err) {
console.warn('[smtp] initial sync failed:', err)
}
}
export const main = sdk.setupMain(async ({ effects }) => {
console.info(i18n('Starting Recap...'))
// Subscribe to System SMTP changes before the daemon starts so the
// server boots with credentials already in config.json (no race where
// a magic-link request arrives before the first sync).
await syncSystemSmtpToConfig(effects)
// Read current config to determine which mode to boot in. The
// RECAP_MODE env var is passed to the container so the Node server
// can branch on it without re-reading the config file at startup.
const cfg = await configFile.read().once()
const recapMode = cfg?.recap_mode === 'multi' ? 'multi' : 'single'
return sdk.Daemons.of(effects).addDaemon('primary', {
subcontainer: await sdk.SubContainer.of(
effects,
@@ -23,6 +87,9 @@ export const main = sdk.setupMain(async ({ effects }) => {
'--',
'/usr/local/bin/docker_entrypoint.sh',
],
env: {
RECAP_MODE: recapMode,
},
},
ready: {
display: i18n('Web Interface'),
+20 -10
View File
@@ -1,22 +1,32 @@
export const short = {
en_US:
'Turn videos and podcasts into structured topic summaries with clickable timestamps.',
'Turn YouTube videos and podcasts into structured topic summaries with clickable timestamps. Free relay credits, bring-your-own API key, or self-host the AI.',
}
export const long = {
en_US:
'Recap downloads audio from YouTube videos and podcast RSS feeds, ' +
'transcribes them using Google Gemini, and produces structured topic-by-topic ' +
'summaries with timestamps. Features include channel and podcast subscriptions ' +
'with automatic new episode detection, a background processing queue with ' +
'configurable delays, auto-download per subscription, organized history with ' +
'folders, and a responsive web interface. ' +
'Requires a Google Gemini API key (free tier available at aistudio.google.com/apikey).',
'Recaps downloads audio from YouTube videos and podcast RSS feeds, transcribes them, ' +
'and produces structured topic-by-topic summaries with clickable timestamps. ' +
'Pluggable AI provider system: pair any supported transcription provider with any ' +
'analysis provider per request. Supported: Google Gemini (multimodal — transcription + ' +
'analysis, with speaker labels), Anthropic Claude (analysis), OpenAI (GPT for analysis, ' +
'Whisper for transcription), OpenAI-compatible APIs (DeepSeek, Groq, Together, Fireworks, ' +
'vLLM, etc.), Ollama (local LLMs), and Whisper-compatible endpoints (whisper.cpp, ' +
'faster-whisper-server, NVIDIA Parakeet). ' +
'Free tier ships with a small allotment of relay credits so you can summarize a ' +
'few videos on day one without any setup. Bring your own API key or point at a ' +
'self-hosted model for unlimited use. ' +
'Paid tiers add channel and podcast subscriptions with automatic new-episode detection, ' +
'a background processing queue, auto-download per subscription, organized history with ' +
'folders, and a monthly allotment of relay credits.',
}
export const alertInstall = {
en_US:
'After installing, configure your Google Gemini API key using the "Set Gemini API Key" ' +
'action in the service menu. A free API key is available at aistudio.google.com/apikey. ' +
'After installing, the fastest path is to skip the activation screen and use your free ' +
'relay credits to summarize a few videos. ' +
'For unlimited use: either activate a Recaps license (paid features + monthly relay ' +
'credits), or paste your own AI provider API key in Settings → API Keys & Endpoints. ' +
'Set an admin password via the "Set Admin Password" action if you want to gate access. ' +
'Note: The embedded YouTube player will not work if you are connected to a VPN.',
}
+14 -2
View File
@@ -3,7 +3,7 @@ import { alertInstall, long, short } from './i18n'
export const manifest = setupManifest({
id: 'recap',
title: 'Recap',
title: 'Recaps',
license: 'Proprietary',
packageRepo: 'https://ten31.xyz',
upstreamRepo: 'https://ten31.xyz',
@@ -31,5 +31,17 @@ export const manifest = setupManifest({
start: null,
stop: null,
},
dependencies: {},
dependencies: {
// Optional: enables the local-LLM analysis path. When installed on
// the same StartOS server, the "Set Ollama Server URL" action
// auto-pre-populates with the package's container IP + port, so
// users don't have to type anything. Recap works fine without it
// (cloud providers stay available).
ollama: {
description:
'Run local LLMs (Llama, Mistral, etc.) for topic analysis without a cloud API. Recaps auto-detects the install and pre-fills its connection URL.',
optional: true,
s9pk: null,
},
},
})

Some files were not shown because too many files have changed in this diff Show More