Add Daily Digest plan; record render-loop invariants + deploy model in AGENTS.md
This commit is contained in:
@@ -84,6 +84,8 @@ vendor/keysat-licensing-client/ local-link Keysat SDK
|
||||
- **The relay owns cloud Pro/Max tier + expiry** (core-decoupling; `docs/core-decoupling-plan.md`). In multi-mode, paid status is `users.tier` — cached from the relay, keyed by the Recaps user-id — NOT a per-user Keysat license. Don't gate cloud paid features by `keysat_license`; the license only matters for self-hosted "take it home" portability. Cloud requests carry `X-Recap-User-Id` + the operator key; server-to-server tier reads/writes go through `providers/relay.js`.
|
||||
- **Self-serve purchase has two rails, both prepaid (no auto-renew yet).** Bitcoin = a BTCPay invoice rendered as an INLINE Lightning QR on-screen — the relay returns the BOLT11 server-to-server, so the buyer never loads BTCPay (replicate the buy-credits inline flow; do NOT redirect to a hosted checkout). Card = a Zaprite one-time hosted order (Zaprite's API has no recurring — see ROADMAP). Both settle webhooks land on the relay (it owns subscription expiry); the frontend just polls `/api/billing/status`. Expiry reminders go out via the existing System-SMTP transport (`smtp.js`): the relay enumerates who's expiring (`GET /relay/expiring-subscriptions`), Recaps maps user-id → email and sends.
|
||||
- **Tier credit allotments are operator-config-driven, never hardcoded.** The cards' "N relay credits each period" comes from the relay's tier-quota config (`credits_per_period` on `/relay/tier-plans`); `null` → "Unlimited". Don't bake a number or "Unlimited" into the UI.
|
||||
- **`recaps.cc` IS the operator's StartOS box**, served via Start9 Pages + StartTunnel. So `make install` (after a version bump) updates the public cloud site automatically — there is no separate cloud deploy step. A frontend-only change reaches recaps.cc as soon as the box serves the new `public/` files.
|
||||
- **`render()` rebuilds the whole view via `innerHTML` — preserve live media + scroll across it.** It re-attaches the live podcast `<audio>` node (`replaceWith`, src-matched) and restores `.chunks-scroll` scrollTop, so a background re-render (e.g. the ~60s relay-credit poll) doesn't stop playback or bounce the reader to the top. YouTube minimize toggles the `.results-left.minimized` CSS class **in place** — never `render()`, because creating the YT iframe inside a `display:none` container wedges the IFrame API (black frame, needs reload); `ensureYtMounted()` + a `!state.videoMinimized` guard on `needsMount` keep the player from ever being built hidden, and `initPodcastPlayer()` is idempotent (`dataset.inited`). Don't reintroduce a full `render()` on minimize or drop these preservation steps.
|
||||
|
||||
### Client-side contract with the relay
|
||||
|
||||
@@ -125,21 +127,18 @@ unsure whether a change is contract-affecting, assume it is and check.
|
||||
|
||||
## Current state
|
||||
|
||||
**Live on the operator's StartOS box** (app **0.2.157** installed 2026-06-15 + relay **0.2.124**). Note: `recaps.cc` is served from this same box via Start9 Pages + StartTunnel, so a `make install` here updates the public cloud site automatically — there is no separate cloud deploy.
|
||||
**Live on the operator's StartOS box** — app **0.2.157** + relay **0.2.124**. Tests: `cd server && npm test` → **119 pass**.
|
||||
|
||||
- **Self-serve purchase COMPLETE — all 5 phases** (`docs/self-serve-purchase-plan.md`). Signed-in cloud users buy Pro/Max themselves: "Pay with Bitcoin" renders an inline Lightning QR on-screen (no redirect); "Pay by card" mints a Zaprite one-time order (the card link shows only when the operator has configured Zaprite). Prepaid 30-day periods; the relay owns tier + expiry; both settle webhooks land at `extendUserTier`. Expiry-reminder emails (7d / 1d / lapsed) ride the existing System SMTP; operator test trigger: `POST /api/admin/reminders/run` with `{test_email}`. Tier cards show the real per-period credit allotment from the relay quota config (this box: Max = 120, Pro = 50).
|
||||
- **Core-decoupling live** (relay owns cloud tier; `docs/core-decoupling-plan.md`) and **per-tenant subscriptions live** (`docs/per-tenant-subscriptions-plan.md`).
|
||||
- The Bitcoin pill matches the standard purple; the relay-side **internal-meeting re-polish fix** (re-attributes topic summaries to the operator's corrected speaker names) shipped this session.
|
||||
**Done & live:** self-serve Pro/Max purchase (Bitcoin inline-Lightning + Zaprite card, prepaid, relay owns tier/expiry), core-decoupling, per-tenant subscriptions, and expiry-reminder emails (`POST /api/admin/reminders/run {test_email}`). Plans in `docs/*-plan.md`.
|
||||
|
||||
**This session (2026-06-15):** ran the 2026-06-14 full-eval (`EVALUATION.md`) and cleared its 3 P0 + 4 P1 findings. Five code fixes shipped with tests + a reviewer pass — library-import file-write, podcast SSRF, ESM `require`, multi-mode concurrency lock, and the `X-Forwarded-For` trial-cap bypass — committed + pushed (`d0e9842`), 119 tests pass. The leaked Gemini key was purged from all git history and force-pushed; **the rewrite re-hashed every commit** (solo private repo, no other clones — harmless). Registry-submission blockers deferred.
|
||||
**Shipped this session (committed + pushed + verified live):** **0.2.156** — iOS sign-in "network error" flake; both sign-in paths now retry 3× with growing backoff (`91af0b7`). **0.2.157** — mobile/UX cluster: YT-minimize black-frame, podcast-audio + scroll loss on background re-render, redundant loading box, and a best-effort iOS scroll tweak (`693bb98`). Mechanics now captured as conventions above.
|
||||
|
||||
**Also this session — iOS sign-in flake fixed (shipped as 0.2.156, built + installed + verified on the box):** an iPad user hit a spurious "network error" on the first tap of *Send sign-in link*, with the second tap succeeding. Root cause is the classic iOS Safari behavior of dispatching a `POST` onto a pooled keep-alive socket the server/proxy has already closed; unlike a GET it isn't transparently re-sent, so it surfaces as a transport `TypeError`. The existing single 500 ms auto-retry was too quick — it reused the same dead socket. Both sign-in entry points (`public/auth.html` `postWithRetry`, `public/index.html` `fetchWithRetry`) now retry 3× with growing backoff (0 → +400 ms → +1.6 s) to outlast Safari evicting the socket. Frontend-only, no server change; the embedded JS has no test harness. Mitigation not cure — if it ever recurs, confirm via box logs whether `/auth/request-link` is hit once (request never arrived → my diagnosis) or twice (failure on the response path → different bug) before widening the backoff.
|
||||
|
||||
**Also this session — mobile/UX bug cluster from the inbox (shipped as 0.2.157, built + installed + verified; reviewer pass clean, no blockers):** four `public/index.html` fixes. (1) **Video minimize → black/needs-refresh:** `toggleVideoMinimize()` called `render()`, which rebuilt the YouTube `#yt-player` iframe inside the `display:none` minimized container and wedged the IFrame API. Now minimize toggles the `.results-left.minimized` CSS class in place (iframe stays mounted); a `!state.videoMinimized` guard on render's `needsMount` + a new `ensureYtMounted()` (called from the expand paths) ensure the player is never created in a hidden container. (2) **Background processing reset transcript scroll + killed podcast audio:** root cause was the ~60s relay-credit poll calling `render()`, which rebuilt the `<audio id="podcast-audio">` and `.chunks-scroll`. `render()` now preserves the live `<audio>` node across the innerHTML swap (`replaceWith` when the src matches — exploits the spec's async "pause on disconnect") and restores `.chunks-scroll` scrollTop; `initPodcastPlayer()` is idempotent (`dataset.inited`) so the preserved node doesn't double its listeners. (3) **Redundant centered "Processing…" box** removed (pizza-tracker breadcrumb already covers that window). (4) **Mobile can't-scroll-to-top:** added `-webkit-overflow-scrolling:touch` + `overscroll-behavior:contain` to `.chunks-scroll` — **best-effort, UNVERIFIED**; it's iOS-Safari-layout-specific and couldn't be reproduced off-device, so it needs an on-iPad check (and a screen recording if it persists). Inline JS syntax verified via `node --check` on the extracted script.
|
||||
**In progress — Daily Digest** (`docs/daily-digest-plan.md`, **proposed, awaiting go-ahead**): opt-in (off by default) daily email of the last 24h of library recaps, each a 1–2 paragraph overview synthesized from the recap's stored topic summaries; clones the `subscription-reminders.js` scan pattern. Next: build **phase 1** (schema `users.digest_enabled`/`last_digest_at` + opt-in toggle + settings UI) — but **first resolve open Q4: does `/relay/analyze` fit the synthesis call, or is a new relay capability needed?** (cross-repo: would touch `../recap-relay`). The other 3 open Qs (synthesis-cost owner, send hour, single-mode) have defaults in the plan.
|
||||
|
||||
**Pending operator actions:**
|
||||
1. (optional) Rotate the Gemini key in AI Studio — the purge removed it from the repo, but the key itself is still live. Then delete the pre-purge backup: `rm /Users/macpro/Projects/recap-keyleak-purge-backup.bundle` (it contains the old key).
|
||||
2. Real-world cloud tests: first on-device Bitcoin purchase (Core tenant → Upgrade → Pay with Bitcoin → badge flips); enable cards (relay "Set Zaprite Connection" + webhook `https://<relay-host>/relay/zaprite/webhook`); eyeball a reminder email (`POST /api/admin/reminders/run` `{test_email}`).
|
||||
3. If recaps.cc ever gains a CDN/LB hop in front of the app, set `RECAP_TRUSTED_PROXY_HOPS` accordingly or the trial-cap bypass reopens.
|
||||
1. **Verify the mobile can't-scroll-to-top fix on the iPad** — UNVERIFIED in 0.2.157 (iOS-layout-specific, not reproducible off-device); send a screen recording if it persists. Inbox item kept open + annotated.
|
||||
2. (optional) Rotate the still-live Gemini key in AI Studio, then `rm /Users/macpro/Projects/recap-keyleak-purge-backup.bundle`.
|
||||
3. Real-world cloud tests: first Bitcoin purchase; enable Zaprite cards (relay "Set Zaprite Connection" + webhook); eyeball a reminder email.
|
||||
4. If recaps.cc ever gains a CDN/LB hop, set `RECAP_TRUSTED_PROXY_HOPS` or the trial-cap bypass reopens.
|
||||
|
||||
**Next dev steps + open decisions** live in `ROADMAP.md`: the eval's **P2 known-debt** list (SSE error-string scrub, credit-debit TOCTOU, `GET /api/history` perf, dependency CVEs, integration tests for the untested hot paths, doc drift) and **P3** hardening/cleanup, plus the standing decisions (Zaprite recurring, "take Recaps home" broken for relay-tier users, cloud paid-only, no CI lint/type-check). Tests: `cd server && npm test` → **119 pass**.
|
||||
**Backlog** in `ROADMAP.md`: eval **P2** known-debt (SSE error-string scrub, credit-debit TOCTOU, multi-tenant gemini-key bypass, `GET /api/history` perf, dependency CVEs, integration tests, doc drift) + **P3** cleanup, and standing decisions (Zaprite recurring, "take Recaps home" broken for relay-tier users, cloud paid-only, no CI lint/type-check).
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
# Daily Digest — plan
|
||||
|
||||
Status: **proposed** (awaiting go-ahead). Captures the design agreed with Grant on
|
||||
2026-06-15. Build only after sign-off.
|
||||
|
||||
## Goal
|
||||
|
||||
An **opt-in** (off by default) daily "wake-up" email to recaps.cc users: the recaps
|
||||
added to their library in the last ~24 hours, each shown as a **synthesized 1–2
|
||||
paragraph overview** generated from that recap's existing per-topic summaries. Turns
|
||||
passive subscriptions into a daily touchpoint without making the user open the app.
|
||||
|
||||
## Decisions (locked 2026-06-15)
|
||||
|
||||
- **Content** — "overnight recaps": library additions since the user's last digest.
|
||||
- **Audience / opt-in** — multi-mode (recaps.cc) first; **off by default**; per-user toggle.
|
||||
- **Per-episode depth** — a 1–2 paragraph overview *synthesized from the stored topic
|
||||
summaries* (`chunks`). NOT raw full text (too long, Gmail clips >~102 KB), NOT a
|
||||
one-sentence blurb (too thin). This is Grant's call and it's what bounds email size.
|
||||
- **Volume** — per-episode size is bounded by the 2-paragraph synthesis. Still cap at
|
||||
~10 episodes per email with an "and N more in your library →" overflow link for
|
||||
extreme days.
|
||||
- **Cadence** — once per user per ~24h at a fixed server-time hour (default 08:00).
|
||||
Timezone-aware send is a v2. **Skip the email entirely when nothing is new.**
|
||||
- **Dedup** — a per-user `last_digest_at` watermark; each digest covers recaps created
|
||||
since that instant, so nothing repeats and nothing is missed.
|
||||
|
||||
## Data (grounded in code)
|
||||
|
||||
- Saved recap record (`server/history.js` `saveToHistory`): `id`, `title`, `type`,
|
||||
`url`, `createdAt` (ISO), `topicCount`, `chunks` (topics, each with bullet
|
||||
summaries), `entries` (transcript), `speakers`/`speakerNames`. **No top-level
|
||||
summary is stored** → the 1–2 paragraph overview must be synthesized.
|
||||
- Multi-mode users live in the `users` table (`id`, `email`, …); a user's library
|
||||
scope is their user id.
|
||||
|
||||
## Architecture
|
||||
|
||||
Mirror `server/subscription-reminders.js` (the proven daily-scan-plus-email pattern:
|
||||
self-gating, deduped, never throws).
|
||||
|
||||
- **`server/daily-digest.js`** (new)
|
||||
- `runDigestScan({ force })`: gate on `isSmtpReady()` + public URL set. For each
|
||||
opted-in user, list sessions with `createdAt > last_digest_at`; if none, skip. For
|
||||
each new recap, get-or-generate its overview (see below), render the email,
|
||||
`sendMail`, then advance the watermark. Returns a `{sent, skipped}` summary; never
|
||||
throws.
|
||||
- `startDigestScheduler()`: boot delay + interval, fires near the target hour.
|
||||
Idempotent; safe to start unconditionally in multi mode.
|
||||
- **Synthesis** — `synthesizeEpisodeOverview(record)`: send the recap's topic titles +
|
||||
bullet summaries to the relay LLM with a "write a 1–2 paragraph overview" prompt.
|
||||
**Cache** the result back onto the session JSON (e.g. `digestOverview`) so it's
|
||||
generated once and could later power an in-app episode overview. **Sanitize
|
||||
operator-internal strings at this boundary** (Parakeet/CUDA/LAN IPs etc. must not
|
||||
reach cloud users — existing repo convention).
|
||||
- **Email** — `renderDigestEmail({ brandName, episodes, manageUrl, unsubscribeUrl })`
|
||||
in `server/email-template.js`, matching the existing reminder/magic-link templates.
|
||||
- **Opt-in storage** — migration in `server/db.js`: add `users.digest_enabled`
|
||||
(default 0) and `users.last_digest_at` (ms, nullable). Toggle endpoint in
|
||||
`server/account-routes.js` (requires session). Settings-modal toggle in
|
||||
`public/index.html`.
|
||||
- **Unsubscribe** — a one-click tokenized GET link in every email that flips
|
||||
`digest_enabled = 0` without requiring login (signed token), plus the in-app toggle.
|
||||
Consent + deliverability hygiene on the young recaps.cc domain.
|
||||
- **Operator test trigger** — `POST /api/admin/digest/run { test_email }`, mirroring
|
||||
the reminders test hook, so it can be smoke-tested without waiting a day.
|
||||
|
||||
## Cost / credits
|
||||
|
||||
The synthesis is one small relay LLM call per new recap per opted-in user, run once and
|
||||
cached. Bounded by (opted-in users × new recaps/day). **Recommend operator-absorbed**
|
||||
(it's a retention feature, input is already-short topic summaries) rather than drawing
|
||||
the user's credits. Confirm.
|
||||
|
||||
## Open questions (defaults chosen; confirm or adjust)
|
||||
|
||||
1. **Synthesis cost owner** — operator-absorbed (default) vs user credits?
|
||||
2. **Send hour** — 08:00 server time (default)?
|
||||
3. **Single-mode operator digest** — defer to a follow-on (default: multi-mode only v1)?
|
||||
4. **Relay contract** — does an existing relay endpoint (`/relay/analyze`) fit the
|
||||
"summarize these topic summaries into 2 paragraphs" call, or is a small new relay
|
||||
capability/prompt-mode needed? If new, update `../recap-relay` + both repos'
|
||||
`AGENTS.md`/`ROADMAP.md` per the cross-repo rule. **Resolve before phase 2.**
|
||||
|
||||
## Build phases
|
||||
|
||||
1. Schema + opt-in toggle (migration, account endpoint, settings UI).
|
||||
2. Synthesis + cache (relay call + write-back + operator-string scrub). Resolve the
|
||||
relay-contract question first.
|
||||
3. Email template + scan loop + scheduler + watermark dedup + overflow cap.
|
||||
4. Operator test trigger.
|
||||
5. Tests — pure-function coverage (episode selection vs watermark, cap/overflow, empty
|
||||
→ skip), in the `subscription-reminders` test style.
|
||||
Reference in New Issue
Block a user