Two operators in a row hit the same crash-loop on upgrade:
Error: running migrations
Caused by:
migration 9 was previously applied but has been modified
sqlx records a SHA-384 of each migration's bytes when first applied,
then verifies the on-disk bytes still match on every subsequent boot.
Cross-build drift (trailing newlines, line-ending normalization, etc.)
produces different bytes for semantically-identical SQL — and sqlx
refuses to start. Recovery required SSHing in and running:
sqlite3 /data/keysat.db "DELETE FROM _sqlx_migrations WHERE version = 9;"
That's bad UX. Worse, every operator going through this version
range hits it once.
Self-heal: db::init now wraps sqlx::migrate!().run() with detection
for MigrateError::VersionMismatch(N) on a constant allowlist of
migrations certified safe to re-run (IDEMPOTENT_MIGRATIONS, just [9]
for now). When triggered, the daemon clears the stale row, retries,
logs a WARN explaining what happened, and continues. No SSH dance.
Allowlist gate is critical — auto-clearing checksums on additive
ALTER TABLE migrations like 0010 would error on retry (SQLite has
no ADD COLUMN IF NOT EXISTS). Only migrations explicitly designed
as drop-and-rebuild (like 0009) and tested via the
`migration_NNNN_is_idempotent` pattern in tests/migrations.rs
qualify.
Regression test in tests/migrations.rs exactly simulates the
production incident:
1. apply all migrations cleanly
2. poison v9's recorded checksum with bogus bytes
3. confirm raw sqlx::migrate! bails (proves the poisoning works)
4. call db::init — must succeed by clearing + re-applying v9
5. confirm v9 + v10 are both recorded with non-poisoned checksums
Test count: 38 (was 37; +1 db_init_self_heals test).
For operators currently stuck on the :49 crash-loop: just upgrade
to :50 from the StartOS marketplace. The :50 daemon will see the
mismatch on first boot, auto-clear v9's row, re-apply (0009 is
idempotent by design), and continue to 0010. No manual sqlite3 needed.
Phase 5 (invoice records the rate):
- repo::create_invoice_with_currency takes the listed currency,
listed value, exchange_rate_centibps, and exchange_rate_source as
optional params; create_invoice (the legacy form) becomes a thin
wrapper that passes None for all four. SAT-priced flows are
unchanged.
- purchase::start now branches on product.price_currency: SAT keeps
the existing path; USD/EUR calls rates::convert_to_sats and pins
the listed price + rate to the local invoice row for audit. The
buyer is still billed in BTC (BTCPay invoice is sat-denominated)
but the audit trail records what they SAW vs what they were
charged.
- Test paid_purchase_in_usd_records_listed_currency_and_rate seeds
a manual rate pin ($50k/BTC), creates a USD-priced product
($49.00), runs through purchase, asserts the invoice row carries
listed_currency='USD', listed_value=4900, rate_centibps=
500_000_000, source='manual_pin', amount_sats=98_000.
Phase 3 (buy page renders fiat):
- Server-rendered initial price respects product.price_currency:
USD products show "49.00 USD" (cents converted to display dollars)
instead of sats. Tier-picker JS still formats per-tier prices in
sats — that's a v0.3 polish when we plumb the rate into the JS
render path. Most operators ship single-policy products at first,
so the static initial render is the high-leverage piece.
Phase 6 (currency-aware discount codes):
- POST /v1/admin/discount-codes accepts optional `discount_currency`
field ('SAT' default, 'USD', 'EUR'). Whitelisted in the handler.
- repo::create_discount_code is now a thin wrapper around
create_discount_code_with_currency; the new helper persists
discount_currency to the column added in 0010. Existing SAT-only
codes keep working unchanged.
Test count: 37 (was 36; +1 paid_purchase_in_usd test).
Multi-currency design phases 1-6 all shipped (1: schema in :48; 2:
admin UI write in :48-:49; 3: buy page; 4: rate fetcher; 5: invoice
audit; 6: discount currency). Phase 7 (recurring subscriptions
re-quote) is v0.3 territory — needs the recurring-billing scaffolding
from Zaprite first.
Migration 0010 adds the columns needed to price products + policies
in something other than satoshis (USD, EUR, BTC at higher denoms)
while keeping every existing operator's data behaviorally identical.
This is the foundation work; admin UI write path, buy page
rendering, and rate fetcher land in subsequent phases. See
MULTI_CURRENCY_DESIGN.md at the parent licensing/ folder for the
full design.
Schema changes (all additive):
- products gain price_currency (TEXT NOT NULL DEFAULT 'SAT') and
price_value (INTEGER NOT NULL DEFAULT 0). Backfill copies
price_sats → price_value on every existing row, so SAT-priced
products carry their information identically through the
migration.
- policies gain price_currency_override (nullable, NULL = inherit
from product) and price_value_override (nullable, mirrors the
existing price_sats_override).
- invoices gain four nullable columns: listed_currency, listed_value,
exchange_rate_centibps, exchange_rate_source. NULL on every
current row; populated by the daemon when an invoice is created
against a fiat-priced product.
- discount_codes gains discount_currency (DEFAULT 'SAT'). 'percent'
codes are currency-agnostic; 'fixed_sats' and 'set_price' codes
use this column to express "$10 off" or "set price to $25"
against fiat-priced products.
- New index idx_products_currency for future "list products by
currency" admin views.
Read path:
- Product struct gains price_currency + price_value fields
(#[serde(default)] for back-compat with any cached/persisted
shapes that predate them).
- row_to_product extracts the new columns; falls back to SAT/
price_sats if a row predates 0010 (defensive — migration always
runs at boot, but no reason to crash if it didn't).
- All four product SELECTs add the new columns.
Write path (legacy SAT-only callers):
- create_product dual-writes price_sats AND price_value to the
same value, with price_currency = 'SAT'.
- update_product dual-writes price_sats and price_value when the
caller passes a new sat price.
Migration regression test:
- migration_0010_backfills_existing_products_to_sat seeds three
products (free, $100, $2500-equivalent) and a policy with a
sat override BEFORE 0010 runs, applies 0010, asserts every row
ends up with price_currency = 'SAT' and price_value =
price_sats. Catches any future change that breaks the
backfill contract.
- migration_0009_is_idempotent now pinned to 0009 by filename
(was: "the last migration"). 0010+ are not idempotent (ALTER
TABLE ADD COLUMN can't be retried in SQLite); the
idempotency test is specifically for 0009 because that
migration's whole point was being safely re-runnable.
Test count: 33 (was 32; +1 migration_0010_backfills test).
Decisions locked in (per MULTI_CURRENCY_DESIGN open questions):
- Default currency on new products: SAT. Operators explicitly
pick USD for fiat-priced products.
- Multi-currency available to all tiers (NOT gated behind Pro/
Patron) — the right product call.
- Rate source priority: Kraken → Coinbase → CoinGecko (lands
in Phase 4 of the design).
- Recurring subscriptions: SAT-priced subs charge the same sat
amount each cycle (no rate adjustment needed); USD-priced subs
re-quote each cycle so the dollar amount is stable.
Closes the silent-loss hole in outbound webhook delivery. The worker
in src/webhooks.rs retries failed deliveries with exponential backoff
up to 10 attempts, then sets next_attempt_at = NULL and walks away.
Pre-this-commit, those "dead-lettered" rows sat in webhook_deliveries
forever with no surface for the operator to discover, inspect, or
recover from them — a subscriber that was down for >6h during a
license-issuance burst would silently lose those events forever.
What's new:
- repo::DeliveryStatusFilter — enum with parse() so query strings
map cleanly to SQL predicates.
- repo::list_deliveries — endpoint_id + status + limit, newest first.
- repo::requeue_delivery — resets attempt_count=0, clears delivered_at
and last_error, sets next_attempt_at=now. The worker picks it up on
the next 5s tick.
- src/api/webhook_deliveries.rs — admin module with two handlers:
- GET /v1/admin/webhook-deliveries?endpoint_id=…&status=…&limit=…
- POST /v1/admin/webhook-deliveries/:id/retry (audit-logged as
webhook_delivery.retry; 404 on missing id)
- Routes registered in src/api/mod.rs alongside the existing
webhook_endpoints CRUD.
- tests/api.rs gains webhook_dlq_lists_failed_and_retry_requeues:
seeds three deliveries directly via SQL (one each: delivered,
pending, dead-lettered), exercises the list filter, runs the retry,
asserts the row migrates from failed→pending, audit row is written,
404 on bad id, 400 on bad status filter.
Worker code is unchanged. The DLQ is operator-actionable infrastructure
on top of the existing retry semantics.
Test count: 23 (9 unit + 4 migration + 10 API), up from 22.
Grant