Two operators in a row hit the same crash-loop on upgrade:
Error: running migrations
Caused by:
migration 9 was previously applied but has been modified
sqlx records a SHA-384 of each migration's bytes when first applied,
then verifies the on-disk bytes still match on every subsequent boot.
Cross-build drift (trailing newlines, line-ending normalization, etc.)
produces different bytes for semantically-identical SQL — and sqlx
refuses to start. Recovery required SSHing in and running:
sqlite3 /data/keysat.db "DELETE FROM _sqlx_migrations WHERE version = 9;"
That's bad UX. Worse, every operator going through this version
range hits it once.
Self-heal: db::init now wraps sqlx::migrate!().run() with detection
for MigrateError::VersionMismatch(N) on a constant allowlist of
migrations certified safe to re-run (IDEMPOTENT_MIGRATIONS, just [9]
for now). When triggered, the daemon clears the stale row, retries,
logs a WARN explaining what happened, and continues. No SSH dance.
Allowlist gate is critical — auto-clearing checksums on additive
ALTER TABLE migrations like 0010 would error on retry (SQLite has
no ADD COLUMN IF NOT EXISTS). Only migrations explicitly designed
as drop-and-rebuild (like 0009) and tested via the
`migration_NNNN_is_idempotent` pattern in tests/migrations.rs
qualify.
Regression test in tests/migrations.rs exactly simulates the
production incident:
1. apply all migrations cleanly
2. poison v9's recorded checksum with bogus bytes
3. confirm raw sqlx::migrate! bails (proves the poisoning works)
4. call db::init — must succeed by clearing + re-applying v9
5. confirm v9 + v10 are both recorded with non-poisoned checksums
Test count: 38 (was 37; +1 db_init_self_heals test).
For operators currently stuck on the :49 crash-loop: just upgrade
to :50 from the StartOS marketplace. The :50 daemon will see the
mismatch on first boot, auto-clear v9's row, re-apply (0009 is
idempotent by design), and continue to 0010. No manual sqlite3 needed.
Migration 0010 adds the columns needed to price products + policies
in something other than satoshis (USD, EUR, BTC at higher denoms)
while keeping every existing operator's data behaviorally identical.
This is the foundation work; admin UI write path, buy page
rendering, and rate fetcher land in subsequent phases. See
MULTI_CURRENCY_DESIGN.md at the parent licensing/ folder for the
full design.
Schema changes (all additive):
- products gain price_currency (TEXT NOT NULL DEFAULT 'SAT') and
price_value (INTEGER NOT NULL DEFAULT 0). Backfill copies
price_sats → price_value on every existing row, so SAT-priced
products carry their information identically through the
migration.
- policies gain price_currency_override (nullable, NULL = inherit
from product) and price_value_override (nullable, mirrors the
existing price_sats_override).
- invoices gain four nullable columns: listed_currency, listed_value,
exchange_rate_centibps, exchange_rate_source. NULL on every
current row; populated by the daemon when an invoice is created
against a fiat-priced product.
- discount_codes gains discount_currency (DEFAULT 'SAT'). 'percent'
codes are currency-agnostic; 'fixed_sats' and 'set_price' codes
use this column to express "$10 off" or "set price to $25"
against fiat-priced products.
- New index idx_products_currency for future "list products by
currency" admin views.
Read path:
- Product struct gains price_currency + price_value fields
(#[serde(default)] for back-compat with any cached/persisted
shapes that predate them).
- row_to_product extracts the new columns; falls back to SAT/
price_sats if a row predates 0010 (defensive — migration always
runs at boot, but no reason to crash if it didn't).
- All four product SELECTs add the new columns.
Write path (legacy SAT-only callers):
- create_product dual-writes price_sats AND price_value to the
same value, with price_currency = 'SAT'.
- update_product dual-writes price_sats and price_value when the
caller passes a new sat price.
Migration regression test:
- migration_0010_backfills_existing_products_to_sat seeds three
products (free, $100, $2500-equivalent) and a policy with a
sat override BEFORE 0010 runs, applies 0010, asserts every row
ends up with price_currency = 'SAT' and price_value =
price_sats. Catches any future change that breaks the
backfill contract.
- migration_0009_is_idempotent now pinned to 0009 by filename
(was: "the last migration"). 0010+ are not idempotent (ALTER
TABLE ADD COLUMN can't be retried in SQLite); the
idempotency test is specifically for 0009 because that
migration's whole point was being safely re-runnable.
Test count: 33 (was 32; +1 migration_0010_backfills test).
Decisions locked in (per MULTI_CURRENCY_DESIGN open questions):
- Default currency on new products: SAT. Operators explicitly
pick USD for fiat-priced products.
- Multi-currency available to all tiers (NOT gated behind Pro/
Patron) — the right product call.
- Rate source priority: Kraken → Coinbase → CoinGecko (lands
in Phase 4 of the design).
- Recurring subscriptions: SAT-priced subs charge the same sat
amount each cycle (no rate adjustment needed); USD-priced subs
re-quote each cycle so the dollar amount is stable.
The v0.1.0:40 migration was correct on clean installs but crashed at
COMMIT on any database with rows in discount_redemptions: SQLite's
deferred FK check saw the dropped parent's bookkeeping as unsatisfied
even after the rename. Fix is to rebuild discount_redemptions in the
same transaction (stash → drop → rebuild → restore) plus orphan
cleanup. Migration is idempotent; operators on :40 with a checksum
mismatch recover by deleting the version=9 row from _sqlx_migrations
and restarting.
Lands the missing migration test scaffolding too. The four tests in
licensing-service/tests/migrations.rs apply migrations against a
realistic populated database (products, policies, invoices, licenses,
machines, discount codes, redemptions, webhooks, tip attempts). The
regression test fails with the exact 787 error against the v40
migration — would have caught the bug pre-release.
KEYSAT_INTEGRATION.md is removed from this repo; it now lives in the
parent licensing/ folder.
Grant