Opt-in community analytics + admin UI surface
Closes the last T2 plan item. Off by default; toggling on requires the operator to confirm a collector URL (an empty URL is "armed but silent"). The toggle lives on the admin Overview page next to the public-key card — the right place for a privacy-affecting choice since it's where operators actually live. What's sent (per the in-card "Show me exactly what gets sent" disclosure, and pinned by the test): - install_uuid: random UUIDv4 generated on first opt-in. NOT derived from operator_name, store id, public URL, or any other identifier. Wipeable via the Reset button. - daemon_version (CARGO_PKG_VERSION). - tier (creator/pro/patron/unlicensed) — the same string the admin tier endpoint already exposes. - counts: products, active_licenses, settled_invoices — each floored to the nearest 5 (anti-fingerprinting; an exact license count uniquely identifies an operator over time). - uptime_bucket: <1d / 1-7d / 1-4w / >4w (bucketed, not exact). What's NOT sent (test asserts none of these strings appear in the preview heartbeat): operator_name, public_url, store_id, api_key, buyer_email, btcpay_url. Also no product/policy slugs or names, no license/invoice ids, no fingerprints, no webhook secrets. Backend: - src/analytics.rs — heartbeat builder, opt-in check, daily background tick (5min initial grace period after boot). - src/api/community.rs — GET / POST / reset admin endpoints. - main.rs spawns the background tick unconditionally; the tick is a no-op if disabled OR no collector URL configured. Frontend (web/index.html, Overview page): - Toggle + collector URL input + privacy disclosure showing the EXACT JSON shape that would be sent (renders the live preview heartbeat from /v1/admin/community-analytics). - "Reset install_uuid" button so an operator who's been beaconing under one identifier can start fresh. Also includes the configureBtcpay.ts idempotency change from v0.1.0:46 (already committed; touched again here only because the diff includes the .ts file in the same dirty-tree push). Test count: 32 (was 31; +1 community_analytics_opt_in_and_privacy_contract which seeds 23 licenses and verifies the heartbeat reports 20 — proves the floor-to-5 anti-fingerprinting is in effect).
This commit is contained in:
Grant
@@ -0,0 +1,223 @@
|
||||
//! Opt-in community analytics.
|
||||
//!
|
||||
//! Off by default. When the operator toggles it on (via the admin UI),
|
||||
//! the daemon periodically POSTs a small anonymous heartbeat to a
|
||||
//! configurable collector URL. The shape is designed to be useful
|
||||
//! for "is Keysat being used and growing?" without exposing anything
|
||||
//! that identifies a specific operator.
|
||||
//!
|
||||
//! ## What's sent (and what isn't)
|
||||
//!
|
||||
//! Sent:
|
||||
//! - `install_uuid` — random UUIDv4 generated on first opt-in,
|
||||
//! stored in the settings table. NOT derived from operator
|
||||
//! identity, store id, or any user-supplied value. Resetting
|
||||
//! analytics opt-in regenerates it.
|
||||
//! - `daemon_version` — e.g. `"0.1.0:46"`.
|
||||
//! - `tier` — `"unlicensed" | "creator" | "pro" | "patron"`.
|
||||
//! - `counts` — rounded down to the nearest 5 to prevent
|
||||
//! fingerprinting an operator by exact license count.
|
||||
//! - `uptime_seconds` — bucketed to "<1d" / "1-7d" / "1-4w" / ">4w".
|
||||
//!
|
||||
//! Not sent:
|
||||
//! - Operator name, public URL, BTCPay URL, store id.
|
||||
//! - Any product or policy slug, name, or description.
|
||||
//! - Any buyer email, license id, invoice id, or fingerprint.
|
||||
//! - Admin API key, webhook secrets, or any other credential.
|
||||
//!
|
||||
//! The opt-in toggle lives in the admin UI (Overview page), with a
|
||||
//! "what gets sent" disclosure and a one-click opt-out. The daemon
|
||||
//! never starts the heartbeat task speculatively — the toggle has
|
||||
//! to be on AND a collector URL has to be configured.
|
||||
|
||||
use crate::api::AppState;
|
||||
use crate::db::repo;
|
||||
use serde::Serialize;
|
||||
use std::time::Duration;
|
||||
use uuid::Uuid;
|
||||
|
||||
pub const SETTING_ENABLED: &str = "community_analytics_enabled";
|
||||
pub const SETTING_INSTALL_UUID: &str = "community_install_uuid";
|
||||
pub const SETTING_COLLECTOR_URL: &str = "community_collector_url";
|
||||
|
||||
/// Default upstream collector. v0.1.0:47 ships with this empty —
|
||||
/// no URL means no requests, even if `enabled = true`. We'll set
|
||||
/// the public collector URL on a future release once the
|
||||
/// keysat.xyz/community endpoint is live.
|
||||
const DEFAULT_COLLECTOR_URL: Option<&str> = None;
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct Heartbeat {
|
||||
pub install_uuid: String,
|
||||
pub daemon_version: &'static str,
|
||||
pub tier: &'static str,
|
||||
pub counts: HeartbeatCounts,
|
||||
pub uptime_bucket: &'static str,
|
||||
pub schema_version: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct HeartbeatCounts {
|
||||
pub products: i64,
|
||||
pub active_licenses: i64,
|
||||
pub settled_invoices: i64,
|
||||
}
|
||||
|
||||
const HEARTBEAT_SCHEMA_VERSION: u32 = 1;
|
||||
|
||||
/// Round down to the nearest `step`. `floor_to(23, 5) == 20`. Used
|
||||
/// to prevent fingerprinting an operator by their exact license
|
||||
/// count — a heartbeat that says "20-24 active licenses" is
|
||||
/// sufficient signal without being unique.
|
||||
fn floor_to(value: i64, step: i64) -> i64 {
|
||||
if step <= 0 {
|
||||
return value;
|
||||
}
|
||||
(value / step) * step
|
||||
}
|
||||
|
||||
fn uptime_bucket(secs: u64) -> &'static str {
|
||||
let day = 86_400;
|
||||
let week = 7 * day;
|
||||
let four_weeks = 4 * week;
|
||||
if secs < day {
|
||||
"<1d"
|
||||
} else if secs < week {
|
||||
"1-7d"
|
||||
} else if secs < four_weeks {
|
||||
"1-4w"
|
||||
} else {
|
||||
">4w"
|
||||
}
|
||||
}
|
||||
|
||||
/// Build a heartbeat snapshot from current state. Always callable —
|
||||
/// returns the snapshot synchronously without sending anything.
|
||||
/// `started_at_secs_since_epoch` is the daemon's start time (used
|
||||
/// to compute the uptime bucket).
|
||||
pub async fn build_heartbeat(
|
||||
state: &AppState,
|
||||
install_uuid: &str,
|
||||
started_at_secs: u64,
|
||||
) -> anyhow::Result<Heartbeat> {
|
||||
let products: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM products")
|
||||
.fetch_one(&state.db)
|
||||
.await?;
|
||||
let active_licenses: i64 =
|
||||
sqlx::query_scalar("SELECT COUNT(*) FROM licenses WHERE status = 'active'")
|
||||
.fetch_one(&state.db)
|
||||
.await?;
|
||||
let settled_invoices: i64 =
|
||||
sqlx::query_scalar("SELECT COUNT(*) FROM invoices WHERE status = 'settled'")
|
||||
.fetch_one(&state.db)
|
||||
.await?;
|
||||
|
||||
let now = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(started_at_secs);
|
||||
let uptime = now.saturating_sub(started_at_secs);
|
||||
|
||||
let tier_label = crate::api::tier::current(state).await.label;
|
||||
|
||||
Ok(Heartbeat {
|
||||
install_uuid: install_uuid.to_string(),
|
||||
daemon_version: env!("CARGO_PKG_VERSION"),
|
||||
tier: tier_label,
|
||||
counts: HeartbeatCounts {
|
||||
products: floor_to(products, 5),
|
||||
active_licenses: floor_to(active_licenses, 5),
|
||||
settled_invoices: floor_to(settled_invoices, 5),
|
||||
},
|
||||
uptime_bucket: uptime_bucket(uptime),
|
||||
schema_version: HEARTBEAT_SCHEMA_VERSION,
|
||||
})
|
||||
}
|
||||
|
||||
/// Read the opt-in flag. Returns Ok(false) on any storage error so
|
||||
/// we always default to "off" — never accidentally beacon because
|
||||
/// the settings table is unreachable.
|
||||
pub async fn is_enabled(state: &AppState) -> bool {
|
||||
match repo::settings_get(&state.db, SETTING_ENABLED).await {
|
||||
Ok(Some(v)) => v == "1" || v.eq_ignore_ascii_case("true"),
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Get-or-create the install UUID. Idempotent; the first call after
|
||||
/// opt-in writes a fresh UUIDv4 to the settings table, all later
|
||||
/// calls read it back.
|
||||
pub async fn ensure_install_uuid(state: &AppState) -> anyhow::Result<String> {
|
||||
if let Some(existing) = repo::settings_get(&state.db, SETTING_INSTALL_UUID).await? {
|
||||
if !existing.is_empty() {
|
||||
return Ok(existing);
|
||||
}
|
||||
}
|
||||
let fresh = Uuid::new_v4().to_string();
|
||||
repo::settings_set(&state.db, SETTING_INSTALL_UUID, Some(&fresh)).await?;
|
||||
Ok(fresh)
|
||||
}
|
||||
|
||||
/// Spawn the heartbeat-sending background task. No-op every tick if
|
||||
/// the opt-in toggle is off OR no collector URL is configured.
|
||||
///
|
||||
/// Tick cadence: every 24 hours, with a small initial delay so we
|
||||
/// don't hit the collector during boot if many operators restart at
|
||||
/// once. Aligned roughly to the day so heartbeats don't cluster.
|
||||
pub fn spawn(state: AppState) {
|
||||
let started_at_secs = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(0);
|
||||
|
||||
tokio::spawn(async move {
|
||||
// 5-minute grace period after boot before the first
|
||||
// heartbeat. Avoids beaconing during the warm-up window
|
||||
// when caches are empty and counts might be misleading.
|
||||
tokio::time::sleep(Duration::from_secs(300)).await;
|
||||
loop {
|
||||
if let Err(e) = tick(&state, started_at_secs).await {
|
||||
tracing::warn!(error = %e, "community-analytics heartbeat failed");
|
||||
}
|
||||
tokio::time::sleep(Duration::from_secs(86_400)).await;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async fn tick(state: &AppState, started_at_secs: u64) -> anyhow::Result<()> {
|
||||
if !is_enabled(state).await {
|
||||
return Ok(());
|
||||
}
|
||||
let collector_url = match repo::settings_get(&state.db, SETTING_COLLECTOR_URL).await? {
|
||||
Some(u) if !u.is_empty() => u,
|
||||
_ => match DEFAULT_COLLECTOR_URL {
|
||||
Some(u) => u.to_string(),
|
||||
None => return Ok(()), // explicitly opted in but no URL configured — silent no-op
|
||||
},
|
||||
};
|
||||
|
||||
let install_uuid = ensure_install_uuid(state).await?;
|
||||
let payload = build_heartbeat(state, &install_uuid, started_at_secs).await?;
|
||||
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(Duration::from_secs(10))
|
||||
.build()?;
|
||||
let resp = client
|
||||
.post(&collector_url)
|
||||
.json(&payload)
|
||||
.send()
|
||||
.await?;
|
||||
if !resp.status().is_success() {
|
||||
anyhow::bail!(
|
||||
"collector responded with HTTP {}: {}",
|
||||
resp.status(),
|
||||
resp.text().await.unwrap_or_default()
|
||||
);
|
||||
}
|
||||
tracing::info!(
|
||||
collector = %collector_url,
|
||||
install_uuid = %install_uuid,
|
||||
"community-analytics heartbeat sent"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,173 @@
|
||||
//! Admin endpoints for the opt-in community analytics toggle.
|
||||
//!
|
||||
//! Three endpoints:
|
||||
//! GET /v1/admin/community-analytics — current state + a
|
||||
//! preview of what
|
||||
//! would be sent
|
||||
//! POST /v1/admin/community-analytics — set enabled +
|
||||
//! collector_url
|
||||
//! POST /v1/admin/community-analytics/reset — wipes the install
|
||||
//! UUID (so a future
|
||||
//! opt-in generates
|
||||
//! a fresh anonymous
|
||||
//! identifier)
|
||||
//!
|
||||
//! The toggle is intentionally a multi-step decision: enabling
|
||||
//! requires the operator to also confirm a collector URL. The
|
||||
//! daemon never beacons without both being set.
|
||||
|
||||
use crate::analytics::{
|
||||
self, SETTING_COLLECTOR_URL, SETTING_ENABLED, SETTING_INSTALL_UUID,
|
||||
};
|
||||
use crate::api::admin::{request_context, require_admin};
|
||||
use crate::api::AppState;
|
||||
use crate::db::repo;
|
||||
use crate::error::{AppError, AppResult};
|
||||
use axum::{extract::State, http::HeaderMap, Json};
|
||||
use serde::Deserialize;
|
||||
use serde_json::{json, Value};
|
||||
|
||||
pub async fn get(
|
||||
State(state): State<AppState>,
|
||||
headers: HeaderMap,
|
||||
) -> AppResult<Json<Value>> {
|
||||
require_admin(&state, &headers)?;
|
||||
let enabled = analytics::is_enabled(&state).await;
|
||||
let collector_url = repo::settings_get(&state.db, SETTING_COLLECTOR_URL).await?;
|
||||
let install_uuid = repo::settings_get(&state.db, SETTING_INSTALL_UUID).await?;
|
||||
|
||||
// Preview: build a heartbeat snapshot RIGHT NOW so the operator
|
||||
// sees exactly what would be sent. This is the privacy-by-
|
||||
// demonstration move — nothing happens behind their back.
|
||||
let preview = match install_uuid.as_deref() {
|
||||
Some(uuid) if !uuid.is_empty() => {
|
||||
// started_at = now-since-epoch; preview shows uptime "<1d"
|
||||
let now = std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(0);
|
||||
let snap = analytics::build_heartbeat(&state, uuid, now).await?;
|
||||
serde_json::to_value(snap).unwrap_or(serde_json::Value::Null)
|
||||
}
|
||||
_ => {
|
||||
// Show what a heartbeat WOULD look like with a placeholder
|
||||
// uuid so operators can see the shape before opting in.
|
||||
let snap = analytics::build_heartbeat(
|
||||
&state,
|
||||
"00000000-0000-0000-0000-000000000000",
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(0),
|
||||
)
|
||||
.await?;
|
||||
serde_json::to_value(snap).unwrap_or(serde_json::Value::Null)
|
||||
}
|
||||
};
|
||||
|
||||
Ok(Json(json!({
|
||||
"enabled": enabled,
|
||||
"collector_url": collector_url,
|
||||
"install_uuid": install_uuid,
|
||||
"preview_heartbeat": preview,
|
||||
})))
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct SetReq {
|
||||
pub enabled: bool,
|
||||
pub collector_url: Option<String>,
|
||||
}
|
||||
|
||||
pub async fn set(
|
||||
State(state): State<AppState>,
|
||||
headers: HeaderMap,
|
||||
Json(req): Json<SetReq>,
|
||||
) -> AppResult<Json<Value>> {
|
||||
let actor_hash = require_admin(&state, &headers)?;
|
||||
let (ip, ua) = request_context(&headers);
|
||||
|
||||
// Validate URL shape if one was supplied. We don't try to reach
|
||||
// it — the heartbeat task does that on its own schedule.
|
||||
let collector_url_clean: Option<String> = req
|
||||
.collector_url
|
||||
.as_deref()
|
||||
.map(|s| s.trim().to_string())
|
||||
.filter(|s| !s.is_empty());
|
||||
if let Some(url) = collector_url_clean.as_deref() {
|
||||
if !(url.starts_with("http://") || url.starts_with("https://")) {
|
||||
return Err(AppError::BadRequest(
|
||||
"collector_url must start with http:// or https://".into(),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
// Enabling without a collector URL is allowed (collector_url
|
||||
// can be set later, OR the future built-in default URL will
|
||||
// kick in once it ships). But surface the situation in the
|
||||
// response so the SPA can show "enabled but not yet beaconing"
|
||||
// state if relevant.
|
||||
let enabled_str = if req.enabled { "1" } else { "0" };
|
||||
repo::settings_set(&state.db, SETTING_ENABLED, Some(enabled_str)).await?;
|
||||
repo::settings_set(
|
||||
&state.db,
|
||||
SETTING_COLLECTOR_URL,
|
||||
collector_url_clean.as_deref(),
|
||||
)
|
||||
.await?;
|
||||
|
||||
// Generate the install UUID on first opt-in. (No-op on
|
||||
// subsequent toggles — the UUID persists across enable/disable
|
||||
// cycles unless explicitly reset, so a flip-flop doesn't make
|
||||
// the same install look like a new one.)
|
||||
if req.enabled {
|
||||
analytics::ensure_install_uuid(&state).await?;
|
||||
}
|
||||
|
||||
let _ = repo::insert_audit(
|
||||
&state.db,
|
||||
"admin_api_key",
|
||||
Some(&actor_hash),
|
||||
if req.enabled { "community_analytics.enable" } else { "community_analytics.disable" },
|
||||
Some("setting"),
|
||||
Some(SETTING_ENABLED),
|
||||
ip.as_deref(),
|
||||
ua.as_deref(),
|
||||
&json!({
|
||||
"enabled": req.enabled,
|
||||
"collector_url_set": collector_url_clean.is_some(),
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
|
||||
Ok(Json(json!({
|
||||
"enabled": req.enabled,
|
||||
"collector_url": collector_url_clean,
|
||||
})))
|
||||
}
|
||||
|
||||
/// Wipes the install UUID. After a reset, the next opt-in generates
|
||||
/// a fresh UUID — useful for an operator who's been beaconing under
|
||||
/// one identifier and wants to start over (e.g., after a DB restore
|
||||
/// from a snapshot taken before they opted in originally).
|
||||
pub async fn reset(
|
||||
State(state): State<AppState>,
|
||||
headers: HeaderMap,
|
||||
) -> AppResult<Json<Value>> {
|
||||
let actor_hash = require_admin(&state, &headers)?;
|
||||
let (ip, ua) = request_context(&headers);
|
||||
repo::settings_set(&state.db, SETTING_INSTALL_UUID, None).await?;
|
||||
let _ = repo::insert_audit(
|
||||
&state.db,
|
||||
"admin_api_key",
|
||||
Some(&actor_hash),
|
||||
"community_analytics.reset",
|
||||
Some("setting"),
|
||||
Some(SETTING_INSTALL_UUID),
|
||||
ip.as_deref(),
|
||||
ua.as_deref(),
|
||||
&json!({}),
|
||||
)
|
||||
.await;
|
||||
Ok(Json(json!({ "ok": true })))
|
||||
}
|
||||
@@ -69,6 +69,7 @@ pub mod self_license;
|
||||
pub mod session_layer;
|
||||
pub mod tier;
|
||||
pub mod validate;
|
||||
pub mod community;
|
||||
pub mod db_info;
|
||||
pub mod recover;
|
||||
pub mod webhook;
|
||||
@@ -324,6 +325,16 @@ pub fn router(state: AppState) -> Router {
|
||||
// Database health snapshot — operator-facing sanity check
|
||||
// against the catastrophic-loss risk; see db_info.rs.
|
||||
.route("/v1/admin/db-info", get(db_info::get))
|
||||
// Opt-in community analytics. Off by default; toggling on
|
||||
// requires the operator to confirm a collector URL.
|
||||
.route(
|
||||
"/v1/admin/community-analytics",
|
||||
get(community::get).post(community::set),
|
||||
)
|
||||
.route(
|
||||
"/v1/admin/community-analytics/reset",
|
||||
post(community::reset),
|
||||
)
|
||||
// Discount / referral codes.
|
||||
.route(
|
||||
"/v1/admin/discount-codes",
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
//! background tasks. Tests bypass that wrapper and construct `AppState`
|
||||
//! programmatically.
|
||||
|
||||
pub mod analytics;
|
||||
pub mod api;
|
||||
pub mod btcpay;
|
||||
pub mod config;
|
||||
|
||||
@@ -6,7 +6,9 @@
|
||||
//! changes between targets.
|
||||
|
||||
use anyhow::Context;
|
||||
use keysat::{api, btcpay, config, crypto, db, license_self, payment, reconcile, webhooks};
|
||||
use keysat::{
|
||||
analytics, api, btcpay, config, crypto, db, license_self, payment, reconcile, webhooks,
|
||||
};
|
||||
use std::sync::Arc;
|
||||
use tower_http::trace::TraceLayer;
|
||||
use tracing_subscriber::{fmt, prelude::*, EnvFilter};
|
||||
@@ -80,6 +82,10 @@ async fn main() -> anyhow::Result<()> {
|
||||
// Spawn background loops before handing state to the router.
|
||||
reconcile::spawn(state.clone());
|
||||
webhooks::spawn_delivery_worker(state.clone());
|
||||
// Opt-in community analytics — every tick checks the toggle
|
||||
// and short-circuits if disabled (default), so spawning is safe
|
||||
// unconditionally.
|
||||
analytics::spawn(state.clone());
|
||||
|
||||
// Hourly session reaper — drops sessions whose expires_at < now.
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user