recap-relay/docs/guides/internal-meetings.md

paths

paths
server/routes/internal-meetings.js server/speaker-clustering.js server/post-cluster-polish.js server/meeting-extras.js server/meeting-speaker-edits.js server/chunked-analyze.js server/backends/hardware.js

Internal-meetings / diarization / speaker subsystem

Subsystem guide for the internal-meetings feature: the upload → transcribe → diarize → cluster → analyze → polish pipeline, and the post-hoc speaker-edit tools the operator dashboard exposes. Whole-repo facts (stack, commands, endpoint contract, tier/billing) live in ../../AGENTS.md; this file lazy-loads when you edit the files it's scoped to.

Pipeline (how speakers are produced)

1. Chunk audio into ~5-min pieces (relay_hardware_tx_chunk_minutes) with a few seconds overlap.
2. Per-chunk diarize at Spark Control /api/audio/diarize-chunk: Sortformer emits chunk-local labels (Speaker_0/1), TitaNet emits a 192-dim voice fingerprint per local speaker. Labels are meaningless across chunks; fingerprints are not.
3. Cross-chunk cluster (speaker-clustering.js, clusterSpeakers): average-linkage agglomerative clustering over all fingerprints by cosine similarity → global Speaker_A/B/…. Then a small-cluster suppression pass folds brief clusters into anchors or Speaker_Unknown.
4. Analyze (windowed, chunked-analyze.js) → section {title, summary, startIndex, endIndex}.
5. Polish (post-cluster-polish.js): runNameInference infers real names from the transcript, then runSummaryPolish rewrites each section summary to attribute statements to those names.
6. Extras (meeting-extras.js): decisions / action items / open questions / key quotes.
7. Audio is deleted after processing (success or failure) — the relay never retains uploaded audio.

Conventions

• A saved meeting record stores the per-chunk TitaNet fingerprints in rec.diarization. Because the audio is gone, this is what makes re-clustering possible offline — no re-upload, no Spark Control round-trip.
• Speaker labels live in FOUR places that every edit must keep in sync: rec.transcript_segments[].speaker, rec.chunks[].entries[].speaker (+ .speaker_override), rec.speakers (per-cluster stats), and rec.extras (tldr.primary_speakers, decisions[].agreed_by, action_items[].owner, key_quotes[].speaker). Display names are a separate map: rec.speaker_names.
• Over-merging (two people clustered as one) is tuned by relay_hardware_voice_clustering_threshold (raise it, e.g. 70→80, to split similar voices) plus the suppression knobs relay_hardware_anchor_min_speaking_sec / relay_hardware_small_cluster_max_speaking_sec / relay_hardware_uncertain_margin_pct. All operator-config-driven; never hardcode.

Post-hoc speaker-edit endpoints (server/meeting-speaker-edits.js)

Operator-dashboard edits to a saved record, mounted under /admin/internal-meetings/:id/* (routing in server/routes/internal-meetings.js). Every edit must keep the four label locations above in sync.

• PATCH /admin/internal-meetings/:id/speakers — rename a cluster (display name only; pre-existing).
• PATCH /admin/internal-meetings/:id/entries — per-line speaker_override (pre-existing).
• PATCH /admin/internal-meetings/:id/merge-speakers — fold cluster(s) into one (ONE person split as two). Pure, offline, no LLM.
• POST /admin/internal-meetings/:id/recluster — re-run clustering at a new threshold (TWO people merged as one). Pure, offline (uses rec.diarization fingerprints); resets speaker_names, per-line overrides, and extras attributions — operator re-labels afterward. 400 if no fingerprints saved.
• POST /admin/internal-meetings/:id/repolish — re-run runSummaryPolish with the current names (no re-inference) so topic summaries re-attribute after a rename/merge. The ONLY LLM-backed edit; needs the analyze hardware online; 400 if no named speakers.

Test coverage: server/test/speaker-clustering.test.js, server/test/meeting-speaker-edits.test.js, server/test/polish-speaker-labels.test.js (node --test).