Phases 2-6: detection, visual timeline, backend hand-off, voiceprints

Phase 2 (call detection): CallDetector using CoreAudio per-process mic
attribution (anarlog technique) — robust start+stop for Zoom/Teams/Signal/Meet,
ignoring our own recording; auto-record toggle. Built; pending live multi-app
confirmation by the user.

Phase 3 (visual timeline foundation): AppAdapter protocol + SpeakerObservation,
TimelineBuilder (hysteresis/overlap/self-merge/aliases), VisualTimeline (schema
1.1), TextRecognizer (Vision OCR), FrameSampler + GridCallAnalyzer (name OCR +
saturated-highlight active-speaker attribution), SignalAdapter, VisualObserver
(window capture; frames released, never saved; minimized->visual_gap, idle != gap).
Synthetic-frame tested; adapter geometry pending real Signal fixtures + live
VisualObserver validation.

Phase 5 (backend hand-off): SparkControlClient (multipart label-merge, sequential,
TLS-skip, 503 Retry-After/413), SessionPackager (chunk plan + WAV slice + timeline
slice/rebase), TranscriptAssembler + SpeakersFile, TranscriptPipeline. Validated
END-TO-END against the live backend (chunk -> label-merge -> speakers.json).

Phase 6 (voiceprints): VoiceprintStore (known_voiceprints, persist named
fingerprints, skip Unknown). Wired: 'Send to backend' button + transcript status,
auto-send toggle (default off) + self-name setting.

All adversarial-review findings fixed. App + XCTest suite build; tests pass.
This commit is contained in:
Grant Gilliam
2026-06-06 00:15:49 -05:00
parent fd7e1a5907
commit 863136aeec
27 changed files with 2108 additions and 22 deletions
@@ -0,0 +1,31 @@
import Foundation
import CoreVideo
/// Signal Desktop adapter. Signal shows avatars/initials with a coloured ring
/// around the active speaker; names may also be available via the Electron
/// Accessibility tree (preferred over OCR when we enable it). Geometry/threshold
/// here are first-pass and will be calibrated against real Signal screenshots.
struct SignalAdapter: AppAdapter {
static let bundleIDs = ["org.whispersystems.signal-desktop"]
let adapterVersion = "signal-0.1.0"
let preferredFPS = 3
private let analyzer: GridCallAnalyzer
init() {
var config = GridCallAnalyzer.Config()
// Signal tiles are squarish with initials centred; tune with fixtures.
config.tileExpandX = 1.6
config.tileExpandY = 1.8
self.analyzer = GridCallAnalyzer(config: config)
}
func analyze(frame: CVPixelBuffer, at t: TimeInterval) -> [SpeakerObservation] {
analyzer.analyze(pixelBuffer: frame, at: t)
}
// Exposed for fixture/synthetic tests.
func analyze(cgImage: CGImage, at t: TimeInterval) -> [SpeakerObservation] {
analyzer.analyze(cgImage: cgImage, at: t)
}
}