Phases 2-6: detection, visual timeline, backend hand-off, voiceprints
Phase 2 (call detection): CallDetector using CoreAudio per-process mic attribution (anarlog technique) — robust start+stop for Zoom/Teams/Signal/Meet, ignoring our own recording; auto-record toggle. Built; pending live multi-app confirmation by the user. Phase 3 (visual timeline foundation): AppAdapter protocol + SpeakerObservation, TimelineBuilder (hysteresis/overlap/self-merge/aliases), VisualTimeline (schema 1.1), TextRecognizer (Vision OCR), FrameSampler + GridCallAnalyzer (name OCR + saturated-highlight active-speaker attribution), SignalAdapter, VisualObserver (window capture; frames released, never saved; minimized->visual_gap, idle != gap). Synthetic-frame tested; adapter geometry pending real Signal fixtures + live VisualObserver validation. Phase 5 (backend hand-off): SparkControlClient (multipart label-merge, sequential, TLS-skip, 503 Retry-After/413), SessionPackager (chunk plan + WAV slice + timeline slice/rebase), TranscriptAssembler + SpeakersFile, TranscriptPipeline. Validated END-TO-END against the live backend (chunk -> label-merge -> speakers.json). Phase 6 (voiceprints): VoiceprintStore (known_voiceprints, persist named fingerprints, skip Unknown). Wired: 'Send to backend' button + transcript status, auto-send toggle (default off) + self-name setting. All adversarial-review findings fixed. App + XCTest suite build; tests pass.
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
import Foundation
|
||||
import CoreVideo
|
||||
|
||||
/// Signal Desktop adapter. Signal shows avatars/initials with a coloured ring
|
||||
/// around the active speaker; names may also be available via the Electron
|
||||
/// Accessibility tree (preferred over OCR when we enable it). Geometry/threshold
|
||||
/// here are first-pass and will be calibrated against real Signal screenshots.
|
||||
struct SignalAdapter: AppAdapter {
|
||||
static let bundleIDs = ["org.whispersystems.signal-desktop"]
|
||||
let adapterVersion = "signal-0.1.0"
|
||||
let preferredFPS = 3
|
||||
|
||||
private let analyzer: GridCallAnalyzer
|
||||
|
||||
init() {
|
||||
var config = GridCallAnalyzer.Config()
|
||||
// Signal tiles are squarish with initials centred; tune with fixtures.
|
||||
config.tileExpandX = 1.6
|
||||
config.tileExpandY = 1.8
|
||||
self.analyzer = GridCallAnalyzer(config: config)
|
||||
}
|
||||
|
||||
func analyze(frame: CVPixelBuffer, at t: TimeInterval) -> [SpeakerObservation] {
|
||||
analyzer.analyze(pixelBuffer: frame, at: t)
|
||||
}
|
||||
|
||||
// Exposed for fixture/synthetic tests.
|
||||
func analyze(cgImage: CGImage, at t: TimeInterval) -> [SpeakerObservation] {
|
||||
analyzer.analyze(cgImage: cgImage, at: t)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user