Grant Gilliam
63cf3026ff
Cross-platform research (Grant) flagged that the colour-border cue differs by app; checking the real brand colours against the detector found a concrete bug: the global 0.5 saturation threshold MISSES Teams' violet ring (#6264A7 ≈ 0.41, light variants ~0.27) entirely and Meet's lighter blue glow (#8ab4f8 ≈ 0.44). Those adapters would have detected nothing. - FrameSampler.saturatedPoints: add a tunable threshold + optional hue-band gate (degrees) so a lowered threshold doesn't pick up warm video. - GridCallAnalyzer.Config: colorSaturation / colorMinBrightness / colorHueRange, plumbed to the colour-border path (defaults preserve prior behaviour). - MeetAdapter sat→0.35 (catch the glow); TeamsAdapter sat→0.22 + hue 215–275° (catch the faint violet, reject other colours); ZoomAdapter sat 0.45 + hue 40–150° (vivid green/yellow). Values are first-pass pending real-fixture calibration; the hue gate is the main calibration lever. Tests: Teams now detects the faint violet ring and rejects a green one; Meet/Zoom vivid cases still pass. 27/27 XCTest.
49 lines
2.0 KiB
Swift
49 lines
2.0 KiB
Swift
import Foundation
|
||
import CoreVideo
|
||
|
||
/// Zoom adapter (native app: us.zoom.xos).
|
||
///
|
||
/// Zoom's active-speaker cue is a **coloured border** around the speaking tile —
|
||
/// by default a green/yellow outline (configurable in Zoom settings). The
|
||
/// participant **name sits in the tile's bottom-LEFT corner**, so the tile is
|
||
/// estimated extending up and to the right of the name.
|
||
///
|
||
/// Gotchas to calibrate against real fixtures later:
|
||
/// - **Speaker view** shows one big tile; the active speaker fills it (no useful
|
||
/// per-tile border) — handle by attributing speech to the large tile.
|
||
/// - The self-view tile and screen-share change the layout.
|
||
///
|
||
/// Detection *logic* is validated on synthetic frames; geometry constants are a
|
||
/// first pass pending real Zoom screenshots.
|
||
struct ZoomAdapter: AppAdapter {
|
||
static let bundleIDs = ["us.zoom.xos"]
|
||
let adapterVersion = "zoom-0.1.0"
|
||
let preferredFPS = 3
|
||
|
||
private let analyzer: GridCallAnalyzer
|
||
|
||
init() {
|
||
var config = GridCallAnalyzer.Config()
|
||
config.nameAnchor = .bottomLeft
|
||
config.detectColoredBorder = true // green/yellow speaking border (vivid)
|
||
config.detectWhiteBorder = false
|
||
// Zoom's frame is vivid (green #2d8c3c ≈ 0.68, yellow ≈ 0.96); the green→
|
||
// yellow hue band spans ~45–140°. Keep a generous-but-not-trivial threshold;
|
||
// require all-four-sides distribution (handled upstream) to reject bright video.
|
||
config.colorSaturation = 0.45
|
||
config.colorHueRange = 40...150
|
||
config.tileExpandX = 3.0
|
||
config.tileExpandY = 5.0
|
||
self.analyzer = GridCallAnalyzer(config: config)
|
||
}
|
||
|
||
func analyze(frame: CVPixelBuffer, at t: TimeInterval) -> [SpeakerObservation] {
|
||
analyzer.analyze(pixelBuffer: frame, at: t)
|
||
}
|
||
|
||
// Exposed for fixture/synthetic tests.
|
||
func analyze(cgImage: CGImage, at t: TimeInterval) -> [SpeakerObservation] {
|
||
analyzer.analyze(cgImage: cgImage, at: t)
|
||
}
|
||
}
|