Fix Meet visual: reject solid avatar tiles + screen-share OCR
Root cause of the "4 people → 2 speakers" Meet call: the colored-border detector
read solid camera-off avatar tiles (orange "J", magenta "G") as active speakers
for the ENTIRE call. Those whole-call phantom spans dominated backend name
attribution, collapsing every remote voice onto one name — and the giant filled
bbox also swallowed screen-share text (WERUNBTC.COM ×49) as a speaker.
Validated against 9 real fixtures (harness over the real MeetAdapter):
Detection:
- FrameSampler.thinColoredPoints: coloured counterpart of thinWhitePoints — keeps
thin border/ring/pill edges, drops solid colour fills.
- GridCallAnalyzer.isHollow: reject a highlight component whose interior is filled
(a solid tile) vs a hollow ring (a real border). Config.maxInteriorFill (0.2 default).
- MeetAdapter: detect thin BLUE edges only (hue 180–240°, measured from the
fixtures), maxInteriorFill 0.3 (real Meet rings ≈0.2–0.3, solid tiles ≈0.36).
- Result on fixtures: John Arnold/Grant Gilliam (solid tiles) now NEVER detected;
Matt Odell/Mark detected when their blue cue is present. Sparse but never wrong —
correct for a naming hint over audio diarization.
OCR name hygiene:
- isLikelyName rejects domain-like screen-share text ("WERUNBTC.COM", OCR'd ".GOM").
- cleaned() strips trailing punctuation ("Mark." → "Mark").
- TimelineBuilder.canonicalizeByFrequency folds rare OCR misspellings into a
dominant near-twin name ("Matt Odel"/"MattOdell" → "Matt Odell", "Mare" → "Mark").
Tests: hollow-ring, extended OCR filter, fuzzy-merge. 65 pass.
This commit is contained in:
@@ -120,6 +120,43 @@ struct FrameSampler {
|
||||
return points
|
||||
}
|
||||
|
||||
/// Grid-sampled saturated pixels that lie on a THIN structure (a non-saturated
|
||||
/// pixel within `edgeGap` on some axis) — the coloured counterpart of
|
||||
/// `thinWhitePoints`. This keeps a thin speaking BORDER/ring/pill but drops the
|
||||
/// solid interior of a colour FILL (e.g. Meet's orange/magenta camera-off avatar
|
||||
/// tiles), whose pixels are surrounded by the same colour. Pair with `hueRange`
|
||||
/// to keep only the cue's colour (Meet's blue ring) and reject the thin edges a
|
||||
/// solid tile still has against the background (orange/magenta boundaries).
|
||||
func thinColoredPoints(threshold: Double = 0.35, minBrightness: Double = 60,
|
||||
hueRange: ClosedRange<Double>? = nil,
|
||||
edgeGap: Int = 6, gridStep: Int = 4) -> [CGPoint] {
|
||||
func isCue(_ x: Int, _ y: Int) -> Bool {
|
||||
guard x >= 0, x < width, y >= 0, y < height else { return false }
|
||||
let i = (y * width + x) * 4
|
||||
let r = Double(pixels[i]), g = Double(pixels[i + 1]), b = Double(pixels[i + 2])
|
||||
let mx = max(r, g, b), mn = min(r, g, b)
|
||||
let sat = mx > 0 ? (mx - mn) / mx : 0
|
||||
guard sat > threshold, mx > minBrightness else { return false }
|
||||
if let hr = hueRange { return hr.contains(Self.hueDegrees(r, g, b, mx, mn)) }
|
||||
return true
|
||||
}
|
||||
var points: [CGPoint] = []
|
||||
var y = edgeGap
|
||||
while y < height - edgeGap {
|
||||
var x = edgeGap
|
||||
while x < width - edgeGap {
|
||||
if isCue(x, y) {
|
||||
let thin = !isCue(x - edgeGap, y) || !isCue(x + edgeGap, y)
|
||||
|| !isCue(x, y - edgeGap) || !isCue(x, y + edgeGap)
|
||||
if thin { points.append(CGPoint(x: x, y: y)) }
|
||||
}
|
||||
x += gridStep
|
||||
}
|
||||
y += gridStep
|
||||
}
|
||||
return points
|
||||
}
|
||||
|
||||
/// HSV hue in degrees (0…360) from RGB and its precomputed max/min channels.
|
||||
private static func hueDegrees(_ r: Double, _ g: Double, _ b: Double, _ mx: Double, _ mn: Double) -> Double {
|
||||
let d = mx - mn
|
||||
|
||||
Reference in New Issue
Block a user