Signal: detect the white speaking border (not a coloured one)

Signal's active-speaker cue is a 3px #ffffff rounded border (saturation ≈ 0),
which the saturation-based highlight detector could never see. Per the
Signal-Desktop source review:

- FrameSampler.thinWhitePoints: grid-sample near-white pixels that sit on a
  THIN structure (a non-white pixel within edgeGap on some axis) so a border/
  ring counts but a solid white blob (face, bright video) does not.
- GridCallAnalyzer: combine coloured (saturated) + white (thin) highlight
  pixels; exclude name-text regions so the white footer name can't be mistaken
  for the border; estimate the tile UP from the name footer (nameAtBottom);
  attribute each highlight pixel to exactly one tile by containment (nearest
  centre as tiebreak) so a border can't bleed into an adjacent tile.
- SignalAdapter: white border on, coloured off, name-at-bottom geometry.

Synthetic 4-tile harness now isolates each speaker with no adjacent-tile bleed;
all 15 XCTest cases pass. Real-screenshot geometry calibration still pending.
This commit is contained in:
Grant Gilliam
2026-06-06 09:52:10 -05:00
parent 863136aeec
commit a56b47143c
4 changed files with 116 additions and 46 deletions
+33 -1
View File
@@ -60,8 +60,40 @@ struct FrameSampler {
return [top, bottom, left, right].map { meanSaturation(inPixelRect: $0) }.max() ?? 0
}
private func isNearWhite(_ x: Int, _ y: Int, minChannel: Double) -> Bool {
guard x >= 0, x < width, y >= 0, y < height else { return false }
let i = (y * width + x) * 4
return Double(pixels[i]) >= minChannel
&& Double(pixels[i + 1]) >= minChannel
&& Double(pixels[i + 2]) >= minChannel
}
/// Grid-sampled near-white pixels that lie on a THIN structure (a non-white
/// pixel within `edgeGap` on some axis) i.e. a border/ring/audio-bar, not a
/// solid white blob (face, bright video). This is Signal's white speaking
/// border (saturation 0, so `saturatedPoints` can't see it).
func thinWhitePoints(minChannel: Double = 200, edgeGap: Int = 6, gridStep: Int = 4) -> [CGPoint] {
var points: [CGPoint] = []
var y = edgeGap
while y < height - edgeGap {
var x = edgeGap
while x < width - edgeGap {
if isNearWhite(x, y, minChannel: minChannel) {
let thin = !isNearWhite(x - edgeGap, y, minChannel: minChannel)
|| !isNearWhite(x + edgeGap, y, minChannel: minChannel)
|| !isNearWhite(x, y - edgeGap, minChannel: minChannel)
|| !isNearWhite(x, y + edgeGap, minChannel: minChannel)
if thin { points.append(CGPoint(x: x, y: y)) }
}
x += gridStep
}
y += gridStep
}
return points
}
/// Grid-sampled pixel positions (top-left origin) that are strongly saturated
/// AND bright enough to be a UI highlight i.e. the speaking ring/border.
/// AND bright enough to be a UI highlight i.e. a coloured speaking ring/border.
func saturatedPoints(threshold: Double = 0.5, minBrightness: Double = 60, gridStep: Int = 6) -> [CGPoint] {
var points: [CGPoint] = []
var y = 0