v0.12.0:0 - WhisperX as a one-click dashboard install + managed service

Replaces the manual rsync+build+run with a proper spark-control feature.
First in the audio path that doesn't require shell access on Spark 2.

What's in the box
─────────────────
* image/whisperx_container/   - the build context (Dockerfile, requirements,
  app/main.py FastAPI wrapper). Mainline pipeline: faster-whisper for STT +
  pyannote 3.1 for diarization + wav2vec2 forced alignment. Single endpoint
  /v1/audio/transcribe-with-speakers returns the exact same shape spark-
  control's existing endpoint does, so the recap-relay PR spec needs no
  changes when we cut over.

* image/app/whisperx_install.py - install manager. ships build context to
  Spark 2 over SSH, runs `docker build`, runs `docker run` with 40 GB
  memory cap (vs Sortformer's unbounded which thrashed Spark 2 on a 90-min
  file), polls /health until both Whisper + pyannote report loaded.

* Audio proxy: /api/audio/transcribe-with-speakers now prefers WhisperX
  when its /health reports diarizer_loaded=true, falls back to the legacy
  Parakeet + Sortformer path otherwise. Same response shape either way.
  Clean cutover, easy rollback (`docker rm whisperx-asr`).

* Dashboard (Audio / Speech tab):
  - "Add WhisperX" banner appears when not installed, with a primary
    "Install WhisperX" button. One click triggers the install.
  - Build progress dialog with phase + elapsed timer + live build log via
    SSE (`/api/whisperx/install/{job_id}/stream`).
  - After install, WhisperX auto-registers as a managed service alongside
    Parakeet and Magpie (Start/Restart/Stop, deep-check, auto-restart).
  - Banner self-hides once /api/whisperx/status reports healthy.

New endpoints
─────────────
  GET  /api/whisperx/status
  POST /api/whisperx/install
  GET  /api/whisperx/install/{job_id}
  GET  /api/whisperx/install/{job_id}/stream  (SSE phase + log)

Config additions (env)
──────────────────────
  WHISPERX_HOST       (defaults to spark2_host)
  WHISPERX_USER       (defaults to spark2_user)
  WHISPERX_CONTAINER  (default: whisperx-asr)
  WHISPERX_PORT       (default: 8002)
  WHISPERX_MODEL      (default: medium; tiny/base/small/medium/large-v3)

Dockerfile
──────────
Added COPY whisperx_container /app/whisperx_container so the runtime
install manager can read the build context from inside the spark-control
image and ship it over SSH.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Keysat
2026-05-18 21:02:26 -05:00
parent cfc1c408d4
commit 5a0bfba6a3
14 changed files with 1033 additions and 3 deletions
+66
View File
@@ -24,6 +24,7 @@ from .overrides import add_custom, delete_custom, extract_knobs_from_args, load_
from .services import docker_state, run_action, services_from_settings
from .speech_models import SpeechModelsManager
from .ssh import ssh_run
from .whisperx_install import WhisperXInstaller
from .swap import SwapManager
from .updates import UpdateManager, get_update_status
from .validate import validate_launch
@@ -39,6 +40,7 @@ hardware_probe = HardwareProbe(settings)
nim_manager = NimManager(settings)
deep_health = DeepHealth(settings)
speech_models = SpeechModelsManager(settings)
whisperx_installer = WhisperXInstaller(settings)
app = FastAPI(title="spark-control", version="0.1.0")
@@ -535,6 +537,70 @@ async def post_speech_models_restart() -> dict:
return result
# ---- WhisperX install (Phase 2 of the WhisperX migration) ----
@app.get("/api/whisperx/status")
async def get_whisperx_status() -> dict:
"""Is WhisperX installed + healthy on Spark 2 right now?"""
return await whisperx_installer.status()
@app.post("/api/whisperx/install")
async def post_whisperx_install() -> dict:
"""One-click install: ships the WhisperX build context from inside
spark-control to Spark 2, runs `docker build` + `docker run`, polls
/health until both models are loaded. Streams progress via the matching
GET /api/whisperx/install/{job_id}/stream SSE endpoint."""
try:
job = await whisperx_installer.trigger()
except RuntimeError as e:
raise HTTPException(409, str(e))
return {"job_id": job.id, "started_at": job.started_at}
@app.get("/api/whisperx/install/{job_id}")
async def get_whisperx_install(job_id: str) -> dict:
job = whisperx_installer.get(job_id)
if not job:
raise HTTPException(404, "unknown job")
return {
"id": job.id,
"state": job.state,
"phase": job.phase,
"lines": job.lines,
"started_at": job.started_at,
"finished_at": job.finished_at,
"returncode": job.returncode,
}
@app.get("/api/whisperx/install/{job_id}/stream")
async def stream_whisperx_install(job_id: str) -> StreamingResponse:
job = whisperx_installer.get(job_id)
if not job:
raise HTTPException(404, "unknown job")
async def event_stream():
last_idx = 0
last_phase = ""
last_state = ""
while True:
new_lines = job.lines[last_idx:]
last_idx = len(job.lines)
for line in new_lines:
yield f"data: {json.dumps({'line': line})}\n\n"
if job.phase != last_phase or job.state != last_state:
yield f"event: phase\ndata: {json.dumps({'phase': job.phase, 'state': job.state})}\n\n"
last_phase = job.phase
last_state = job.state
if job.finished_at:
yield f"event: done\ndata: {json.dumps({'state': job.state, 'returncode': job.returncode})}\n\n"
return
await asyncio.sleep(0.6)
return StreamingResponse(event_stream(), media_type="text/event-stream")
@app.get("/api/endpoints")
async def get_endpoints() -> dict:
"""Service-discovery summary. Stable shape; other apps on the LAN can poll this