v0.1.8: two-Spark pipeline + kept-warm serving

- Secondary-Spark models now work in air-gapped mode: the LiteLLM proxy is
  dual-homed onto the default bridge (docker network connect) to reach the
  secondary's published vLLM port; grader containers stay on the --internal
  network with zero egress. The head-only enforcement is replaced by a
  secondary-configured check.
- Extraction runs in parallel with grading when the extractor's model and
  every grader model in the wave sit on different Sparks (separate GPUs).
- Keep-warm: single-wave jobs no longer tear the wave down between decks
  (was a ~6-min 31B reload per deck); a kept-warm wave that fails preflight
  is restarted once. Adjudicator reuses the live wave when its model is
  already serving instead of cycling the shared proxy.
- clear_resident_containers (preJobStopContainers) now stops names on every
  configured Spark; health() reports containers on both Sparks.
- Verified with a mocked dry-run of the full job loop (3 decks: one
  bring-up, zero mid-job teardowns, parallel overlap, stale-wave restart).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jonathan Kirkwood
2026-07-31 15:01:10 -05:00
co-authored by Claude Fable 5
parent 7ec5222834
commit 506e6c79bd
7 changed files with 158 additions and 46 deletions
+5 -4
View File
@@ -29,10 +29,11 @@ const inputSpec = InputSpec.of({
name: 'Network Mode',
description:
'Air-gapped: graders reach ONLY the on-Spark model proxy — zero internet, ' +
'board decks never leave your hardware (models must be pre-pulled into the ' +
'Spark HF cache, all on the head Spark). Local services: graders may also ' +
'reach LAN services like SearXNG and the second Spark (this network has ' +
'egress unless you firewall it).',
'board decks never leave your hardware (models must be pre-pulled into ' +
'each Spark HF cache; models on either Spark are fine — the proxy routes ' +
'to the second Spark over your LAN). Local services: graders may also ' +
'reach LAN services like SearXNG (this network has egress unless you ' +
'firewall it).',
default: 'airgapped',
values: {
airgapped: 'Air-gapped (no network, recommended)',