v0.1.8: two-Spark pipeline + kept-warm serving
- Secondary-Spark models now work in air-gapped mode: the LiteLLM proxy is dual-homed onto the default bridge (docker network connect) to reach the secondary's published vLLM port; grader containers stay on the --internal network with zero egress. The head-only enforcement is replaced by a secondary-configured check. - Extraction runs in parallel with grading when the extractor's model and every grader model in the wave sit on different Sparks (separate GPUs). - Keep-warm: single-wave jobs no longer tear the wave down between decks (was a ~6-min 31B reload per deck); a kept-warm wave that fails preflight is restarted once. Adjudicator reuses the live wave when its model is already serving instead of cycling the shared proxy. - clear_resident_containers (preJobStopContainers) now stops names on every configured Spark; health() reports containers on both Sparks. - Verified with a mocked dry-run of the full job loop (3 decks: one bring-up, zero mid-job teardowns, parallel overlap, stale-wave restart). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
7ec5222834
commit
506e6c79bd
@@ -29,10 +29,11 @@ const inputSpec = InputSpec.of({
|
||||
name: 'Network Mode',
|
||||
description:
|
||||
'Air-gapped: graders reach ONLY the on-Spark model proxy — zero internet, ' +
|
||||
'board decks never leave your hardware (models must be pre-pulled into the ' +
|
||||
'Spark HF cache, all on the head Spark). Local services: graders may also ' +
|
||||
'reach LAN services like SearXNG and the second Spark (this network has ' +
|
||||
'egress unless you firewall it).',
|
||||
'board decks never leave your hardware (models must be pre-pulled into ' +
|
||||
'each Spark HF cache; models on either Spark are fine — the proxy routes ' +
|
||||
'to the second Spark over your LAN). Local services: graders may also ' +
|
||||
'reach LAN services like SearXNG (this network has egress unless you ' +
|
||||
'firewall it).',
|
||||
default: 'airgapped',
|
||||
values: {
|
||||
airgapped: 'Air-gapped (no network, recommended)',
|
||||
|
||||
Reference in New Issue
Block a user