The dashboard menu is now the set of models actually downloaded on the
Sparks, not a hard-coded catalog. models.yaml + overrides are reframed as
launch recipes matched to an on-disk model by repo; an on-disk model with
no recipe is flagged needs_setup and its launch settings are inferred from
its config.json for a one-time operator confirmation (discovery.py).
- delete now removes weights AND the menu card (delete_from_disk sweeps all
hosts; the delete endpoint resolves keys via the live menu)
- new GET /api/models/suggest; /api/models returns the menu + a recipes list
(download autocomplete); GET /api/models/disk-status removed
- dropped the two legacy Qwen recipes (235B FP8, 2.5 72B)
- tests: +test_discovery.py (cache parsing, infer_recipe, build_menu merge)
Add models that live as a directory on a Spark (e.g. LoRA-merged fine-tunes),
not just Hugging Face repos.
- ModelDef gains local_path; a model must set exactly one of repo / local_path.
The validator also enforces the local-path whitelist and that any
--chat-template lives inside local_path (only that dir is mounted).
- build_launch_command bind-mounts the dir into the vLLM container at the SAME
host==container path via the launch script's VLLM_SPARK_EXTRA_DOCKER_ARGS hook,
then `vllm serve <dir>`. No launch-cluster.sh change (verified the upstream
expands that var unquoted; contract noted in runbook.md).
- shellsafe.validate_local_path: absolute path, charset whitelist, no '.'/'..'.
- POST /api/models validates the full entry via ModelDef before persisting, so a
bad entry can't be written and then break catalog load; _merge_overrides skips
an invalid override entry instead of failing the whole catalog.
- disk.py size-probes a local path with du; disk-delete refused for local models.
- UI: "+ Add local model" dialog, `local` badge, path shown instead of an HF
link, delete button hidden for local models.
- Tests: local launch + injection round-trip, chat-template location, traversal,
exactly-one-source, _merge_overrides skip-invalid (94 pass). Reviewer-agent
pass; findings addressed.
The 0.8.1:0 probe wrapped the entire path (including $HOME) in
shlex.quote, which produces single quotes — preventing shell
variable expansion. The resulting `[ -d '$HOME/.cache/...' ]` test
looked for a literal path starting with the string $HOME and
always failed, so every model reported as "not downloaded" and no
trash icons rendered.
Fix: embed $HOME in a double-quoted shell context (which allows
expansion) and validate the cache dirname against a whitelist
[A-Za-z0-9._-]+ rather than relying on shlex quoting. The dirname
is fully constrained by HF's naming rules + our org--name munging,
so the whitelist is tight enough.
Verified against Spark 1: probe now correctly reports the
25,075,981,924 bytes (23.4 GB) of Qwen3.6's cache dir.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each model card now shows whether its weights are present on disk
(with GB size) or not yet downloaded. When present and the model
isn't currently loaded, a trash icon appears; clicking it pops a
confirmation showing exactly how many GB will be freed and on
which Spark(s), then runs rm -rf on the HF cache directory via SSH.
Cluster-mode models are removed from both Sparks; solo-mode from
Spark 1 only. Safety rails: refuses to delete the currently-loaded
model, refuses during an in-flight swap or download, and the
catalog entry stays intact so it can be re-downloaded anytime.
Backend:
- new image/app/disk.py: probe_disk + delete_from_disk over SSH
- GET /api/models/disk-status — parallel probe across all catalog models
- DELETE /api/models/{key}/disk — guarded rm -rf, logs to connectivity events
Frontend:
- on-disk / not-downloaded pills on every card
- trash icon-btn in card-actions row (hidden when not on disk)
- confirmation dialog showing per-host bytes-to-free
- disk-status re-checked every 60s
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Keysat