v0.2.3 - Per-model Advanced settings + catalog-add for downloaded models

Backend:
- overrides.py: read/write /data/models-overrides.yaml (knobs + custom entries)
- apply_knobs_to_args(): strip matching flags from bundled vllm_args and append knob values, so knob changes properly override bundled defaults
- extract_knobs_from_args(): seed UI knob values from bundled args so the Advanced dialog has correct starting state
- models.py: load_catalog merges overrides on top of bundled yaml
- GET /api/models returns effective_knobs per model
- PUT /api/models/{key}/knobs persists knob changes
- POST /api/models adds a custom catalog entry
- DELETE /api/models/{key} removes a custom entry (bundled models cannot be deleted)
- swap_manager.reload_catalog() called after each mutation so swaps see latest

Frontend:
- New 'Advanced' button on every card opens a modal dialog: max-model-len input, gpu-memory-utilization slider, three optimization checkboxes (fastsafetensors, prefix caching, FP8 KV cache). Save persists; Cancel discards. Custom models also have a Delete button.
- After a successful download, automatically open the 'Add to catalog' dialog pre-filled with the repo, with the same knob defaults — user just enters key, display name, and clicks Save.
- Custom catalog entries are tagged with a blue 'custom' pill on the card.

Package: bump 0.2.3:0; main.ts sets MODELS_OVERRIDES=/data/models-overrides.yaml so overrides persist on the StartOS volume.
This commit is contained in:
Grant
2026-05-12 11:30:47 -05:00
parent 474417b458
commit 75fd0846b4
8 changed files with 490 additions and 12 deletions
+1
View File
@@ -46,6 +46,7 @@ export const main = sdk.setupMain(async ({ effects }) => {
MAGPIE_HOST: cfg.magpie_host,
MAGPIE_USER: cfg.magpie_user,
MAGPIE_CONTAINER: cfg.magpie_container,
MODELS_OVERRIDES: '/data/models-overrides.yaml',
BIND_PORT: String(uiPort),
},
},
+2 -2
View File
@@ -1,10 +1,10 @@
import { VersionInfo, IMPOSSIBLE } from '@start9labs/start-sdk'
export const v0_1_0 = VersionInfo.of({
version: '0.2.2:0',
version: '0.2.3:0',
releaseNotes: {
en_US:
'Update checking for spark-vllm-docker. Dashboard footer shows "N commits behind upstream" when applicable; click for the commit log, then "Apply update" runs git pull + ./build-and-copy.sh -c on Spark 1 with a streamed log and elapsed timer. No auto-apply — you confirm each update.',
'Per-model Advanced settings + downloaded-model catalog flow. Each card now has an Advanced button: max context tokens, GPU memory %, and optimization toggles (fastsafetensors, prefix caching, FP8 KV cache). After a download finishes, a dialog appears to add the model to the catalog with those same knobs as launch defaults. Custom models can be deleted. Overrides persist in /data/models-overrides.yaml and survive package updates.',
},
migrations: {
up: async ({ effects }) => {},