Files
Keysat df9f244eae v0.26.0:0 - disk-driven model menu (scan sparks; recipes; needs-setup)
The dashboard menu is now the set of models actually downloaded on the
Sparks, not a hard-coded catalog. models.yaml + overrides are reframed as
launch recipes matched to an on-disk model by repo; an on-disk model with
no recipe is flagged needs_setup and its launch settings are inferred from
its config.json for a one-time operator confirmation (discovery.py).

- delete now removes weights AND the menu card (delete_from_disk sweeps all
  hosts; the delete endpoint resolves keys via the live menu)
- new GET /api/models/suggest; /api/models returns the menu + a recipes list
  (download autocomplete); GET /api/models/disk-status removed
- dropped the two legacy Qwen recipes (235B FP8, 2.5 72B)
- tests: +test_discovery.py (cache parsing, infer_recipe, build_menu merge)
2026-06-18 11:09:56 -05:00

464 lines
25 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="color-scheme" content="dark">
<title>spark-control</title>
<link rel="stylesheet" href="/static/style.css">
</head>
<body>
<header class="topbar">
<div class="brand">
<span class="logo-dot"></span>
<span>spark-control</span>
</div>
<div class="current" id="current">
<span class="muted">connecting…</span>
</div>
<a id="open-webui-link" class="topbar-btn hidden" href="#" target="_blank" rel="noopener" title="Open Open WebUI">Open chat ↗</a>
</header>
<main>
<section id="setup-banner" class="banner hidden">
<strong>Configuration needed.</strong>
<span>Run the <em>Configure Sparks</em> action in StartOS to set hostnames, then run <em>Test Connection</em>.</span>
</section>
<section id="hardware-panel" class="hardware-panel hidden">
<div class="section-header">
<h2 class="section-title">Spark hardware</h2>
<button id="open-connectivity" class="btn small-btn">Connectivity log</button>
</div>
<div id="hardware-grid" class="hardware-grid"></div>
<dialog id="connectivity-dialog" class="modal">
<form method="dialog" class="modal-form">
<h3>Spark connectivity history</h3>
<p class="muted small">Most recent up/down transitions per Spark. Tracked since this dashboard was installed.</p>
<div id="connectivity-content" class="connectivity-content"></div>
<div class="modal-actions">
<button type="button" id="connectivity-close" class="btn">Close</button>
</div>
</form>
</dialog>
</section>
<section id="endpoint-panel" class="endpoint-panel hidden collapsed">
<div class="ep-header">
<div class="ep-title muted small">OpenAI-compatible endpoint</div>
<button type="button" class="icon-btn ep-collapse-btn" id="ep-collapse" title="Show / hide endpoint details" aria-label="Toggle endpoint details">
<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><polyline points="6 9 12 15 18 9"></polyline></svg>
</button>
</div>
<div class="ep-body">
<div class="ep-row">
<span class="ep-label">Base URL</span>
<code class="ep-value copyable" id="ep-url" data-copy-self title="Click to copy"></code>
<button class="icon-btn" data-copy="#ep-url" title="Copy base URL" aria-label="Copy">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="9" width="13" height="13" rx="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>
</button>
</div>
<div class="ep-row">
<span class="ep-label">Model ID</span>
<code class="ep-value copyable" id="ep-model" data-copy-self title="Click to copy"></code>
<button class="icon-btn" data-copy="#ep-model" title="Copy model ID" aria-label="Copy">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="9" width="13" height="13" rx="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>
</button>
</div>
<details class="ep-curl">
<summary class="muted small">curl example</summary>
<pre id="ep-curl-snippet" class="snippet copyable" data-copy-self title="Click to copy"></pre>
<button class="icon-btn" data-copy="#ep-curl-snippet" title="Copy snippet" aria-label="Copy">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="9" width="13" height="13" rx="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>
</button>
</details>
</div><!-- /.ep-body -->
</section>
<section id="swap-panel" class="swap-panel hidden">
<div class="swap-header">
<span class="spinner"></span>
<span id="swap-title">Swap in progress</span>
<span class="spacer"></span>
<span class="timer" id="swap-elapsed">0:00</span>
</div>
<div class="phase-row">
<div class="phase" id="swap-phase">Starting…</div>
<div class="phase-detail muted small" id="swap-phase-detail"></div>
</div>
<div class="phase-track">
<div class="phase-fill" id="swap-phase-fill"></div>
</div>
<details id="swap-log-details">
<summary class="muted small">Show technical logs</summary>
<pre id="swap-log" class="log"></pre>
</details>
</section>
<section id="lock-banner" class="banner lock-banner hidden">
<span class="lock-icon" aria-hidden="true">🔒</span>
<span id="lock-text">GPU swap path reserved</span>
<span class="spacer"></span>
<button id="lock-release" class="btn small-btn">Release</button>
</section>
<nav id="dashboard-tabs" class="dashboard-tabs hidden" role="tablist">
<button type="button" class="dashboard-tab" data-tab="llm" role="tab" aria-selected="true">LLM</button>
<button type="button" class="dashboard-tab" data-tab="audio" role="tab" aria-selected="false">Audio / Speech</button>
</nav>
<div class="tab-content" id="tab-audio" role="tabpanel" aria-labelledby="tab-audio-trigger">
<section id="services-panel" class="services hidden">
<div class="section-header">
<h2 class="section-title">Always-on services</h2>
<button id="open-nim" class="btn small-btn">+ Install NIM</button>
</div>
<div id="services-grid" class="services-grid"></div>
<dialog id="nim-dialog" class="modal">
<form method="dialog" class="modal-form" id="nim-form">
<h3>Install a NVIDIA NIM container</h3>
<p class="muted small" id="nim-key-warn"></p>
<p class="muted small">Pick a curated container below or paste any image from <a href="#" id="nim-catalog-link" target="_blank" rel="noopener">the NGC NIM catalog</a>. Spark Control will <code>docker pull</code> and <code>docker run</code> it on the target Spark.</p>
<div id="nim-suggested" class="nim-grid"></div>
<fieldset class="modal-fieldset">
<legend>Custom image</legend>
<label class="modal-row"><span>Image (nvcr.io/...)</span><input type="text" id="nim-image" placeholder="nvcr.io/nim/nvidia/<name>:latest"></label>
<label class="modal-row"><span>Container name</span><input type="text" id="nim-container" placeholder="my-service"></label>
<label class="modal-row"><span>Port</span><input type="number" id="nim-port" min="1" max="65535"></label>
<label class="modal-row"><span>Kind</span>
<select id="nim-kind">
<option value="nim">NIM (other)</option>
<option value="stt">STT (speech-to-text)</option>
<option value="tts">TTS (text-to-speech)</option>
<option value="vision">Vision</option>
<option value="embedding">Embedding</option>
</select>
</label>
<label class="modal-row"><span>Target Spark</span>
<select id="nim-host">
<option value="spark2">Spark 2 (default for support services)</option>
<option value="spark1">Spark 1 (head node)</option>
</select>
</label>
</fieldset>
<div class="modal-actions">
<button type="button" id="nim-cancel" class="btn">Cancel</button>
<button type="submit" class="btn primary" id="nim-start">Install</button>
</div>
</form>
</dialog>
<dialog id="nim-progress-dialog" class="modal">
<form method="dialog" class="modal-form">
<h3 id="nim-prog-title">Installing…</h3>
<div class="phase-row">
<div class="phase" id="nim-prog-phase">Starting…</div>
<span class="spacer"></span>
<span class="timer" id="nim-prog-elapsed">0:00</span>
</div>
<details open>
<summary class="muted small">Log</summary>
<pre id="nim-prog-log" class="log"></pre>
</details>
<div class="modal-actions">
<button type="button" id="nim-prog-close" class="btn">Close</button>
</div>
</form>
</dialog>
<dialog id="mb-update-dialog" class="modal">
<form method="dialog" class="modal-form">
<h3 id="mb-update-title">Updating matrix-bridge…</h3>
<div class="phase-row">
<div class="phase" id="mb-update-phase">Starting…</div>
<span class="spacer"></span>
<span class="timer" id="mb-update-elapsed">0:00</span>
</div>
<details open>
<summary class="muted small">Log</summary>
<pre id="mb-update-log" class="log"></pre>
</details>
<div class="modal-actions">
<button type="button" id="mb-update-close" class="btn">Close</button>
</div>
</form>
</dialog>
<dialog id="mb-logs-dialog" class="modal">
<form method="dialog" class="modal-form">
<h3 id="mb-logs-title">matrix-bridge logs</h3>
<p class="muted small">Last 100 lines from <code>docker logs</code> on the Spark.</p>
<pre id="mb-logs-pre" class="log"></pre>
<div class="modal-actions">
<button type="button" id="mb-logs-refresh" class="btn">Refresh</button>
<span class="spacer"></span>
<button type="button" id="mb-logs-close" class="btn">Close</button>
</div>
</form>
</dialog>
</section>
<section id="speech-models-panel" class="speech-models hidden">
<div class="section-header">
<h2 class="section-title">Speech model patches</h2>
</div>
<p class="muted small sm-blurb">
Spark Control adds Sortformer speaker diarization to the third-party Parakeet ASR
container via two Python overlays (<code>diarizer.py</code> + a patched <code>main.py</code>).
Overlays survive container restart but not a fresh redeploy — if the parakeet container is
ever rebuilt, click <strong>Reapply patches</strong> below to restore them.
</p>
<div id="speech-models-card" class="speech-models-card"></div>
<dialog id="speech-models-progress-dialog" class="modal">
<form method="dialog" class="modal-form">
<h3>Reapplying speech-model patches…</h3>
<p class="muted small">Copying overlays into the parakeet container, verifying syntax, restarting, waiting for both models to load. Takes ~60120 s.</p>
<div id="sm-prog-steps" class="sm-prog-steps"></div>
<div class="modal-actions">
<button type="button" id="sm-prog-close" class="btn" disabled>Close</button>
</div>
</form>
</dialog>
</section>
</div><!-- /#tab-audio -->
<div class="tab-content" id="tab-llm" role="tabpanel" aria-labelledby="tab-llm-trigger">
<section id="models-section">
<div class="section-header">
<h2 class="section-title">LLM swap</h2>
<button id="open-download" class="btn small-btn">+ Download a new model</button>
<button id="open-local" class="btn small-btn">+ Add local model</button>
</div>
<dialog id="catalog-dialog" class="modal">
<form method="dialog" class="modal-form" id="catalog-form">
<h3>Set up this model</h3>
<p class="muted small">This model is downloaded, but Spark Control needs to know how to launch it. We've guessed from the model's own files — confirm or adjust, and it's saved so you're never asked again.</p>
<p id="cd-detected" class="muted small cd-detected hidden"></p>
<label class="modal-row"><span>Key (URL-safe id)</span><input type="text" id="cd-key" required pattern="[a-zA-Z0-9_-]+" readonly></label>
<label class="modal-row"><span>Display name</span><input type="text" id="cd-name" required></label>
<label class="modal-row"><span>Repo (read-only)</span><input type="text" id="cd-repo" readonly></label>
<label class="modal-row"><span>Size (GB)</span><input type="number" id="cd-size" step="0.1" min="0"></label>
<label class="modal-row"><span>Mode</span>
<select id="cd-mode">
<option value="solo">solo (Spark 1 only)</option>
<option value="cluster">cluster (both Sparks via Ray)</option>
</select>
</label>
<label class="modal-row"><span>Description (optional)</span><textarea id="cd-desc" rows="3"></textarea></label>
<fieldset class="modal-fieldset">
<legend>Default launch knobs</legend>
<label class="modal-row"><span>Max context (tokens)</span><input type="number" id="cd-mml" step="1024" min="1024" value="32768"></label>
<label class="modal-row"><span>GPU memory %</span><input type="range" id="cd-gmu" min="0.5" max="0.95" step="0.01" value="0.85"> <output id="cd-gmu-out">0.85</output></label>
<label class="modal-row inline"><input type="checkbox" id="cd-fst" checked> Fast safetensors loading</label>
<label class="modal-row inline"><input type="checkbox" id="cd-pcache" checked> Prefix caching</label>
<label class="modal-row inline"><input type="checkbox" id="cd-fp8" checked> FP8 KV cache</label>
</fieldset>
<div class="modal-actions">
<button type="button" id="cd-cancel" class="btn">Cancel</button>
<button type="submit" id="cd-submit" class="btn primary">Save settings</button>
</div>
</form>
</dialog>
<dialog id="local-model-dialog" class="modal">
<form method="dialog" class="modal-form" id="local-model-form">
<h3>Add a local / fine-tuned model</h3>
<p class="muted small">For a model that lives as a directory on a Spark (e.g. a fine-tune), not a Hugging Face repo. The directory is bind-mounted into the vLLM container at the same path when you swap to it. It must already exist on the Spark.</p>
<label class="modal-row"><span>Key (URL-safe id)</span><input type="text" id="lm-key" required pattern="[a-zA-Z0-9_-]+"></label>
<label class="modal-row"><span>Display name</span><input type="text" id="lm-name" required></label>
<label class="modal-row"><span>Model directory (absolute path on the Spark)</span><input type="text" id="lm-path" required placeholder="e.g. /home/you/models/my-finetune"></label>
<label class="modal-row"><span>Chat template path (optional)</span><input type="text" id="lm-chat" placeholder="e.g. /home/you/models/my-finetune/chat_template.jinja"></label>
<label class="modal-row"><span>Size (GB)</span><input type="number" id="lm-size" step="0.1" min="0"></label>
<label class="modal-row"><span>Mode</span>
<select id="lm-mode">
<option value="solo">solo (Spark 1 only)</option>
<option value="cluster">cluster (both Sparks via Ray)</option>
</select>
</label>
<label class="modal-row"><span>Description (optional)</span><textarea id="lm-desc" rows="3"></textarea></label>
<fieldset class="modal-fieldset">
<legend>Default launch knobs</legend>
<label class="modal-row"><span>Max context (tokens)</span><input type="number" id="lm-mml" step="1024" min="1024" value="32768"></label>
<label class="modal-row"><span>GPU memory %</span><input type="range" id="lm-gmu" min="0.5" max="0.95" step="0.01" value="0.85"> <output id="lm-gmu-out">0.85</output></label>
<label class="modal-row inline"><input type="checkbox" id="lm-fst" checked> Fast safetensors loading</label>
<label class="modal-row inline"><input type="checkbox" id="lm-pcache" checked> Prefix caching</label>
<label class="modal-row inline"><input type="checkbox" id="lm-fp8" checked> FP8 KV cache</label>
</fieldset>
<div class="modal-actions">
<button type="button" id="lm-cancel" class="btn">Cancel</button>
<button type="submit" class="btn primary">Add local model</button>
</div>
</form>
</dialog>
<dialog id="disk-delete-dialog" class="modal">
<form method="dialog" class="modal-form">
<h3>Remove this model from the Sparks?</h3>
<p id="dd-summary" class="muted small"></p>
<ul class="muted small dd-hosts" id="dd-hosts"></ul>
<p class="muted small">This deletes the weights and removes the card from the menu. You can always download it again later (re-downloading restores its saved settings).</p>
<p id="dd-error" class="muted small dd-error hidden"></p>
<div class="modal-actions">
<button type="button" id="dd-cancel" class="btn">Cancel</button>
<button type="button" id="dd-confirm" class="btn danger">Remove from disk &amp; menu</button>
</div>
</form>
</dialog>
<dialog id="sshkey-dialog" class="modal">
<form method="dialog" class="modal-form">
<h3 id="sshkey-title">SSH public key</h3>
<p id="sshkey-intro" class="muted small"></p>
<div class="sshkey-row">
<pre id="sshkey-value" class="snippet copyable" data-copy-self title="Click to copy"></pre>
<button type="button" class="icon-btn" data-copy="#sshkey-value" title="Copy public key" aria-label="Copy public key">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="9" width="13" height="13" rx="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg>
</button>
</div>
<p class="muted small">To let this Spark log in to another machine (e.g. your Mac), run this in a terminal <em>on that machine</em>:</p>
<pre id="sshkey-install" class="snippet copyable" data-copy-self title="Click to copy"></pre>
<div class="modal-actions">
<button type="button" id="sshkey-close" class="btn">Close</button>
</div>
</form>
</dialog>
<dialog id="advanced-dialog" class="modal">
<form method="dialog" class="modal-form" id="advanced-form">
<h3 id="adv-title">Advanced settings</h3>
<p class="muted small">Custom values are stored in the package volume and survive package updates. Empty fields fall back to defaults.</p>
<label class="modal-row"><span>Max context (tokens)</span><input type="number" id="adv-mml" step="1024" min="1024"></label>
<label class="modal-row"><span>GPU memory %</span><input type="range" id="adv-gmu" min="0.5" max="0.95" step="0.01"> <output id="adv-gmu-out"></output></label>
<label class="modal-row inline"><input type="checkbox" id="adv-fst"> Fast safetensors loading <span class="muted small">(faster cold start)</span></label>
<label class="modal-row inline"><input type="checkbox" id="adv-pcache"> Prefix caching <span class="muted small">(speeds up repeated prefixes)</span></label>
<label class="modal-row inline"><input type="checkbox" id="adv-fp8"> FP8 KV cache <span class="muted small">(halves context memory)</span></label>
<div class="modal-actions">
<button type="button" id="adv-delete" class="btn danger hidden">Delete model</button>
<span class="spacer"></span>
<button type="button" id="adv-cancel" class="btn">Cancel</button>
<button type="submit" class="btn primary">Save</button>
</div>
</form>
</dialog>
<section id="download-panel" class="download-panel hidden">
<div class="download-form" id="download-form">
<label class="dl-row">
<span class="dl-label">HuggingFace repo</span>
<input type="text" id="dl-repo" placeholder="e.g. RedHatAI/Qwen3.6-35B-A3B-NVFP4" autocomplete="off" list="dl-suggestions">
<datalist id="dl-suggestions"></datalist>
<a id="dl-hf-link" class="dl-hf-link hidden" href="#" target="_blank" rel="noopener" title="Open on Hugging Face"></a>
</label>
<div class="dl-help muted small">
Type any repo, or pick a known one from the list. <a href="https://huggingface.co/models?other=vllm" target="_blank" rel="noopener">Browse vLLM-compatible models</a>
· NVFP4-quantized models (e.g. <code>RedHatAI/...</code>) are best for Blackwell hardware
</div>
<div class="dl-row">
<span class="dl-label">Where</span>
<label class="radio"><input type="radio" name="dl-mode" value="spark1" checked> Spark 1 only</label>
<label class="radio"><input type="radio" name="dl-mode" value="spark2"> Spark 2 only</label>
<label class="radio"><input type="radio" name="dl-mode" value="cluster"> Both Sparks (for cluster models)</label>
</div>
<div class="dl-help muted small">
For <strong>solo</strong> models, download to wherever you'll run them. For <strong>cluster</strong> models (-tp 2), both Sparks need the weights — "Both" downloads to one Spark and rsyncs to the other in parallel.
</div>
<div class="dl-actions">
<button id="dl-cancel" class="btn">Cancel</button>
<button id="dl-start" class="btn primary">Start download</button>
</div>
</div>
<div class="download-progress hidden" id="download-progress">
<div class="dl-header">
<span class="spinner"></span>
<span id="dl-title">Downloading…</span>
<span class="spacer"></span>
<span class="timer" id="dl-elapsed">0:00</span>
</div>
<div class="phase-row">
<div class="phase" id="dl-phase">Connecting…</div>
<div class="phase-detail muted small" id="dl-phase-detail"></div>
</div>
<div class="phase-track">
<div class="phase-fill" id="dl-progress-fill"></div>
</div>
<div class="dl-stats muted small" id="dl-stats"></div>
<details id="dl-log-details">
<summary class="muted small">Show technical logs</summary>
<pre id="dl-log" class="log"></pre>
</details>
</div>
</section>
<section id="cards" class="cards"></section>
</section>
<section id="schedule-panel" class="schedule-panel hidden">
<div class="section-header">
<h2 class="section-title">Scheduled jobs</h2>
</div>
<p class="muted small">Registered by your own automation. Spark Control only displays these — it doesn't run them.</p>
<div id="schedule-list" class="schedule-list"></div>
</section>
<section id="update-banner" class="update-banner hidden">
<div class="ub-context muted small">
Updates to <strong><a href="https://github.com/eugr/spark-vllm-docker" target="_blank" rel="noopener">eugr/spark-vllm-docker</a></strong>
— the upstream project that orchestrates vLLM on your Sparks (launch-cluster.sh, recipes, mods). These are <em>not</em> firmware, OS, or model updates.
</div>
<div class="ub-row">
<span id="ub-text">Checking for updates…</span>
<span class="spacer"></span>
<button id="ub-explain" class="btn small-btn hidden">✨ Explain context</button>
<button id="ub-details" class="btn small-btn hidden">Show details</button>
<button id="ub-apply" class="btn small-btn primary hidden">Apply update</button>
</div>
<details id="ub-list" class="hidden">
<summary class="muted small">Pending commits</summary>
<pre id="ub-log" class="snippet"></pre>
</details>
<details id="ub-explain-section" class="hidden">
<summary class="muted small">Explained by the loaded LLM</summary>
<div id="ub-explain-content" class="explain-content"></div>
</details>
<div id="ub-progress" class="hidden">
<div class="phase-row">
<div class="phase" id="ub-phase">Applying update…</div>
<span class="spacer"></span>
<span class="timer" id="ub-elapsed">0:00</span>
</div>
<details>
<summary class="muted small">Show technical logs</summary>
<pre id="ub-stream" class="log"></pre>
</details>
</div>
</section>
</div><!-- /#tab-llm -->
<footer class="footer">
<div class="health">
<span class="health-item" id="h-vllm"><span class="dot"></span> vLLM</span>
<span class="health-item" id="h-parakeet"><span class="dot"></span> Parakeet</span>
<span class="health-item" id="h-kokoro"><span class="dot"></span> Kokoro</span>
<span class="health-item" id="h-embeddings"><span class="dot"></span> Embeddings</span>
<span class="health-item" id="h-qdrant"><span class="dot"></span> Qdrant</span>
</div>
<div class="muted small" id="updated"></div>
</footer>
</main>
<script src="/static/app.js"></script>
</body>
</html>