diff --git a/AGENTS.md b/AGENTS.md index 1492b72..a893cbc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,7 +55,7 @@ Subsystem guidance lives in `docs/guides/` and loads when matching files are tou ## Current state -- **Live: v0.27.2:0 — "Vision check" tool + Qwen3.6 marked vision-capable.** Installed on `immense-voyage` (`start-cli` confirms `0.27.2:0`); **committed/pushed; registry+Gitea-release publish pending Grant's real-card UI confirmation** (couldn't self-verify the front-end — mDNS package subdomain won't resolve from the agent shell). Prompted by a key discovery: **the daily-driver `RedHatAI/Qwen3.6-35B-A3B-NVFP4` is itself a vision model** (`Qwen3_5MoeForConditionalGeneration`, `vision_config` + `model_visual.safetensors` on disk) — its recipe was just mislabelled `[reasoning]`. Verified end-to-end: a business-card OCR request to its `/v1` vision API returned **7/7 fields perfect** (~97 tok/s, no patches). So: (1) `models.yaml` qwen36 now `[vision, reasoning]`; (2) new **Vision check** button on the *running* vision-capable card (`app.js` `openVisionCheck`/`runVisionCheck`) — upload an image + prompt → POST to the existing dumb-passthrough `/v1/chat/completions` proxy (no backend change; same-origin so CSRF-clean) → shows the model's text. 161 pytest green. +- **Live: v0.27.3:0 — Qwen3.6 vision works end-to-end (incl. full-size phone photos).** Installed on `immense-voyage` (`start-cli` confirms `0.27.3:0`). Two-part story: **(A) the daily driver `RedHatAI/Qwen3.6-35B-A3B-NVFP4` is itself a vision model** (`Qwen3_5MoeForConditionalGeneration`, `vision_config` + `model_visual.safetensors` on disk) — recipe was mislabelled `[reasoning]`, now `[vision, reasoning]`. Real business card read **7/7 fields perfect** (~97 tok/s, no patches). **(B) oversized-image fix:** a 12MP phone photo expands to ~11.8k vision tokens → exceeds vLLM's ~4096-image-token cap → **400 "Failed to apply Qwen3VLProcessor … token count mismatch."** Fix = cap resolution server-side via `'--mm-processor-kwargs={"max_pixels": 2000000}'` in the qwen36 recipe (auto-downscales big images for *every* `/v1` consumer; verified live — the 12MP image went 400→200). Quoting survives the stack because `launch-cluster.sh` does `printf "%q"` on the serve args (line 163) and `build_launch_command` shlex-quotes (round-trip test passes). **An in-dashboard "Vision check" button shipped in v0.27.2 then was removed in v0.27.3 at the owner's request** (clutter; the `vision` badge already signals capability — don't re-add it). The `/v1/chat/completions` proxy is a dumb passthrough that already forwards image content, so no backend change was needed. 161 pytest green. - **Gemma-4-26B-A4B-NVFP4 eval — RESOLVED as "defer; Qwen covers vision better."** Two independent deep-research agents (this session) confirmed: it does NOT run on the stock `eugr/spark-vllm-docker` stack (crashes on `tie_weights` `NotImplementedError` — the checkpoint declares compressed-tensors in config.json but is modelopt NVFP4). The working path needs the **`vllm/vllm-openai:gemma4-0505-arm64-cu130`** image (lacks Ray → can't go through `launch-cluster.sh`, needs **raw `docker run`** = the deferred raw-docker-swap feature) **+ a bind-mounted patched `gemma4.py`** (upstream PR #39084 unmerged) **+ `--moe-backend marlin`**, AND even then **vision is degraded** by open vLLM bug #40106 (wrong attention on image tokens — hurts OCR specifically). ~52 tok/s vs Qwen's 97. Net: more duct tape for worse vision than the Qwen Grant already runs. Revisit when #40106 + #39084 land. Alternatives agent also flagged **`RedHatAI/Qwen3.5-122B-A10B-NVFP4`** as the proven single-Spark *reasoning* step-up (30–51 tok/s, fits 128 GB, no patches) — a future daily-driver upgrade, orthogonal to vision. - **Live: v0.27.1:0 — fix: "Download a new model" button (uvx PATH).** Commit `1e1e1cb`; installed on `immense-voyage` (`start-cli package list` confirms `0.27.1:0`); pushed to gitea master; **published to Clankistry** (`~/.spark-control/publish.sh`). Root cause: `hf-download.sh` shells out to `uvx`, which the uv installer puts in `~/.local/bin`; Spark Control's *non-interactive* SSH session doesn't source the user's profile, so `~/.local/bin` is off PATH and the download died with "uvx: command not found" (same class as the matrix-bridge non-interactive-SSH gotcha). Fix: `download.build_download_command` prepends `export PATH="$HOME/.local/bin:$PATH"` (server-side `$HOME`, generic for any adopter); extracted to a pure helper with regression tests (`test_download.py`: PATH prefix, no-trailing-space, cluster flags, shlex round-trip). 161 pytest green; verified live. Prompted by Grant adding **Gemma-4-26B**: he downloaded `nvidia/Gemma-4-26B-A4B-NVFP4` (recipe `gemma4-26b` already in catalog) via the now-fixed button — **fix confirmed end-to-end** — and is swapping to it. **Pending: business-card OCR / vision test** once it's up. - **Live: v0.27.0:0 — in-app Settings gear + two bug fixes** (commit `7e07598`; installed on `immense-voyage` — `start-cli package list` confirms `0.27.0:0`; published to Clankistry; pushed to gitea master). Prompted by the second adopter's v0.25 feedback. (1) StartOS "Configure Sparks" action trimmed to the **four required fields**; all optional knobs moved to a **⚙ Settings gear** in the dashboard, backed by a `/data/app_settings.json` overlay (`app_settings.py`) keyed by env-var names, overlaid on `os.environ`, applied **live** via in-place `Settings.reload()` (architecture + the snapshot-holder gotcha are in the fastapi-image guide). Existing installs' values **migrate automatically** on first boot (`seed_from_env`). (2) **Support-service ports now configurable** (`PARAKEET_PORT`/`KOKORO_PORT`/`EMBED_PORT`/`QDRANT_PORT`; `VLLM_PORT` surfaced) — fixes the adopter's false "vLLM down" (theirs is on 8000, not launch-cluster.sh's 8888) and Parakeet 404 (remapped off 8000). (3) **Bug fix:** `GET /api/swap/lock` 404 (was shadowed by `/api/swap/{job_id}`; lock routes now register first). Code review caught a real P1 (the `WebhookNotifier` snapshot — fixed via `swap_webhook.update()` after reload, regression-tested). 157 pytest + live smoke all green. diff --git a/image/app/static/app.js b/image/app/static/app.js index 1c66f7b..0208118 100644 --- a/image/app/static/app.js +++ b/image/app/static/app.js @@ -4,7 +4,6 @@ const state = { models: {}, defaults: {}, current_model_key: null, - vllm_model: null, // model id vLLM currently reports serving (for the vision check) swap_job_id: null, swap_eventsource: null, swap_started_at: null, @@ -121,11 +120,6 @@ function renderCards() { const recipeActions = m.needs_setup ? '' : ` `; - // "Vision check" only makes sense for the model that's actually loaded, and - // only if it can take images — send an image to it and see what it reads. - const visionBtn = (isActive && (m.capabilities || []).includes('vision')) - ? `` - : ''; card.innerHTML = `
${escapeHtml(m.display_name)}
@@ -144,7 +138,7 @@ function renderCards() {
- ${primaryBtn}${recipeActions}${visionBtn} + ${primaryBtn}${recipeActions} ${trashBtn}
@@ -166,80 +160,6 @@ function renderCards() { for (const btn of root.querySelectorAll('[data-disk-del-key]')) { btn.addEventListener('click', () => openDiskDeleteDialog(btn.dataset.diskDelKey)); } - for (const btn of root.querySelectorAll('[data-vision-key]')) { - btn.addEventListener('click', () => openVisionCheck(btn.dataset.visionKey)); - } -} - -// ===================== vision check ===================== - -function openVisionCheck(key) { - const m = state.models[key]; - el('#vc-model').textContent = m ? ` — ${m.display_name}` : ''; - el('#vc-file').value = ''; - el('#vc-preview').classList.add('hidden'); - el('#vc-preview').removeAttribute('src'); - const res = el('#vc-result'); - res.classList.add('hidden'); - res.textContent = ''; - el('#vision-dialog').showModal(); -} - -function previewVisionImage() { - const file = el('#vc-file').files[0]; - const img = el('#vc-preview'); - if (!file) { img.classList.add('hidden'); return; } - img.src = URL.createObjectURL(file); - img.classList.remove('hidden'); -} - -function readFileAsDataURL(file) { - return new Promise((resolve, reject) => { - const fr = new FileReader(); - fr.onload = () => resolve(fr.result); - fr.onerror = () => reject(new Error('could not read the image file')); - fr.readAsDataURL(file); - }); -} - -async function runVisionCheck() { - const file = el('#vc-file').files[0]; - const res = el('#vc-result'); - if (!file) { alert('Pick an image first.'); return; } - const modelId = state.vllm_model; - if (!modelId) { alert('No running model detected — switch to a model first.'); return; } - const prompt = el('#vc-prompt').value.trim() || 'Describe this image.'; - const btn = el('#vc-run'); - btn.disabled = true; btn.textContent = 'Running…'; - res.classList.remove('hidden', 'fail'); - res.textContent = 'Sending the image to the model…'; - try { - const dataUrl = await readFileAsDataURL(file); - const r = await fetchJSON('/v1/chat/completions', { - method: 'POST', - headers: { 'content-type': 'application/json' }, - body: JSON.stringify({ - model: modelId, - max_tokens: 800, - temperature: 0, - messages: [{ - role: 'user', - content: [ - { type: 'text', text: prompt }, - { type: 'image_url', image_url: { url: dataUrl } }, - ], - }], - }), - }); - const msg = r.choices && r.choices[0] && r.choices[0].message; - const text = (msg && msg.content && msg.content.trim()) || '(model returned no text)'; - res.textContent = text; - } catch (e) { - res.classList.add('fail'); - res.textContent = 'Failed: ' + e.message; - } finally { - btn.disabled = false; btn.textContent = 'Run'; - } } const trashIcon = ''; @@ -1223,7 +1143,6 @@ async function pollStatus() { try { const status = await fetchJSON('/api/status'); state.current_model_key = status.current_model_key; - state.vllm_model = (status.vllm || {}).current_model || null; state.configured = status.configured; renderBanner(status); renderCurrent(status); @@ -2410,8 +2329,6 @@ async function init() { }); el('#sshkey-close').addEventListener('click', () => el('#sshkey-dialog').close()); el('#open-local').addEventListener('click', openLocalModelDialog); - el('#vc-run').addEventListener('click', runVisionCheck); - el('#vc-file').addEventListener('change', previewVisionImage); el('#lock-release').addEventListener('click', releaseLock); setupCatalogDialog(); setupAdvancedDialog(); diff --git a/image/app/static/index.html b/image/app/static/index.html index e304ab5..f4b920c 100644 --- a/image/app/static/index.html +++ b/image/app/static/index.html @@ -365,20 +365,6 @@ - - -