spark-control

connecting…

Configuration needed. Run the Configure Sparks action in StartOS to set your two Spark IPs and SSH users. Everything else (ports, services, integrations) lives under ⚙ Settings above.

Settings

Optional cluster knobs — vLLM/service ports, container names, support-service hosts, and integrations. The two Spark IPs and SSH users are set once via the Configure Sparks action in StartOS; everything else is here. Changes apply immediately. Stored on this server and included in StartOS backups.

Loading…

Spark hardware

Spark connectivity history

Most recent up/down transitions per Spark. Tracked since this dashboard was installed.

OpenAI-compatible endpoint

Base URL —

Model ID —

curl example

Swap in progress 0:00

Starting…

Show technical logs

🔒 GPU swap path reserved

Always-on services

Install a NVIDIA NIM container

Pick a curated container below or paste any image from the NGC NIM catalog. Spark Control will docker pull and docker run it on the target Spark.

Custom image Image (nvcr.io/...) Container name Port Kind Target Spark

Installing…

Starting…

0:00

Log

Updating matrix-bridge…

Starting…

0:00

Log

matrix-bridge logs

Last 100 lines from docker logs on the Spark.

Speech model patches

Spark Control adds Sortformer speaker diarization to the third-party Parakeet ASR container via two Python overlays (diarizer.py + a patched main.py). Overlays survive container restart but not a fresh redeploy — if the parakeet container is ever rebuilt, click Reapply patches below to restore them.

Reapplying speech-model patches…

Copying overlays into the parakeet container, verifying syntax, restarting, waiting for both models to load. Takes ~60–120 s.

LLM swap

Set up this model

This model is downloaded, but Spark Control needs to know how to launch it. We've guessed from the model's own files — confirm or adjust, and it's saved so you're never asked again.

Key (URL-safe id) Display name Repo (read-only) Size (GB) Mode Description (optional)

Default launch knobs Max context (tokens) GPU memory % 0.85 Fast safetensors loading Prefix caching FP8 KV cache

Add a local / fine-tuned model

For a model that lives as a directory on a Spark (e.g. a fine-tune), not a Hugging Face repo. The directory is bind-mounted into the vLLM container at the same path when you swap to it. It must already exist on the Spark.

Key (URL-safe id) Display name Model directory (absolute path on the Spark) Chat template path (optional) Size (GB) Mode Description (optional)

Default launch knobs Max context (tokens) GPU memory % 0.85 Fast safetensors loading Prefix caching FP8 KV cache

Remove this model from the Sparks?

This deletes the weights and removes the card from the menu. You can always download it again later (re-downloading restores its saved settings).

SSH public key

To let this Spark log in to another machine (e.g. your Mac), run this in a terminal on that machine:

Advanced settings

Custom values are stored in the package volume and survive package updates. Empty fields fall back to defaults.

Max context (tokens) GPU memory % Fast safetensors loading (faster cold start) Prefix caching (speeds up repeated prefixes) FP8 KV cache (halves context memory)

HuggingFace repo ↗

Type any repo, or pick a known one from the list. Browse vLLM-compatible models · NVFP4-quantized models (e.g. RedHatAI/...) are best for Blackwell hardware

Where Spark 1 only Spark 2 only Both Sparks (for cluster models)

For solo models, download to wherever you'll run them. For cluster models (-tp 2), both Sparks need the weights — "Both" downloads to one Spark and rsyncs to the other in parallel.

Downloading… 0:00

Connecting…

Show technical logs

Scheduled jobs

Registered by your own automation. Spark Control only displays these — it doesn't run them.

Updates to eugr/spark-vllm-docker — the upstream project that orchestrates vLLM on your Sparks (launch-cluster.sh, recipes, mods). These are not firmware, OS, or model updates.

Checking for updates…

Pending commits

Explained by the loaded LLM

Applying update…

0:00

Show technical logs

vLLM Parakeet Kokoro Embeddings Qdrant