v0.23.0:0 - local / fine-tuned model support
Add models that live as a directory on a Spark (e.g. LoRA-merged fine-tunes), not just Hugging Face repos. - ModelDef gains local_path; a model must set exactly one of repo / local_path. The validator also enforces the local-path whitelist and that any --chat-template lives inside local_path (only that dir is mounted). - build_launch_command bind-mounts the dir into the vLLM container at the SAME host==container path via the launch script's VLLM_SPARK_EXTRA_DOCKER_ARGS hook, then `vllm serve <dir>`. No launch-cluster.sh change (verified the upstream expands that var unquoted; contract noted in runbook.md). - shellsafe.validate_local_path: absolute path, charset whitelist, no '.'/'..'. - POST /api/models validates the full entry via ModelDef before persisting, so a bad entry can't be written and then break catalog load; _merge_overrides skips an invalid override entry instead of failing the whole catalog. - disk.py size-probes a local path with du; disk-delete refused for local models. - UI: "+ Add local model" dialog, `local` badge, path shown instead of an HF link, delete button hidden for local models. - Tests: local launch + injection round-trip, chat-template location, traversal, exactly-one-source, _merge_overrides skip-invalid (94 pass). Reviewer-agent pass; findings addressed.
This commit is contained in:
@@ -14,7 +14,7 @@ Shape:
|
||||
custom:
|
||||
- key: my-new-model
|
||||
display_name: My New Model (from download)
|
||||
repo: my-org/my-model
|
||||
repo: my-org/my-model # an HF repo; OR set local_path instead (exactly one)
|
||||
size_gb: 20
|
||||
mode: solo
|
||||
description: null
|
||||
@@ -25,6 +25,12 @@ Shape:
|
||||
fastsafetensors: true
|
||||
prefix_caching: true
|
||||
kv_cache_dtype: fp8
|
||||
- key: my-finetune # a local/fine-tuned model (a directory on the Spark)
|
||||
display_name: My Fine-tune
|
||||
local_path: /home/you/models/my-finetune
|
||||
size_gb: 59
|
||||
mode: solo
|
||||
vllm_args: [--chat-template=/home/you/models/my-finetune/chat_template.jinja]
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import os
|
||||
|
||||
Reference in New Issue
Block a user