docs(ai): record model-output robustness patterns
CI / proof-of-work (Next.js app) (push) Waiting to run
CI / start9/0.4 (StartOS package code) (push) Waiting to run

Capture what the first live SparkControl/Qwen run taught: looseInt decimal tolerance, the exerciseMatch name->library auto-mapping, and the thinking-token latency characteristic + its lever. Durable subsystem knowledge for future sessions touching the generate flows.
This commit is contained in:
Keysat
2026-06-19 16:23:36 -05:00
parent 891bf09d7e
commit d4557304a5
+24
View File
@@ -76,6 +76,30 @@ stores the JSON in the (reused) `parsedProgram` column.
(configs POST/PATCH, `ai/test`, any probe route) and must appear in those routes' 403
enumeration strings. `ai/test` and `generate` work for free once it's in `getProvider`.
## Model-output robustness (esp. local models)
Local models (Qwen via SparkControl, Ollama) don't honor the JSON contract as tightly
as the cloud APIs, so the parse/apply path is deliberately tolerant. Two layers, both
added after the first SparkControl run surfaced the failures live:
- **Decimal integers** (1.2.0:8): models emit `"rpe": 7.5` / `"reps": 8.0` where the
schema expects ints. `looseInt(z.number().int()…)` (`programSchema.ts`, used by
`workoutSchema.ts`) rounds a number to the nearest int **before** the `.int()` check —
wrap every integer field in both schemas with it. Transform-before-validate, so inferred
types are unchanged. Without it, one stray decimal fails the ENTIRE parse.
- **Exercise→library name matching** (1.2.0:9): models return a good `exerciseName` with a
null or invented `exerciseId`. `lib/ai/exerciseMatch.ts` (`resolveExerciseIds`) normalizes
the name (lowercase, strip the `(barbell)`-style qualifier + punctuation) and auto-maps
only **unique confident** matches; ambiguous/unknown stay null so the UI flags them for
manual mapping. Wired into BOTH generate flows at the parse→display boundary
(`GenerateWorkoutClient`, `GenerateClient`) — re-resolve there if you add a third flow.
- **Latency characteristic (not a bug):** a thinking model (Qwen3.x) spends most of its
output tokens on internal reasoning, streamed as `reasoning_content` — which the OpenAI
streamer ignores (it reads only `delta.content`). So `tokensOut` can be ~10× the visible
JSON and a generation runs minutes (e.g. 7.4k out, 2.8k-char JSON, ~3 min on a DGX Spark
at ~41 tok/s). The lever is **disabling thinking on the vLLM/SparkControl side** (or via a
`chat_template_kwargs:{enable_thinking:false}` request param); left on by owner's choice.
## SSRF / provider-URL safety
- Any `fetch` to a user-supplied provider base URL MUST go through