Jonathan KirkwoodandClaude Fable 5
b1d7aed9f4
- Deterministic scoring.py (quant 60 / qual 40 / flags -15, profitability heaviest) - Per-company JSON ledger with forecast-target chaining deck N-1 -> N - Single-shot sandbox agent with guided-JSON fallback ladder (no tool loop) - Portfolio dashboard with sparklines, KPI hit rates, BDEF category bars - 48 unit tests green; endpoints smoke-tested; npm check+build green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
110 lines
6.0 KiB
Markdown
110 lines
6.0 KiB
Markdown
# Boardroom Map — private board-deck grading on your DGX Sparks
|
||
|
||
Boardroom Map is a StartOS service (`.s9pk`) that **grades portfolio-company
|
||
board decks with a panel of local LLMs** on your NVIDIA DGX Sparks. Drop each
|
||
company's deck into `inbox/<company-slug>/`; the panel grades it against the
|
||
**BDEF v1.1 framework** (Girdley + Munger/Buffett), an optional local
|
||
**adjudicator** reconciles the panel, and a deterministic Python scorer computes
|
||
a 0–100 composite that lands on the company's **running scorecard ledger**. A
|
||
web dashboard shows per-company trends. There is **no frontier model and no
|
||
cloud API key** — in the default air-gapped mode the decks and their grades
|
||
never leave your hardware.
|
||
|
||
It is a sibling of [Chambers](../chambers) and reuses the same control-plane
|
||
pattern (a GPU-free orchestrator on StartOS driving the Sparks over SSH), but
|
||
swaps the free-form document-review panel for a **deterministic deck-grading
|
||
pipeline** with pinned KPI targets and per-company ledgers.
|
||
|
||
## The scoring model
|
||
|
||
`composite (0–100) = quant 60 + qual 40 − red flags (capped at 15)`
|
||
|
||
- **Quantitative 60:** profitability KPI attainment **30** (heaviest slice),
|
||
other measurable KPIs **20**, **forecast integrity 10** — deck N's actuals are
|
||
chained against deck N−1's stated targets, so moved goalposts cost points.
|
||
KPI credit is linear above a floor ratio (default 0.5 → zero credit below).
|
||
- **Qualitative 40:** eight BDEF categories (A–H) × 5 points, scored by the
|
||
panel with evidence quotes; thin evidence scales down.
|
||
- **Red flags:** up to **−15**; silently dropped KPIs are auto-flagged (capped),
|
||
and flags raised by a single grader are damped by 0.5.
|
||
|
||
Every knob lives in config (`weights`, per-company `pinnedTargets` and
|
||
`kpiAliases`) so the model can be retuned without a rebuild.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
StartOS box (control plane, no GPU) DGX Spark(s)
|
||
┌────────────────────────────────────┐ ┌───────────────────────────────┐
|
||
│ FastAPI dashboard + job runner │ SSH │ per-job Docker network │
|
||
│ • inbox/<company-slug>/ (decks) │ ───────▶│ (──internal in airgapped) │
|
||
│ • extract text (PDF/DOCX/TXT/MD) │ rsync │ ┌─────────┐ ┌────────────┐ │
|
||
│ • plan model "waves" │ ───────▶│ │ vLLM(s) │◀─│ LiteLLM │ │
|
||
│ • extractor → graders → adjudicator│ │ └─────────┘ │ router │ │
|
||
│ • deterministic composite scorer │◀─────── │ ┌──────────────┐ ▲ │ │
|
||
│ • per-company ledgers + scorecards │ rsync │ │ grader ×N │──┘ │ │
|
||
└────────────────────────────────────┘ │ │ (read-only, │ │ │
|
||
│ │ sandboxed) │ │ │
|
||
│ └──────────────┘ │ │
|
||
└───────────────────────────────┘
|
||
```
|
||
|
||
- **Graders** are one-shot, read-only, hardened containers (non-root,
|
||
`--cap-drop ALL`, read-only rootfs, no docker socket). In air-gapped mode they
|
||
sit on an `--internal` network and can reach only the model proxy.
|
||
- **Waves:** the job runner serves models in waves bounded by
|
||
`maxConcurrentModels`, so a panel can span more models than fit in GPU memory
|
||
at once.
|
||
- **Air-gap modes:** `airgapped` (default — graders reach only the on-Spark
|
||
model proxy, zero egress, models pre-pulled) or `local_services` (graders may
|
||
reach LAN services like SearXNG and the second Spark — has egress unless
|
||
firewalled).
|
||
- **Confidentiality:** decks are extracted to text on the StartOS box; only
|
||
text crosses to the Sparks, and it is wiped from the Sparks after the job.
|
||
Scorecards and ledgers live only on the StartOS box.
|
||
|
||
## Setup order
|
||
|
||
Configure Sparks → Test Spark Connection → Configure Models → Configure Graders
|
||
→ Configure Grading (rubric, air-gap, weights) → Configure Companies (slugs,
|
||
KPI aliases, pinned targets — especially profitability thresholds) → drop decks
|
||
into `inbox/<company-slug>/2026-Q2-deck.pdf` → **Grade Decks** → watch the
|
||
dashboard.
|
||
|
||
## Repo layout
|
||
|
||
```
|
||
startos/ StartOS package definition (TypeScript / start-sdk)
|
||
manifest/ main.ts interfaces.ts versions/ file-models/ actions/
|
||
orchestrator/ The control-plane app (Python)
|
||
app.py FastAPI dashboard + JSON API
|
||
jobs.py the job runner (extract → serve waves → grade → adjudicate → score)
|
||
serving.py vLLM + LiteLLM router on the Sparks, in waves
|
||
graders.py launch the grading panel
|
||
adjudicator.py the local lead grader
|
||
extraction.py PDF/DOCX/TXT/MD → text (on the StartOS box)
|
||
preflight.py probe models before launching graders
|
||
spark_client.py SSH/rsync helpers
|
||
bdef.md the baked-in BDEF v1.1 rubric
|
||
bm_config.py config defaults (mirrors startos/file-models/config.ts)
|
||
sandbox/ grader image (built ON the Spark, not packed in the s9pk)
|
||
grader_agent.py grader.Dockerfile build.sh
|
||
```
|
||
|
||
## Build
|
||
|
||
GitHub CI (`.github/workflows/build.yml`) or a local build with `start-cli`
|
||
(see the s9pk-build-on-mac recipe). The vLLM and grader images are built **on
|
||
the Sparks**, not packed into the `.s9pk`.
|
||
|
||
```
|
||
npm ci && npm run check && npm run build # type-check + bundle
|
||
make # pack the .s9pk (needs start-cli)
|
||
```
|
||
|
||
## Status
|
||
|
||
v0.1 — source complete, `tsc`-clean and Python-syntax-clean. Not yet validated
|
||
against live Sparks. HF model pre-pull on the head Spark is required for
|
||
air-gapped runs.
|