Implement BDEF v1.1 grading: scoring core, per-deck pipeline, ledger, dashboard, StartOS layer
- Deterministic scoring.py (quant 60 / qual 40 / flags -15, profitability heaviest) - Per-company JSON ledger with forecast-target chaining deck N-1 -> N - Single-shot sandbox agent with guided-JSON fallback ladder (no tool loop) - Portfolio dashboard with sparklines, KPI hit rates, BDEF category bars - 48 unit tests green; endpoints smoke-tested; npm check+build green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jonathan Kirkwood
co-authored by
Claude Fable 5
Claude Fable 5
parent
1dde915540
commit
b1d7aed9f4
@@ -1,44 +1,75 @@
|
||||
# Boardroom Map — a private document-review panel for your DGX Sparks
|
||||
# Boardroom Map — private board-deck grading on your DGX Sparks
|
||||
|
||||
Boardroom Map is a StartOS service (`.s9pk`) that lets you **drop confidential
|
||||
documents in and have a panel of local LLMs review them** on your NVIDIA DGX
|
||||
Sparks. You pick the models and the personas (lenses), and how many reviews to
|
||||
run; each reviewer writes a report, and an optional **local lead reviewer**
|
||||
synthesizes them into one consolidated report. There is **no frontier model and
|
||||
no cloud API key** — in the default air-gapped mode the documents and reviews
|
||||
Boardroom Map is a StartOS service (`.s9pk`) that **grades portfolio-company
|
||||
board decks with a panel of local LLMs** on your NVIDIA DGX Sparks. Drop each
|
||||
company's deck into `inbox/<company-slug>/`; the panel grades it against the
|
||||
**BDEF v1.1 framework** (Girdley + Munger/Buffett), an optional local
|
||||
**adjudicator** reconciles the panel, and a deterministic Python scorer computes
|
||||
a 0–100 composite that lands on the company's **running scorecard ledger**. A
|
||||
web dashboard shows per-company trends. There is **no frontier model and no
|
||||
cloud API key** — in the default air-gapped mode the decks and their grades
|
||||
never leave your hardware.
|
||||
|
||||
It is a sibling of [Nightshift](../nightshift) and reuses the same control-plane
|
||||
It is a sibling of [Chambers](../chambers) and reuses the same control-plane
|
||||
pattern (a GPU-free orchestrator on StartOS driving the Sparks over SSH), but
|
||||
with the swarm, the git blackboard, and the Claude overseer removed and replaced
|
||||
by an on-demand **document-review pipeline**.
|
||||
swaps the free-form document-review panel for a **deterministic deck-grading
|
||||
pipeline** with pinned KPI targets and per-company ledgers.
|
||||
|
||||
## The scoring model
|
||||
|
||||
`composite (0–100) = quant 60 + qual 40 − red flags (capped at 15)`
|
||||
|
||||
- **Quantitative 60:** profitability KPI attainment **30** (heaviest slice),
|
||||
other measurable KPIs **20**, **forecast integrity 10** — deck N's actuals are
|
||||
chained against deck N−1's stated targets, so moved goalposts cost points.
|
||||
KPI credit is linear above a floor ratio (default 0.5 → zero credit below).
|
||||
- **Qualitative 40:** eight BDEF categories (A–H) × 5 points, scored by the
|
||||
panel with evidence quotes; thin evidence scales down.
|
||||
- **Red flags:** up to **−15**; silently dropped KPIs are auto-flagged (capped),
|
||||
and flags raised by a single grader are damped by 0.5.
|
||||
|
||||
Every knob lives in config (`weights`, per-company `pinnedTargets` and
|
||||
`kpiAliases`) so the model can be retuned without a rebuild.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
StartOS box (control plane, no GPU) DGX Spark(s)
|
||||
┌────────────────────────────────────┐ ┌───────────────────────────────┐
|
||||
│ FastAPI web UI + job runner │ SSH │ per-job Docker network │
|
||||
│ • inbox (drop documents) │ ───────▶│ (──internal in airgapped) │
|
||||
│ FastAPI dashboard + job runner │ SSH │ per-job Docker network │
|
||||
│ • inbox/<company-slug>/ (decks) │ ───────▶│ (──internal in airgapped) │
|
||||
│ • extract text (PDF/DOCX/TXT/MD) │ rsync │ ┌─────────┐ ┌────────────┐ │
|
||||
│ • plan model "waves" │ ───────▶│ │ vLLM(s) │◀─│ LiteLLM │ │
|
||||
│ • launch reviewer containers │ │ └─────────┘ │ router │ │
|
||||
│ • pull reports, synthesize, wipe │◀─────── │ ┌──────────────┐ ▲ │ │
|
||||
│ • reports saved here only │ rsync │ │ reviewer ×N │──┘ │ │
|
||||
│ • extractor → graders → adjudicator│ │ └─────────┘ │ router │ │
|
||||
│ • deterministic composite scorer │◀─────── │ ┌──────────────┐ ▲ │ │
|
||||
│ • per-company ledgers + scorecards │ rsync │ │ grader ×N │──┘ │ │
|
||||
└────────────────────────────────────┘ │ │ (read-only, │ │ │
|
||||
│ │ sandboxed) │ │ │
|
||||
│ └──────────────┘ │ │
|
||||
└───────────────────────────────┘
|
||||
```
|
||||
|
||||
- **Reviewers** are one-shot, read-only, hardened containers (non-root,
|
||||
- **Graders** are one-shot, read-only, hardened containers (non-root,
|
||||
`--cap-drop ALL`, read-only rootfs, no docker socket). In air-gapped mode they
|
||||
sit on an `--internal` network and can reach only the model proxy.
|
||||
- **Waves:** the job runner serves models in waves bounded by
|
||||
`maxConcurrentModels`, so a panel can span more models than fit in GPU memory
|
||||
at once.
|
||||
- **Confidentiality:** documents are extracted to text on the StartOS box; only
|
||||
- **Air-gap modes:** `airgapped` (default — graders reach only the on-Spark
|
||||
model proxy, zero egress, models pre-pulled) or `local_services` (graders may
|
||||
reach LAN services like SearXNG and the second Spark — has egress unless
|
||||
firewalled).
|
||||
- **Confidentiality:** decks are extracted to text on the StartOS box; only
|
||||
text crosses to the Sparks, and it is wiped from the Sparks after the job.
|
||||
Scorecards and ledgers live only on the StartOS box.
|
||||
|
||||
## Setup order
|
||||
|
||||
Configure Sparks → Test Spark Connection → Configure Models → Configure Graders
|
||||
→ Configure Grading (rubric, air-gap, weights) → Configure Companies (slugs,
|
||||
KPI aliases, pinned targets — especially profitability thresholds) → drop decks
|
||||
into `inbox/<company-slug>/2026-Q2-deck.pdf` → **Grade Decks** → watch the
|
||||
dashboard.
|
||||
|
||||
## Repo layout
|
||||
|
||||
@@ -46,25 +77,25 @@ StartOS box (control plane, no GPU) DGX Spark(s)
|
||||
startos/ StartOS package definition (TypeScript / start-sdk)
|
||||
manifest/ main.ts interfaces.ts versions/ file-models/ actions/
|
||||
orchestrator/ The control-plane app (Python)
|
||||
app.py FastAPI UI + JSON API
|
||||
jobs.py the job runner (extract → serve waves → review → synthesize)
|
||||
app.py FastAPI dashboard + JSON API
|
||||
jobs.py the job runner (extract → serve waves → grade → adjudicate → score)
|
||||
serving.py vLLM + LiteLLM router on the Sparks, in waves
|
||||
reviewers.py launch the reviewer panel
|
||||
synthesis.py the local lead reviewer
|
||||
graders.py launch the grading panel
|
||||
adjudicator.py the local lead grader
|
||||
extraction.py PDF/DOCX/TXT/MD → text (on the StartOS box)
|
||||
preflight.py probe models before launching reviewers
|
||||
preflight.py probe models before launching graders
|
||||
spark_client.py SSH/rsync helpers
|
||||
bdef.md the baked-in BDEF v1.1 rubric
|
||||
bm_config.py config defaults (mirrors startos/file-models/config.ts)
|
||||
sandbox/ reviewer image (built ON the Spark, not packed in the s9pk)
|
||||
grader_agent.py reviewer.Dockerfile build.sh
|
||||
openclaw/ what each Spark needs provisioned (OpenClaw's job)
|
||||
sandbox/ grader image (built ON the Spark, not packed in the s9pk)
|
||||
grader_agent.py grader.Dockerfile build.sh
|
||||
```
|
||||
|
||||
## Build
|
||||
|
||||
Same path as Nightshift — GitHub CI (`.github/workflows/build.yml`) or a local
|
||||
build with `start-cli` (see the s9pk-build-on-mac recipe). The vLLM and reviewer
|
||||
images are built **on the Sparks**, not packed into the `.s9pk`.
|
||||
GitHub CI (`.github/workflows/build.yml`) or a local build with `start-cli`
|
||||
(see the s9pk-build-on-mac recipe). The vLLM and grader images are built **on
|
||||
the Sparks**, not packed into the `.s9pk`.
|
||||
|
||||
```
|
||||
npm ci && npm run check && npm run build # type-check + bundle
|
||||
@@ -74,5 +105,5 @@ make # pack the .s9pk (needs start-cli)
|
||||
## Status
|
||||
|
||||
v0.1 — source complete, `tsc`-clean and Python-syntax-clean. Not yet validated
|
||||
against live Sparks. See `openclaw/OPENCLAW_SPEC.md` for the Spark-side
|
||||
provisioning (HF model pre-pull is required for air-gapped runs).
|
||||
against live Sparks. HF model pre-pull on the head Spark is required for
|
||||
air-gapped runs.
|
||||
|
||||
Reference in New Issue
Block a user