Implement BDEF v1.1 grading: scoring core, per-deck pipeline, ledger, dashboard, StartOS layer

- Deterministic scoring.py (quant 60 / qual 40 / flags -15, profitability heaviest)
- Per-company JSON ledger with forecast-target chaining deck N-1 -> N
- Single-shot sandbox agent with guided-JSON fallback ladder (no tool loop)
- Portfolio dashboard with sparklines, KPI hit rates, BDEF category bars
- 48 unit tests green; endpoints smoke-tested; npm check+build green

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jonathan Kirkwood
2026-07-06 14:15:12 -05:00
co-authored by Claude Fable 5
parent 1dde915540
commit b1d7aed9f4
48 changed files with 4907 additions and 971 deletions
+46 -23
View File
@@ -1,13 +1,14 @@
# Boardroom Map
Drop confidential documents in and convene a **panel of local LLMs** running on
your DGX Sparks to review them. You choose the models, the personas (lenses), and
how many reviews. An optional **local lead reviewer** synthesizes the panel into
one consolidated report. There is **no frontier model and no cloud key** — in the
default air-gapped mode the documents and their reviews never leave your hardware.
Drop portfolio-company board decks in and have a **panel of local LLMs** running
on your DGX Sparks grade them against the **BDEF v1.1 framework** (Girdley +
Munger/Buffett). A deterministic scorer turns the panel's grades into a 0100
composite and appends it to each company's **running scorecard ledger**; the web
dashboard shows the trends. There is **no frontier model and no cloud key** — in
the default air-gapped mode your confidential decks never leave your hardware.
Boardroom Map is a *control plane*: nothing serves or runs on your StartOS box (it only
SSHes to the Sparks and extracts document text on CPU).
Boardroom Map is a *control plane*: nothing serves or runs on your StartOS box
(it only SSHes to the Sparks and extracts deck text on CPU).
## Setup (run the Actions in order)
@@ -16,32 +17,54 @@ SSHes to the Sparks and extracts document text on CPU).
2. **Configure Models** — the catalog of local models to serve (alias → HF id →
which Spark → port), and serving knobs. For air-gapped runs every model must be
on the **head Spark** and present in its HF cache.
3. **Configure Reviewers** — the panel: one entry per review, each a model + a
persona (the lens it reads through) + an optional temperature.
4. **Configure Review** — the rubric, the **Network Mode** (air-gapped vs
local-services), synthesis on/off + lead model, and whether to wipe documents
from the Sparks afterward.
3. **Configure Graders** — the panel: one entry per grader, each a model + a
persona (the lens it grades through — e.g. a Munger inversion skeptic, a
Girdley operator, a skeptical CFO) + an optional temperature.
4. **Configure Grading** — the BDEF rubric override (empty = the built-in
BDEF v1.1), the **Network Mode** (air-gapped vs local-services), the
extractor + adjudicator models, deck retention, and the scoring weights.
5. **Configure Companies** — one entry per portfolio company: its inbox **slug**,
display name, KPI aliases, and **pinned KPI targets**. Pin the profitability
thresholds especially — profitability carries the heaviest weight.
## Running a review
## Grading decks
1. Open the **Web UI** and drag your documents (PDF / DOCX / TXT / MD) onto the
inbox (or drop them in the service's `inbox` folder).
2. Click **Run Review** (or enable *auto-run on drop*).
1. Drop each company's deck into its inbox folder, e.g.
`inbox/acme-widgets/2026-Q2-deck.pdf` (PDF / DOCX / TXT / MD), via the
**Web UI** or the service's `inbox` directory.
2. Run **Grade Decks** (or enable *auto-grade on drop*).
3. Watch the activity log. The service extracts text locally, serves the needed
models on the Sparks **in waves** (so a panel can span more models than fit in
GPU memory at once), runs each reviewer, then the lead reviewer, and saves the
reports. Read them in the Web UI or via **View Latest Report**.
GPU memory at once), runs the structured KPI extractor, then each grader, then
the adjudicator, and finally computes the composite and updates the company's
ledger. Read the results on the dashboard or via **View Latest Scorecard**.
## How the score works
The composite is **0100 = quantitative 60 + qualitative 40 red flags (max 15)**:
- **Quant 60** — profitability KPI attainment **30** (the heaviest single slice),
other measurable KPIs **20**, and **forecast integrity 10**: deck N's actuals
are chained against what deck N1 promised, so sandbagging and quietly moved
goalposts cost points.
- **Qual 40** — eight BDEF categories (AH), up to 5 points each, scored by the
panel with evidence quotes (thin evidence scales the score down).
- **Red flags** — up to **15**; silently dropped KPIs are flagged automatically,
and flags raised by only one grader are damped.
Pinned targets from **Configure Companies** are graded every quarter whether or
not the deck mentions them — a deck cannot improve its score by going quiet.
## Network modes
- **Air-gapped (default):** reviewer containers join an `--internal` Docker
- **Air-gapped (default):** grader containers join an `--internal` Docker
network — they can reach only the on-Spark model proxy, with zero internet
egress. Models are served from a pre-pulled HF cache. All models must be on the
head Spark. Strongest confidentiality.
- **Local services:** reviewers may also reach LAN services (e.g. SearXNG) and the
- **Local services:** graders may also reach LAN services (e.g. SearXNG) and the
second Spark. This network has egress unless you firewall it — use only when you
accept that reviewers can reach the network.
accept that graders can reach the network.
The original documents are extracted to plain text on the StartOS box; only that
The original decks are extracted to plain text on the StartOS box; only that
text is shipped to the Sparks, and it is wiped from the Sparks after the job (the
reports are kept on your StartOS box).
scorecards and ledgers are kept on your StartOS box).