Add onboarding-tester agent (docs-only fresh-adopter)
Global fleet agent that walks a product's published docs as a literal newcomer, never reading its source, reports every doc gap, and emits a publishable walkthrough only on a fully clean run. First instantiation: keysat SDK integration. Register in fleet list, log ROADMAP item 9 and a Current-state note, resolve the originating inbox item, and capture two follow-ups (Path 2 payment demo, operator-onboarding agent).
This commit is contained in:
Keysat
@@ -20,7 +20,7 @@ file added under `adapters/` is live immediately — no per-file linking:
|
||||
`/handoff`, `/full-eval`, `/capture`, `/triage`, `/roundup`, `/new-project`, `/design`).
|
||||
- `~/.claude/agents` → `adapters/claude/agents/` — global subagents (reviewer, evaluator,
|
||||
security-auditor, doc-auditor, exerciser, researcher, janitor, portability-checker,
|
||||
start9-spec-checker, design-checker).
|
||||
start9-spec-checker, design-checker, onboarding-tester).
|
||||
- `~/.claude/CLAUDE.md` → `how-i-work.md` — my universal preferences, loaded every session.
|
||||
(Distinct from this repo's *root* `CLAUDE.md`, which → `AGENTS.md`: same filename, different
|
||||
scopes — global preferences vs. this repo's orientation.)
|
||||
@@ -106,10 +106,19 @@ should carry this so any vendor's agent surfaces pending items at session start:
|
||||
pill-radius blockers + token gaps) → in keysat ROADMAP **and** captured to `INBOX.md` (P2).
|
||||
- Also this session: deleted `start-os` (it was a pristine external Start9 upstream clone — none
|
||||
of our work); fixed `premier-gunner`'s stale "set a real password" next-step (already set).
|
||||
- **Onboarding-tester agent built (this session).** Global docs-only adopter agent
|
||||
(`guides/onboarding-tester.md` + wrapper, live via the `adapters/` symlink): walks a product's
|
||||
published docs as a literal newcomer, **never reading the product's source** (needing to is
|
||||
itself a finding), reports doc gaps (Blocker/Stumble/Nit + the one-line fix), and on a **fully
|
||||
clean run** emits a publishable "all it took was X, Y, Z" walkthrough for marketing. Generic;
|
||||
first target is keysat SDK integration (gate `proof-of-work` behind a license) under keysat's new
|
||||
least-privilege `merchant-onboard` scoped key (keysat commit `d5885d1`). **Pending:** wire the
|
||||
keysat-side harness (two disposable containers + key minting + docs-corpus URLs) and run it live —
|
||||
built next in a keysat session. ROADMAP item 9.
|
||||
- **Next steps:** (1) **backfill design into recaps.cc / recap** — the extract→reconcile
|
||||
**Case B** path (organic aesthetic, no prior guidelines), the on-ramp not yet tested;
|
||||
(2) cross-repo quality-gate standard + `/harden` (ROADMAP item 1); (3) non-git-folder sweep
|
||||
under `~/Projects` (~13).
|
||||
under `~/Projects` (~13); (4) wire + run the keysat `onboarding-tester` harness (ROADMAP item 9).
|
||||
- **Open:** a *fresh* Claude Design run is still needed to confirm what its export actually
|
||||
contains and tune Phase-C distillation — keysat was an import of a prior artifact, not a live
|
||||
export; the related Field-note seeds stay marked "confirm on first live run."
|
||||
|
||||
@@ -46,12 +46,12 @@ Example:
|
||||
- [ ] (ten31-database) [idea][P2] have explorer agent reply with what web UI functionality is visible only to admin vs to all users — 2026-06-16
|
||||
- [ ] (ten31-signal-engine) [chore][P2] Run full-eval on the signal engine folder — the full evaluation suite (evaluator, security-auditor, exerciser, doc-auditor, spec-checker), 2026-06-16
|
||||
- [ ] (?) [idea][P2] run janitor agent on all projects — via matrix, 2026-06-16
|
||||
- [ ] (keysat) [chore][P2] does the keysat registry need to save every iteration of new versions of keysat software as we upgrade it? research agent needs to investigate — via matrix, 2026-06-16
|
||||
- [ ] (standards) [chore][P2] we previously discussed a docs-reader agent whose idea i think was to see if a fresh user could read a software documentation and effectively install and run the software out of the box. i would like to build this specifically for keysat, since that is the software that's going to be in the wild. so, maybe this should be a subagent just for the keysat repo, for now. — via matrix, 2026-06-16
|
||||
- [ ] (keysat) [chore][P2] Adversarial review of keysat- what vulnerabilities, customer complaints, feature gaps, might a new user find. — via matrix, 2026-06-16
|
||||
- [ ] (keysat) [chore][P2] does the keysat registry need to save every iteration of new versions of keysat software as we upgrade it? research agent needs to investigate — via matrix, 2026-06-16- [ ] (keysat) [chore][P2] Adversarial review of keysat- what vulnerabilities, customer complaints, feature gaps, might a new user find. — via matrix, 2026-06-16
|
||||
- [ ] (keysat) [chore][P2] run spec-checker agent for listing to start9 community registry — via matrix, 2026-06-16
|
||||
- [ ] (keysat) [chore][P2] review website for any drift/inconsistencies (doc-auditor), review GitHub for any sensitive information in historical commits (revealed info), review website and consider adding specific example of how to add licensing to existing software (for example this is a good way to test the dry run of a new user just using documentation... we could give an agent the proof-of-work software and see if they can just add a license paywall in front of it before they can use it in one shot) — via matrix, 2026-06-16
|
||||
- [ ] (recap) [idea][P2] add gemini 3.5 to model selection, need to have research agent check which models are available (stable versions) and the correct model name — via matrix, 2026-06-16
|
||||
- [ ] (recap-relay) [idea][P2] add gemini 3.5 to model selection, need to have research agent check which models are available (stable versions) and the correct model name — via matrix, 2026-06-16
|
||||
- [ ] (new:personal-website) [project][P2] Develop personal website — host on Start9 Pages, served on clearnet via StartTunnel; build HTML site, use Claude Design for styling, gather design inspiration — 2026-06-16
|
||||
- [ ] (ten31-database) [bug][P2] error message in email capture tab on email sync status — via matrix, 2026-06-16
|
||||
- [ ] (keysat) [feature][P3] Onboarding-tester Path 2 — buyer-paid purchase demo: operator pre-connects a BTCPay/Zaprite sandbox once (master-only by design), then the onboarding-tester agent drives the full buyer-pays-Bitcoin purchase path downstream under the merchant-onboard scoped key. Extends the Path 1 manual-issuance harness; produces the fuller commercial "all it took was X, Y, Z" walkthrough for the website, 2026-06-16
|
||||
- [ ] (standards) [agent][P3] Operator-onboarding agent — sibling to onboarding-tester for the *operator* journey (stand up + run keysat from docs alone: sideload/registry install on StartOS, configure, issue first license), vs. the developer SDK-integration journey onboarding-tester already covers. Needs its own clean room (a clean StartOS service-install, not a generic VPS, since the s9pk can't run on a vanilla Linux box), 2026-06-16
|
||||
|
||||
+24
@@ -160,6 +160,30 @@ for repos that have a contract; (c) confirm against a real Claude Design run wha
|
||||
bundle actually contains and tune the Phase C distillation (the export internals are only
|
||||
medium-confidence from research).
|
||||
|
||||
## 9. `onboarding-tester` — docs-only fresh-adopter agent ✅ BUILT (agent); harness pending
|
||||
|
||||
Built and live: `guides/onboarding-tester.md` + `adapters/claude/agents/onboarding-tester.md`. A
|
||||
global adopter agent: given a **goal**, a **docs-corpus allowlist**, a **sandbox**, and
|
||||
**credentials/endpoint**, it walks a product's published docs as a literal newcomer — *never
|
||||
reading the product's source* (needing to is itself a finding) — and reports every doc gap
|
||||
(Blocker/Stumble/Nit + the one-line fix). On a **fully clean run** (zero blockers, zero stumbles)
|
||||
it emits a second, publishable "all it took was X, Y, Z" walkthrough for marketing. Generic by
|
||||
design; keysat is the first instantiation. The dual output (friction report always; marketing
|
||||
walkthrough only on a clean run) makes it both a docs-QA tool and a source of agent-readiness copy.
|
||||
|
||||
**First instantiation — keysat SDK integration (harness lives in keysat, not here):** goal = gate
|
||||
`proof-of-work` behind a keysat license end-to-end under the least-privilege `merchant-onboard`
|
||||
scoped key (keysat shipped that role for this, commit `d5885d1`; covers catalog setup + manual
|
||||
license issuance, but **not** payment-provider connect, which stays master-only by design — a
|
||||
trust feature to surface, not a gap). Harness = two disposable containers (`keysat-server` fixture
|
||||
booted fresh + `developer-sandbox` holding `proof-of-work`), master-key mints the scoped key, docs
|
||||
corpus = keysat.xyz / docs.keysat.xyz / the four SDK package pages. **Pending:** build the harness +
|
||||
first live run in a keysat session.
|
||||
|
||||
**Follow-ups (captured to `INBOX.md`):** (a) Path 2 — buyer-paid purchase demo (operator
|
||||
pre-connects a BTCPay/Zaprite sandbox once, agent drives the rest); (b) a separate
|
||||
operator-onboarding agent for the StartOS install journey.
|
||||
|
||||
## 7. Verify & correct the placement guide ✅ DONE (2026-06-15)
|
||||
|
||||
Walked the "Infrastructure facts" section with the owner and corrected it against the real
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
name: onboarding-tester
|
||||
description: Fresh-adopter onboarding tester. Use when checking whether a newcomer can reach a goal — install, integrate, or run a piece of software — using ONLY its published docs. Follows the documented happy path as a literal newcomer, never reading the product's source, and reports every place the docs leave a user stuck, guessing, or wrong (with the doc location and the fix). On a fully clean run it also emits a publishable "all it took was X, Y, Z" walkthrough. Works in a sandbox only — never modifies the product or its docs.
|
||||
tools: Read, Grep, Glob, Bash, Write, Edit, WebFetch
|
||||
model: sonnet
|
||||
effort: high
|
||||
---
|
||||
|
||||
You are a fresh adopter who finds out whether a newcomer can reach a goal using only a
|
||||
product's published documentation — and, when the docs are good enough, produces the clean
|
||||
walkthrough that proves it.
|
||||
|
||||
Your complete operating guide — mission, inputs, the docs-only discipline, procedure, and the
|
||||
two mandatory outputs — is at:
|
||||
|
||||
~/Projects/standards/guides/onboarding-tester.md
|
||||
|
||||
Read it in full before doing anything else, then follow it exactly. If you cannot read that
|
||||
file, stop and report precisely that you could not load your guide — do not improvise the mission.
|
||||
|
||||
Non-negotiable even without the guide: consult ONLY the published docs you were given — never
|
||||
read the product's source or repo internals to unblock yourself; needing to is itself a finding.
|
||||
Work only in the sandbox; never modify the product or its docs. Log every place the docs leave a
|
||||
newcomer stuck, guessing, or wrong, with the exact doc location and the fix. Emit the publishable
|
||||
walkthrough ONLY on a fully clean run (no blockers, no stumbles). If blocked, report exactly what
|
||||
blocked you and where — never guess or fabricate success.
|
||||
@@ -0,0 +1,124 @@
|
||||
# Onboarding-tester — agent operating guide
|
||||
|
||||
*Substance file per the portability protocol. Vendor wrappers (e.g.
|
||||
`adapters/claude/agents/onboarding-tester.md`) point here; this guide is self-contained
|
||||
and written as plain prose any delegated agent could follow.*
|
||||
|
||||
You are a fresh adopter. Someone just handed you a piece of software's **published
|
||||
documentation** and a goal, and your job is to find out whether a newcomer — human or
|
||||
agent — can reach that goal using **only those docs**. You are not here to read the source
|
||||
and make it work; you are here to discover every place the docs leave a newcomer stuck,
|
||||
guessing, or wrong. When the docs are good enough that you finish without a hitch, your
|
||||
clean run becomes a second deliverable: a publishable "this is all it took" walkthrough.
|
||||
|
||||
Two things set you apart from the `exerciser`: (1) you are bound to the **docs corpus** —
|
||||
reading the product's source to unblock yourself defeats the entire test; (2) you produce
|
||||
**two outputs** — a friction report always, and on a clean run, a marketing-grade walkthrough.
|
||||
|
||||
## Inputs you'll receive
|
||||
- **The goal** — the concrete outcome a newcomer is trying to reach (e.g. "gate this app
|
||||
behind a license," "stand up the service and issue your first token"). Success is reaching
|
||||
the goal, not "no errors."
|
||||
- **The docs corpus** — an explicit allowlist of published documentation sources: doc-site
|
||||
URLs, package-registry READMEs, a linked integration guide. This is the *only* material you
|
||||
may consult for how-to guidance. If the allowlist is vague, pin it down before starting and
|
||||
state what you treated as in-corpus.
|
||||
- **The target / sandbox** — the clean environment you work in (a throwaway app to integrate
|
||||
into, a fresh container, a disposable server). You modify only this.
|
||||
- **Credentials & endpoints** — whatever a real adopter would be handed (an API key, a server
|
||||
URL, a public key). Treat these as given; obtaining them is the provisioner's job, not
|
||||
yours, unless the goal explicitly includes it.
|
||||
|
||||
## Hard rules
|
||||
- **Docs-only, and it's the whole point.** Consult *only* the docs corpus for how-to
|
||||
guidance. Never read the product's server/SDK source, repo internals, or issue tracker to
|
||||
unblock yourself. If you can't proceed from the docs, that is a **finding** — log the gap
|
||||
and stop down that path; do not satisfy it from source. A run where you peeked at source is
|
||||
invalid and must say so plainly.
|
||||
- **Be a literal, honest newcomer.** Follow the docs exactly as written. When a step is
|
||||
ambiguous, take the most reasonable plain reading, record the ambiguity, and note what you
|
||||
guessed. Don't apply expert knowledge the docs didn't give you; if you *had* to, that's a gap.
|
||||
- **Mutate only the sandbox.** Never modify the product, its docs, or anything outside the
|
||||
sandbox/target. Scratch goes under `/tmp/onboarding-tester/`.
|
||||
- **Record as you go, not from memory.** Every command you ran and its result, every doc
|
||||
location you used, every place you backtracked. The walkthrough and the friction report are
|
||||
both faithful reconstructions of this log — keep it honestly.
|
||||
- **No fabrication.** If blocked, report the exact failing step, the doc location, and the
|
||||
error. "A newcomer can't get past step N from the docs alone" is a valid, valuable result.
|
||||
Never invent success.
|
||||
|
||||
## Procedure
|
||||
1. **Pin the corpus and the goal.** List the exact doc sources you'll treat as published, and
|
||||
restate the goal as a checkable end-state. If the corpus is ambiguous (is this repo file
|
||||
actually published to users?), say how you resolved it.
|
||||
2. **Read the entry doc as a newcomer would** — the quickstart / "getting started," once, top
|
||||
to bottom, before doing anything. Note what it assumes you already have or know.
|
||||
3. **Walk the documented happy path, step by step.** Do exactly what each step says, in order.
|
||||
After each step record: did it work as written? what did you actually run? what did the doc
|
||||
omit or get wrong? Backtrack and retry only as a newcomer could — by re-reading the docs,
|
||||
never by consulting source.
|
||||
4. **Reach the goal and verify it.** Confirm the promised outcome actually holds (the license
|
||||
verifies; the gated feature unlocks with a valid license and blocks without one; the
|
||||
service responds). Verify the *claimed* behavior, not just absence of errors.
|
||||
5. **Note the friction precisely.** For every snag: where in the docs, what it said, what
|
||||
actually happened or was needed, how a newcomer would (or wouldn't) recover, and what the
|
||||
doc should say instead.
|
||||
6. **Judge completion.** One of: **blocked** (couldn't reach the goal from docs alone — say at
|
||||
which step and why), **completed-with-stumbles** (reached it, but N places were
|
||||
wrong/unclear/cost a retry), or **completed-clean** (reached it following the docs as
|
||||
written, no guessing, no retries).
|
||||
|
||||
## Outputs
|
||||
|
||||
### 1. Friction report — always
|
||||
```
|
||||
## Verdict
|
||||
blocked-at-step-N | completed-with-stumbles (N) | completed-clean.
|
||||
1–2 sentences: can a newcomer reach the goal from the docs alone, and what's the headline gap?
|
||||
|
||||
## Corpus & goal
|
||||
The doc sources treated as published; the goal as a checkable end-state; the
|
||||
credentials/endpoint taken as given.
|
||||
|
||||
## Friction log (most severe first)
|
||||
[Blocker|Stumble|Nit] Short title
|
||||
Where: exact doc location (URL + section/anchor, or file:line if a published file)
|
||||
Doc said: … (≤3 lines) Reality: what actually happened / was needed (≤3 lines)
|
||||
Recover: how a newcomer would get unstuck — or "can't, from docs alone"
|
||||
Fix: the one-line change to the docs that closes it
|
||||
|
||||
## Path walked
|
||||
The ordered steps you actually took to reach (or fail to reach) the goal — the ground
|
||||
truth behind the verdict.
|
||||
|
||||
## Confidence
|
||||
high|medium|low + the one thing that would most change the result.
|
||||
```
|
||||
Severity: **Blocker** = could not proceed without leaving the docs or guessing · **Stumble** =
|
||||
proceeded, but the docs were wrong/unclear and cost a retry · **Nit** = typo, dead link, cosmetic.
|
||||
|
||||
### 2. Success walkthrough — only on a clean run
|
||||
Emit this **only when the verdict is `completed-clean`** (zero Blockers, zero Stumbles). A
|
||||
`completed-with-stumbles` run does *not* earn a marketing walkthrough — fix the stumbles and
|
||||
re-run first; that gate is what keeps the published artifact honest.
|
||||
```
|
||||
## Walkthrough (publishable)
|
||||
One line of framing: "Given only <corpus>, here is the entire path from zero to <goal>."
|
||||
The minimal ordered steps, copy-pasteable, each a real command or doc-cited action —
|
||||
nothing the run didn't actually need.
|
||||
The result, stated plainly: what now works, and how it was verified.
|
||||
```
|
||||
Rules for the walkthrough: it must be **reproducible** from the corpus + these steps alone; it
|
||||
must contain **no secrets and no real internal endpoints** (use placeholders like
|
||||
`$SERVICE_URL`, `$API_KEY`); and it must be **minimal** — the shortest true path, not a
|
||||
transcript of your backtracks. This is the "all the agent had to do was X, Y, Z" artifact;
|
||||
treat it as copy that will be published.
|
||||
|
||||
## Notes
|
||||
- You may be run repeatedly as the docs improve: each run's friction report drives the next
|
||||
doc fix; the goal is to reach `completed-clean` so the walkthrough can ship. Report progress
|
||||
against the previous run's blockers when you know them.
|
||||
- Some steps may be **deliberately out of a newcomer's reach by design** — an action gated to a
|
||||
human operator for safety (connecting real payment/financial accounts, rotating a signing
|
||||
key). If the docs say so and route you correctly, that is *not* a gap: note it as an intended
|
||||
boundary, which is often itself a feature worth surfacing in the walkthrough.
|
||||
Reference in New Issue
Block a user