c7ce44d963
Workstream A–C substrate for the Ten31 agentic system: - A1: docs/crm-overview.md; CLAUDE.md conventions + guardrail #9 - A2: additive/reversible core migration (canonical_entities, entity_links, interaction_log, relationship_edges, soft-delete) + ledgered runner - B1/B3: chunking + deterministic entity resolution (backend/ingest) - B2: dense (bge-m3) + BM25 sparse ingest to Qdrant crm_chunks - C: CRM MCP server (reads, retrieval modes, logged writes) — no outbound tools - docs: redaction/re-hydration, Gmail enablement runbook - synthetic test data; .env.example; housekeeping (.gitignore, untrack crm.db, drop legacy files + start9/0.3.5) Verified end-to-end on synthetic data + live Sparks (hybrid > dense on entity queries). Real backfill runs on Ten31 infra; index holds synthetic data only. Branch snapshot also captures pre-existing working-tree changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
232 lines
8.0 KiB
Markdown
232 lines
8.0 KiB
Markdown
# Ten31 Database — StartOS 0.4 deployment guide
|
|
|
|
This guide walks through building the `ten-database` 0.4 service package and
|
|
sideloading it onto an x86_64 StartOS 0.4 beta machine.
|
|
|
|
The `start9/0.4/` folder is intentionally self-contained. It does not share
|
|
any files with `start9/0.3.5/`, so the legacy package stays intact and can
|
|
be rebuilt later if needed.
|
|
|
|
---
|
|
|
|
## 0 — How data preservation works
|
|
|
|
Starting with **0.1.0:40**, this package no longer ships a seed snapshot. The
|
|
0.3.5 → 0.4 migration is complete and the live `/data` volume on the StartOS
|
|
host is the sole source of truth.
|
|
|
|
Key facts:
|
|
|
|
- StartOS preserves the `main` volume across sideloads. Reinstalling a new
|
|
`.s9pk` does **not** touch `/data/crm.db`, `/data/backups/`, or
|
|
`/data/.crm-secret`. Live edits made between releases are kept.
|
|
- Only `Uninstall` from the StartOS UI destroys the `main` volume. As long
|
|
as you only `Stop → Sideload new .s9pk → Start`, your data persists.
|
|
- Use StartOS-level **Backups → Create Backup** for full volume snapshots,
|
|
and the in-app **Settings → Admin → Run Backup** for JSON exports under
|
|
`/data/backups/`.
|
|
|
|
Container paths (unchanged from 0.3.5):
|
|
|
|
- `/data/crm.db` — primary SQLite DB (WAL journal mode)
|
|
- `/data/backups/` — JSON exports
|
|
- `/data/.crm-secret` — JWT signing key (kept across restarts so sessions stick)
|
|
|
|
> Historical note: `0.1.0:39` shipped a baked-in seed snapshot and a
|
|
> first-boot copy guarded by `! -f /data/crm.db`. That code path was removed
|
|
> in `0.1.0:40`. If you ever need to bootstrap a fresh host again, sideload
|
|
> `0.1.0:39` first, let it seed, then upgrade to the latest.
|
|
|
|
---
|
|
|
|
## 1 — Build-machine prerequisites
|
|
|
|
The 0.4 build runs on any machine with:
|
|
|
|
- Node.js ≥ 20 and npm
|
|
- Docker with buildx enabled (Docker Desktop on macOS works; Linux Docker
|
|
must have the `buildx` plugin)
|
|
- `start-cli` (Start9 SDK) — install per
|
|
https://docs.start9.com/packaging/0.4.0.x/environment-setup.html
|
|
- `jq`, `make`, `s3cmd` (s3cmd only if you also plan to `make publish`)
|
|
|
|
Recommended one-time setup:
|
|
|
|
```sh
|
|
# Initialize the Start9 developer key (run once per build machine)
|
|
start-cli init-key
|
|
|
|
# Create ~/.startos/config.yaml so `make install` can sideload:
|
|
cat > ~/.startos/config.yaml <<'YAML'
|
|
# Replace with the hostname of your 0.4 beta node
|
|
host: http://start9.local
|
|
YAML
|
|
```
|
|
|
|
---
|
|
|
|
## 2 — Build the x86_64 .s9pk
|
|
|
|
From the repo root:
|
|
|
|
```sh
|
|
cd start9/0.4
|
|
|
|
# One-time dependency install (pulls start-sdk + friends):
|
|
npm ci
|
|
|
|
# Clean build (produces ten-database_x86_64.s9pk):
|
|
make clean
|
|
make x86
|
|
```
|
|
|
|
Output:
|
|
|
|
- `ten-database_x86_64.s9pk` in `start9/0.4/`
|
|
- Build summary printed by s9pk.mk (title, version, arch, SDK version,
|
|
git hash)
|
|
|
|
> Note: `make` by default builds x86, arm, and riscv. The `Makefile`
|
|
> in this folder overrides `ARCHES := x86` so only x86_64 is produced.
|
|
> If you later need arm64 too, switch to `ARCHES := x86 arm`.
|
|
|
|
### If the build fails
|
|
|
|
Common causes and fixes:
|
|
|
|
- **`.git/HEAD` or `.git/index` missing** — s9pk.mk requires a real git
|
|
repo. It looks at `../../.git` relative to `start9/0.4/` (i.e. the repo
|
|
root). Make sure you're building inside the actual repo.
|
|
- **`start-cli not found`** — install the Start9 SDK CLI.
|
|
- **docker buildx error** — run `docker buildx create --use` once.
|
|
- **Permission denied removing `javascript/` between builds** — macOS
|
|
extended attributes can make ncc output files immutable. Run
|
|
`chmod -R u+w start9/0.4/javascript` and retry, or just `rm -rf
|
|
start9/0.4/javascript` from Finder.
|
|
|
|
---
|
|
|
|
## 3 — Sideload onto the StartOS 0.4 beta node
|
|
|
|
Two options:
|
|
|
|
### Option 1 — `make install` (uses ~/.startos/config.yaml)
|
|
|
|
```sh
|
|
cd start9/0.4
|
|
make install
|
|
```
|
|
|
|
This runs `start-cli package install -s ten-database_x86_64.s9pk` against
|
|
whatever host you set in `~/.startos/config.yaml`.
|
|
|
|
### Option 2 — StartOS web UI
|
|
|
|
1. Copy `ten-database_x86_64.s9pk` onto a machine that can reach the
|
|
StartOS 0.4 UI.
|
|
2. In the UI: **System → Sideload Service → pick the .s9pk → Install.**
|
|
3. After the install completes, open the service and click **Start**.
|
|
|
|
### First-boot verification
|
|
|
|
After upgrading the service:
|
|
|
|
1. Open the Ten31 Database UI from the Interfaces page.
|
|
2. Log in with your existing account — passwords and sessions persist
|
|
because `/data/.crm-secret` is preserved.
|
|
3. Spot-check a few rows in the fundraising grid against what you saw
|
|
before the upgrade.
|
|
4. Run one manual backup (Settings → Admin → Run Backup) to confirm the
|
|
app's write path works.
|
|
|
|
---
|
|
|
|
## 4 — Rollback plan
|
|
|
|
If a new sideload misbehaves:
|
|
|
|
1. **Stop** the service in StartOS — do not Uninstall (that deletes the
|
|
`main` volume).
|
|
2. Sideload the previous `.s9pk` (keep one around) and Start.
|
|
3. Investigate by opening the service logs from the StartOS UI.
|
|
|
|
For full disaster recovery, restore the `main` volume from a StartOS-level
|
|
Backup.
|
|
|
|
---
|
|
|
|
## 5 — File map (what lives where)
|
|
|
|
```
|
|
start9/0.4/
|
|
├── DEPLOY_040.md # this file
|
|
├── README.md # short overview
|
|
├── Dockerfile # self-contained; refs only start9/0.4/ paths
|
|
├── Makefile # thin override: ARCHES := x86
|
|
├── s9pk.mk # shared 0.4 build plumbing (do not edit)
|
|
├── package.json, -lock.json # start-sdk + build tooling
|
|
├── tsconfig.json
|
|
├── docker_entrypoint.sh # ensures /data dirs + JWT secret, starts server.py
|
|
├── healthcheck.sh # curl /api/health (diagnostics only)
|
|
├── icon.svg # service icon
|
|
├── LICENSE
|
|
├── refresh_seed.sh # (LEGACY) scp helper from 0.3.5; kept for reference
|
|
├── assets/
|
|
│ └── ABOUT.md # user-facing install description
|
|
├── seed/ # (LEGACY) historical seed snapshot, NOT shipped
|
|
│ ├── README.md
|
|
│ └── data/ # crm.db + backups from initial 0.3.5 → 0.4 cut
|
|
└── startos/ # SDK source (manifest, main, interfaces…)
|
|
├── index.ts # SDK entry (no edits normally needed)
|
|
├── sdk.ts # typed SDK instance
|
|
├── utils.ts # shared constants
|
|
├── i18n.ts # simple passthrough
|
|
├── manifest/
|
|
│ ├── index.ts # id, title, images, arches, volumes, alerts
|
|
│ └── i18n.ts # localized short/long description
|
|
├── versions/
|
|
│ ├── index.ts # versionGraph wiring
|
|
│ ├── v0.1.0.39.ts # first 0.4 release (with seed)
|
|
│ └── v0.1.0.40.ts # current release (seed removed)
|
|
├── init/index.ts # setupInit ordering
|
|
├── main.ts # daemon + health check
|
|
├── interfaces.ts # HTTP interface on port 8080
|
|
├── backups.ts # Backups.ofVolumes('main')
|
|
├── dependencies.ts # (none)
|
|
└── actions/index.ts # (none)
|
|
```
|
|
|
|
The `seed/` directory and `refresh_seed.sh` are no longer referenced by the
|
|
build and can be deleted from the repo at any time. They are kept on disk
|
|
purely as a historical snapshot of the data that was migrated off the 0.3.5
|
|
host on first cutover.
|
|
|
|
---
|
|
|
|
## 6 — Things to remember
|
|
|
|
- Package id stays `ten-database` across both 0.3.5 and 0.4 so there is
|
|
exactly one service to manage on each host.
|
|
- The service volume id is `main` on both sides and mounts at `/data`
|
|
inside the container. This is what makes data preservation trivial.
|
|
- The 0.4 release is x86_64 only. If you later deploy to aarch64, change
|
|
`ARCHES` in the Makefile and rebuild.
|
|
- The built `.s9pk` is not committed — treat it as a build artifact.
|
|
`.gitignore` already ignores `*.s9pk` and `javascript/`.
|
|
- If you change anything under `startos/`, run `npm run check` (tsc) and
|
|
`npm run build` (ncc) before re-packaging.
|
|
|
|
---
|
|
|
|
## 7 — Quick cheat sheet
|
|
|
|
```sh
|
|
cd start9/0.4
|
|
make clean
|
|
make x86
|
|
make install
|
|
```
|
|
|
|
After `make install` completes, open the service in the StartOS UI,
|
|
hit Start, and verify the app still works.
|