FRAME-SERV-2 — AI INFERENCE BUILD
This site records every change made to this machine (a friend's Framework Desktop) during its setup as a large-model AI inference rig. Every entry in the change log states what was done, why, and exactly how to reverse or tweak it. Nothing on the machine is undocumented.
The Machine
What this site contains
System Audit
The as-found state of the machine (2026-09-09, before any work): hardware, OS, drivers, services, security posture, storage. The baseline the change log measures against.
Read the audit →Architecture
What was built and why: inference backend, model server topology, API endpoints, where weights live, how services start and stop.
Architecture →Models
Which models were selected and why — quality/speed trade-offs, memory budgets on 128G unified, quant choices, tool-use / vision / embedding support.
Models →Benchmarks
Measured numbers, not claims: tokens/s per model, prompt throughput, context behavior, and the exact commands to reproduce every number.
Benchmarks →Runbook / How to Use
The owner's operating manual: start/stop serving, hit the API, swap models, and — equally important — how to use this documentation site itself.
Runbook →Change Log
Every modification to the machine in chronological order, each with a rollback procedure. If anything here looks wrong, this page is the receipt.
Change log →Ground rules of this build
- Documented-or-didn't-happen. Every command that changes system state is logged here with its evidence file and a reversal path.
- Reversible by default. Config files are backed up alongside their changes (dot-suffixed names ignored by their consumers); service changes are unit-file drop-ins, not edits of vendor files.
- Model weights are the only large artifacts. They live on
the dedicated 2TB disk (
/srv/llm/models) so wiping or replacing models never touches the OS. - Benchmarks are recorded, not remembered. Every performance claim on the benchmarks page links to the raw command that produced it.
Built 2026-09-09 · documentation of record for this machine · served from Cloudflare Pages