Change Log

The staged reranker GGUF is named BGE-Reranker-v2-M3-Q8_0.gguf (capital BGE), but the unit referenced lowercase bge-reranker-v2-m3-Q8_0.gguf — it would have failed on first start. Unit corrected to:

-m /srv/llm/models/bge-reranker-v2-m3-GGUF/BGE-Reranker-v2-M3-Q8_0.gguf

sed -i 's|BGE-Reranker-v2-M3-Q8_0.gguf|bge-reranker-v2-m3-Q8_0.gguf|' \
  ~/.config/systemd/user/llama-rerank.service

The hotspot could not sustain the 13 MB GitHub download (three attempts), so the binary was relayed from ashborn-prime — the exact build v248 that prime runs in production, rather than the just-released v255. Deliberate choice: production-proven beats newest.

rm /srv/llm/bin/llama-swap   # or replace with a newer release build

The 17 GB main GGUF failed three times over the cell hotspot (hf_transfer CAS reconstruction error, then stalled plain-mode downloads). Completed over hotspot: mmproj-F16 vision (885 MB) + MTP draft (1.3 GB). The main GGUF is identical on ashborn-prime and will arrive on the operator's USB stick with the other staged models. Stale partial blobs were cleared.

N/A — no machine state changed beyond removing partial downloads.
Restart hotspot download any time:
  /srv/llm/venv/bin/hf download unsloth/Qwen3.8-27B-GGUF \
    Qwen3.8-27B-UD-Q4_K_XL.gguf --local-dir /srv/llm/models/Qwen3.8-27B-GGUF

By default amdgpu caps GPU-visible memory (GTT) at ~50% of RAM (~63 GiB) — the reason 100 GB models fail to load on 128 GiB machines. Added to the kernel command line (verified present in /boot/grub2/grub.cfg):

amdgpu.gttsize=126976 ttm.pages_limit=32505856

Why: the two values form an inseparable pair describing the same 124 GiB pool in different units (MB / 4 KiB pages). Sourced from AMD's ROCm Strix Halo system-optimization guidance and Framework community threads endorsed by AMD engineers. Active after the next reboot (deferred until the model download finishes).

Original cmdline: rhgb quiet (backup: /etc/default/grub.bak-20260909).

sudo cp /etc/default/grub.bak-20260909 /etc/default/grub
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo systemctl reboot

llama-swap binary at /srv/llm/bin/; config /srv/llm/llama-swap-config.yaml defines qwen38-27b (65K ctx, flash-attn, jinja, DRY-sampling block, parallel 2, kv-unified, 99 GPU layers) and an embed endpoint (Qwen3-Embedding-0.6B Q8_0). User units llama-swap.service (proxy :8080) and llama-rerank.service (bge-reranker-v2-m3 Q8_0 CPU, :18190) installed for wg — mirroring the proven ashborn-prime topology. Activation pending model downloads.

rm /srv/llm/llama-swap-config.yaml /srv/llm/bin/llama-swap
rm ~/.config/systemd/user/llama-swap.service ~/.config/systemd/user/llama-rerank.service

From HuggingFace unsloth/Qwen3.8-27B-GGUF: UD-Q4_K_XL main (17 GB) + mmproj-F16 vision (885 MB) + MTP draft head (1.3 GB), ~19.2 GB total, via the hf CLI. Destination: /srv/llm/models/Qwen3.8-27B-GGUF/. Files land dan-owned while the download runs (launched before the wg ownership transfer) — a final chown -R wg:wg /srv/llm runs when it completes.

rm -rf /srv/llm/models/Qwen3.8-27B-GGUF   # weights only; re-downloadable from HF

Installed python3-pip (dnf), created /srv/llm/venv (Python 3.14) with huggingface_hub 1.30.0 + hf_transfer. Used for model downloads; no system Python was touched.

sudo rm -rf /srv/llm/venv
sudo dnf remove python3-pip

Fedora's llama.cpp RPM (b6153, Jan 2026) is 8 months stale and cannot load any of the target models. Built current master (commit 22397c3, 2026-09-09 — includes the Strix Halo Vulkan mat-vec tuning and the Qwen3.8 top-k radix fix) from /srv/llm/src/llama.cpp:

• build-vulkan — -DGGML_VULKAN=ON — built OK
• build-hip — -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 -DGGML_HIP_NO_VMM=ON — built OK

Source stays at /srv/llm/src/llama.cpp for rebuilds; binaries in the two build dirs. Nothing overwritten — the distro RPM remains installed for reference.

rm -rf /srv/llm/src/llama.cpp
sudo dnf remove git cmake clang compiler-rt vulkan-loader-devel vulkan-headers \
  glslc glslang-devel spirv-tools-devel spirv-headers-devel \
  rocm-hip-devel hipblas-devel hipblas-common-devel hipblaslt-devel rocblas-devel

chown -R wg:wg /srv/llm — the entire inference tree (src, venv, models) is owned by wg, so everything keeps working after the dan account is removed at handback. An admin SSH public key (dxverm@ashborn ed25519) was installed into /home/wg/.ssh/authorized_keys for key-based remote administration.

Disclosure: a first, malformed attempt at the key write may have truncated a pre-existing /home/wg/.ssh/authorized_keys if the owner had installed one on 2026-09-06. No other keys were present in the file afterwards; password SSH for wg is untouched, so owner access is unaffected. If you had a key there, re-add it.

sudo chown -R dan:dan /srv/llm
sudo sh -c 'echo -n > /home/wg/.ssh/authorized_keys'   # or restore your own key

dnf install htop btop nvtop tmux lm_sensors sysstat ripgrep unzip tar — pulled in rocm-smi as an nvtop dependency (bonus: GPU monitoring CLI now present).

Why: operator-side monitoring of GPU/thermals during benchmarks.

sudo dnf remove htop btop nvtop tmux lm_sensors sysstat ripgrep unzip tar

Full dnf upgrade -y installed a new kernel; reboot required to activate. Post-reboot verified: amdgpu + amdxdna loaded, llama-server still initializes gfx1151, /srv/llm/models auto-mounted from fstab.

Why: run the updated kernel + firmware before any inference work.

Boot the previous kernel from the GRUB menu (6.19.10-300 still installed).
Old kernel packages can be re-installed if ever removed:
  sudo dnf install kernel-6.19.10-300.fc44

First imported the Tailscale repository signing key (rpm --import https://pkgs.tailscale.com/stable/fedora/repo.gpg), which had been blocking non-interactive dnf runs. Then sudo dnf upgrade -y: systemd 259.8, kernel 7.1.13, linux-firmware 20260810, sudo, tzdata, and the rest of the pending set. dnf history entry #11.

Why: the install was 3 days old with all updates pending, including security fixes (sudo, systemd, kernel).

Package updates are not individually reversible by design.
Rollback anchor: dnf history list  (transaction #11, 2026-09-09 14:54 UTC)
  sudo dnf history rollback 10   # state before this upgrade (includes the
                                 # tailscale key import, which is harmless)

nvme0n1 (Samsung 970 EVO Plus 2TB) was re-formatted mkfs.ext4 -L LLM-MODELS and mounted at /srv/llm/models with a persistent fstab entry (defaults,noatime,nofail), then verified post-reboot. New UUID: 4d7f50e0-24a2-44ac-af2c-5173028f94cd.

Wiped content: a stale .cache directory owned by uid 989 (~77 GB) left over from a previous operating system on this disk — no user documents (verified by read-only mount before wiping, per operator instruction).

Why: dedicated home for model weights, isolated from the OS disk.

sudo sed -i '\|/srv/llm/models|d' /etc/fstab
sudo umount /srv/llm/models
sudo rmdir /srv/llm/models
# NOTE: the stale 77G uid-989 cache is NOT recoverable after this format
# (it was inspected read-only first and held only browser/tool cache data)

The as-found file contained invalid syntax (dan (ALL) ALL=ALL) which made sudo print a parse error on every invocation. Replaced with the valid form.

Before:

dan (ALL) ALL=ALL        # invalid syntax

After:

dan ALL=(ALL) ALL

Verified with sudo visudo -c — no errors. Note: dan is also a member of wheel, so this file is technically redundant; it is kept because it was the owner's explicit grant. The original file is preserved on the machine.

sudo cp /etc/sudoers.d/dan.bak-20260909-orig /etc/sudoers.d/dan
# (backup filename contains dots, so sudoers ignores it as a rule file)