This repository has been archived on 2026-05-20. You can view files and clone it, but cannot push or open issues or pull requests.
media-acquisition/docs/migration.md
obsidian-ai d300d83ce1 init: media-acquisition pipeline scaffold
Self-hosted BitTorrent + arr-stack + catalog-update pipeline targeting
nullstone (Debian 13). Replaces the legacy onyx -> rsync -> import
round-trip.

Contents:
- README.md          headline + ASCII architecture diagram + quickstart
- CLAUDE.md          project rules (mirrors beta-flix style)
- .gitignore         secrets dirs (.env, gluetun, qbt config, ssh keys)
- .gitleaksignore    allowlist nullstone LAN addr + Tailscale CGNAT
- docs/architecture.md   the plan in detail (gluetun + qbt + arr + catalog)
- docs/migration.md  onyx-qbt -> nullstone-qbt runbook (3 phases)
- docs/trackers.md   tracker schema + IP-pinning + ratio notes (user-curated)
- compose/docker-compose.yml  gluetun v3.40 + qbt 5.0.5 (netns=gluetun) +
                              sonarr/radarr/prowlarr (hotio) + betaflix-catalog
- compose/.env.example       documented env-var template (no secrets)
- compose/traefik/arr.yml    file-provider for qbt/sonarr/radarr/prowlarr
                             .s8n.ru subdomains, LAN+TS only via
                             trusted-only@file + authentik-forwardauth@file
- catalog/catalog.py         Flask service, ~340 LoC, /sonarr + /radarr +
                             /healthz; pulls beta-flix, inserts alphabetic
                             row into MEDIA-LIST.md, writes run log, commits
                             + pushes as obsidian-ai. Idempotent via
                             payload-hash cache.
- catalog/Dockerfile         python:3.12-slim + git + tini
- catalog/requirements.txt   flask + jinja2 + requests + gitpython + pyyaml (pinned)
- catalog/templates/*.j2     run log + catalog row Jinja templates
- catalog/README.md          service docs
- scripts/migrate-onyx.sh    phase-2 helper (rsync + .torrent ship, dry-run by default)
- scripts/add-tracker.sh     Prowlarr API helper
- scripts/killswitch-test.sh gluetun kill-switch verification (3 steps)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 01:15:43 +01:00

136 lines
4.7 KiB
Markdown

# Migration — onyx-qbt → nullstone-qbt
State at time of writing: 60+ active torrents on onyx with download dirs on
onyx local disk. **Goal:** keep seeding (don't burn ratios) while shifting
future downloads to nullstone. Two-phase, no big-bang.
---
## Phase 1 — Stand up nullstone stack (no migration yet)
1. **Prep directory tree** on nullstone:
```bash
ssh user@nullstone
sudo mkdir -p /home/user/media/_downloads/{incomplete,complete,watch}
sudo chown -R user:user /home/user/media/_downloads
```
2. **Generate new Proton WG key + provisioning for gluetun.** Don't reuse
`wg-pvpn-A` keys (they're host-routed; conflict risk). Log into Proton
account → WireGuard → new key → name it `nullstone-gluetun-arr` → save
the privkey + assigned address (e.g. `10.2.0.3/32`).
3. **Drop the privkey + address into `compose/.env`:**
```bash
cd /opt/docker/media-acquisition/compose
cp .env.example .env
${EDITOR:-vi} .env
# Set:
# PVPN_WG_PRIVKEY=<the new privkey>
# PVPN_WG_ADDRESSES=10.2.0.3/32
# PVPN_SERVER_COUNTRIES=Netherlands
```
4. **Bring up the stack.** Start gluetun + qbt only first:
```bash
docker compose up -d gluetun qbittorrent
```
5. **Kill-switch test (NON-NEGOTIABLE):**
```bash
bash scripts/killswitch-test.sh
```
If second curl succeeds → leak. Tear down and debug. Do not proceed.
6. **Sacrificial torrent.** Pick something legal + big you don't care about
(e.g. a Linux distro ISO). Add it via qbt webui, watch it land in
`/home/user/media/_downloads/complete/`. Confirm it **does not** appear in JF.
7. **Bring up the rest of the stack.**
```bash
docker compose up -d
```
Configure Prowlarr → Sonarr → Radarr (in that order — Prowlarr pushes
indexers downstream). Set "Use Hardlinks instead of Copy = yes" in
Sonarr/Radarr Media Management.
8. **Test arr → import path.** Sonarr Interactive Search → manual grab → import
into `/media/tv/...`. Verify catalog service commits to Forgejo.
---
## Phase 2 — Migrate onyx torrents
For each active torrent on onyx that you want to keep seeding:
```bash
# On onyx — export .torrent files + qbt's fastresume state
mkdir -p /tmp/qbt-migrate
cp ~/.local/share/qBittorrent/BT_backup/*.torrent /tmp/qbt-migrate/
cp ~/.local/share/qBittorrent/BT_backup/*.fastresume /tmp/qbt-migrate/
# rsync the actual data files to nullstone first (LAN gigabit)
rsync -av --info=progress2 ~/Downloads/qbt/ \
user@192.168.0.100:/home/user/media/_downloads/complete/
```
Then on nullstone qbt webui:
1. Add `.torrent` files in bulk via webui ("Add torrent files…"), save path =
`/downloads/complete/`, **uncheck "Start torrent"**.
2. Force-recheck each added torrent. qbt matches local files → `100%` → seeding.
3. Verify trackers respond. Private trackers may need source-IP rotation —
gluetun exit IP differs from onyx public IP. See `docs/trackers.md`.
4. On onyx: pause torrents one-by-one as nullstone takes over. Don't stop
onyx-qbt entirely until every torrent shows seeding on nullstone for 24h
with no tracker errors.
**Catalog backfill:** for torrents that correspond to already-imported
library items, **don't** trigger arr-import — they're already in canonical
locations. Just seed from `_downloads/complete/`. Catalog stays accurate.
For torrents that were mid-download on onyx but never made it into the
library: re-add on nullstone, let them complete via VPN, then sonarr/radarr
picks them up via the normal path.
**Estimated migration window:** 1 weekend. ~250 GB rsync over LAN gigabit ≈
~30 min wall clock for the data move, then a manual-but-tedious
add-and-recheck loop in qbt.
The wrapper script for steps 1-2 is at `scripts/migrate-onyx.sh`. It does
the rsync + builds a `.torrent` index for a follow-up bulk-add. The
fastresume-rewrite step is documented inline in the script.
---
## Phase 3 — Decommission onyx-qbt
After 7 days clean on nullstone:
1. Stop qbt service on onyx (`systemctl --user stop qbittorrent-nox` or kill
the GUI; depends on how it was launched).
2. Delete `~/Downloads/qbt/` on onyx (only after confirming no in-flight
torrents reference it).
3. Update `ai-lab/CLAUDE.md` device registry note if onyx had a
"downloads role" annotation. (As of 2026-05-20 it does not — onyx has been
the staging host but is not formally documented as such.)
4. Optional: keep the `.torrent` files archive on onyx for 30 days as a
safety net.
---
## Rollback
If nullstone stack starts failing during phase 2:
- `docker compose down` on nullstone.
- Re-enable onyx qbt (Phase 1's stack is non-destructive — onyx torrents still
have their data + fastresume).
- File an issue + revisit phase 1 step 5 (kill-switch test).