This repository has been archived on 2026-05-20. You can view files and clone it, but cannot push or open issues or pull requests.
media-acquisition/docs/migration.md
obsidian-ai d300d83ce1 init: media-acquisition pipeline scaffold
Self-hosted BitTorrent + arr-stack + catalog-update pipeline targeting
nullstone (Debian 13). Replaces the legacy onyx -> rsync -> import
round-trip.

Contents:
- README.md          headline + ASCII architecture diagram + quickstart
- CLAUDE.md          project rules (mirrors beta-flix style)
- .gitignore         secrets dirs (.env, gluetun, qbt config, ssh keys)
- .gitleaksignore    allowlist nullstone LAN addr + Tailscale CGNAT
- docs/architecture.md   the plan in detail (gluetun + qbt + arr + catalog)
- docs/migration.md  onyx-qbt -> nullstone-qbt runbook (3 phases)
- docs/trackers.md   tracker schema + IP-pinning + ratio notes (user-curated)
- compose/docker-compose.yml  gluetun v3.40 + qbt 5.0.5 (netns=gluetun) +
                              sonarr/radarr/prowlarr (hotio) + betaflix-catalog
- compose/.env.example       documented env-var template (no secrets)
- compose/traefik/arr.yml    file-provider for qbt/sonarr/radarr/prowlarr
                             .s8n.ru subdomains, LAN+TS only via
                             trusted-only@file + authentik-forwardauth@file
- catalog/catalog.py         Flask service, ~340 LoC, /sonarr + /radarr +
                             /healthz; pulls beta-flix, inserts alphabetic
                             row into MEDIA-LIST.md, writes run log, commits
                             + pushes as obsidian-ai. Idempotent via
                             payload-hash cache.
- catalog/Dockerfile         python:3.12-slim + git + tini
- catalog/requirements.txt   flask + jinja2 + requests + gitpython + pyyaml (pinned)
- catalog/templates/*.j2     run log + catalog row Jinja templates
- catalog/README.md          service docs
- scripts/migrate-onyx.sh    phase-2 helper (rsync + .torrent ship, dry-run by default)
- scripts/add-tracker.sh     Prowlarr API helper
- scripts/killswitch-test.sh gluetun kill-switch verification (3 steps)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 01:15:43 +01:00

4.7 KiB

Migration — onyx-qbt → nullstone-qbt

State at time of writing: 60+ active torrents on onyx with download dirs on onyx local disk. Goal: keep seeding (don't burn ratios) while shifting future downloads to nullstone. Two-phase, no big-bang.


Phase 1 — Stand up nullstone stack (no migration yet)

  1. Prep directory tree on nullstone:

    ssh user@nullstone
    sudo mkdir -p /home/user/media/_downloads/{incomplete,complete,watch}
    sudo chown -R user:user /home/user/media/_downloads
    
  2. Generate new Proton WG key + provisioning for gluetun. Don't reuse wg-pvpn-A keys (they're host-routed; conflict risk). Log into Proton account → WireGuard → new key → name it nullstone-gluetun-arr → save the privkey + assigned address (e.g. 10.2.0.3/32).

  3. Drop the privkey + address into compose/.env:

    cd /opt/docker/media-acquisition/compose
    cp .env.example .env
    ${EDITOR:-vi} .env
    # Set:
    #   PVPN_WG_PRIVKEY=<the new privkey>
    #   PVPN_WG_ADDRESSES=10.2.0.3/32
    #   PVPN_SERVER_COUNTRIES=Netherlands
    
  4. Bring up the stack. Start gluetun + qbt only first:

    docker compose up -d gluetun qbittorrent
    
  5. Kill-switch test (NON-NEGOTIABLE):

    bash scripts/killswitch-test.sh
    

    If second curl succeeds → leak. Tear down and debug. Do not proceed.

  6. Sacrificial torrent. Pick something legal + big you don't care about (e.g. a Linux distro ISO). Add it via qbt webui, watch it land in /home/user/media/_downloads/complete/. Confirm it does not appear in JF.

  7. Bring up the rest of the stack.

    docker compose up -d
    

    Configure Prowlarr → Sonarr → Radarr (in that order — Prowlarr pushes indexers downstream). Set "Use Hardlinks instead of Copy = yes" in Sonarr/Radarr Media Management.

  8. Test arr → import path. Sonarr Interactive Search → manual grab → import into /media/tv/.... Verify catalog service commits to Forgejo.


Phase 2 — Migrate onyx torrents

For each active torrent on onyx that you want to keep seeding:

# On onyx — export .torrent files + qbt's fastresume state
mkdir -p /tmp/qbt-migrate
cp ~/.local/share/qBittorrent/BT_backup/*.torrent /tmp/qbt-migrate/
cp ~/.local/share/qBittorrent/BT_backup/*.fastresume /tmp/qbt-migrate/

# rsync the actual data files to nullstone first (LAN gigabit)
rsync -av --info=progress2 ~/Downloads/qbt/ \
  user@192.168.0.100:/home/user/media/_downloads/complete/

Then on nullstone qbt webui:

  1. Add .torrent files in bulk via webui ("Add torrent files…"), save path = /downloads/complete/, uncheck "Start torrent".
  2. Force-recheck each added torrent. qbt matches local files → 100% → seeding.
  3. Verify trackers respond. Private trackers may need source-IP rotation — gluetun exit IP differs from onyx public IP. See docs/trackers.md.
  4. On onyx: pause torrents one-by-one as nullstone takes over. Don't stop onyx-qbt entirely until every torrent shows seeding on nullstone for 24h with no tracker errors.

Catalog backfill: for torrents that correspond to already-imported library items, don't trigger arr-import — they're already in canonical locations. Just seed from _downloads/complete/. Catalog stays accurate.

For torrents that were mid-download on onyx but never made it into the library: re-add on nullstone, let them complete via VPN, then sonarr/radarr picks them up via the normal path.

Estimated migration window: 1 weekend. ~250 GB rsync over LAN gigabit ≈ ~30 min wall clock for the data move, then a manual-but-tedious add-and-recheck loop in qbt.

The wrapper script for steps 1-2 is at scripts/migrate-onyx.sh. It does the rsync + builds a .torrent index for a follow-up bulk-add. The fastresume-rewrite step is documented inline in the script.


Phase 3 — Decommission onyx-qbt

After 7 days clean on nullstone:

  1. Stop qbt service on onyx (systemctl --user stop qbittorrent-nox or kill the GUI; depends on how it was launched).
  2. Delete ~/Downloads/qbt/ on onyx (only after confirming no in-flight torrents reference it).
  3. Update ai-lab/CLAUDE.md device registry note if onyx had a "downloads role" annotation. (As of 2026-05-20 it does not — onyx has been the staging host but is not formally documented as such.)
  4. Optional: keep the .torrent files archive on onyx for 30 days as a safety net.

Rollback

If nullstone stack starts failing during phase 2:

  • docker compose down on nullstone.
  • Re-enable onyx qbt (Phase 1's stack is non-destructive — onyx torrents still have their data + fastresume).
  • File an issue + revisit phase 1 step 5 (kill-switch test).