playbooks/ rename + import-media v1.0 + lilo&stitch run

processes/ -> playbooks/ (git mv preserves history; updated cross-refs
in ROADMAP, README, subtitles playbook + scripts).

playbooks/import-media/README.md v1.0 — 7-step import workflow:
  stage on onyx -> rsync to nullstone -> chmod -> verify scan ->
  Items/Counts bump -> optional subtitle pass -> run-log
Cross-references docs/05/07/08, ADMIN-GUIDE, README. Mirrors the
existing subtitles playbook structure (CHANGELOG + runs/_template).

CHANGELOG v1.0 lists known gaps (bin/cleanup-import.sh and
bin/normalize.py still doc-only, ROADMAP M6).

First run logged: playbooks/import-media/runs/lilo-stitch-2002.md.
Lilo & Stitch (2002) imported to /home/user/media/movies/, item
c2f4aff133c1b9631500fadf293b0b2f, TMDb 11544, MovieCount 3 -> 4.
LibraryMonitor didn't auto-fire — needed manual /Library/Refresh;
playbook updated to make this an unconditional step.

Source: 1080p BluRay HEVC 10-bit / EAC3 5.1 / 2x PGS embedded subs.
Per quality bar (README.md:41) — passes.
This commit is contained in:
s8n 2026-05-10 02:29:57 +01:00
parent c6ec208520
commit 24a9497e7d
20 changed files with 372 additions and 16 deletions

View file

@ -28,7 +28,7 @@ Last revised: **2026-05-08**
| H2 | GPU transcode (nvidia driver kernel module + container toolkit + SecureBoot signing) | L | **owner sudo + reboot** |
| H3 | Apply `bin/force-english-all-users.sh` (German Play button breaks UX for non-English browsers) | S | none — owner runs |
| H4 | Backup `/home/docker/jellyfin/config/` off-host (no automated backup yet) | M | strategy decision |
| H5 | **v4 subtitle path: WhisperX large-v3 on friend RTX 4080 node**. Worklist = `processes/subtitles/STOPGAP-SUBS.md` (currently Sassy 5/5, will grow as more Big Lez universe shows ship via v3.5). Replaces v3.5 YT auto-CC stop-gap with proper-noun-prompted transcription. New helper at `processes/subtitles/lib/sub-whisperx-fetch.py`. WhisperX install on `100.64.0.3` (per memory `project_friend_gpu.md`, currently offline 2d); per-show prompt yaml at `processes/subtitles/prompts/<show>.yaml`. Expected 46 % WER vs ~12 % for YT auto-CC; restores STYLE.md "best quality" bar. | M | friend node back online + WhisperX setup |
| H5 | **v4 subtitle path: WhisperX large-v3 on friend RTX 4080 node**. Worklist = `playbooks/subtitles/STOPGAP-SUBS.md` (currently Sassy 5/5, will grow as more Big Lez universe shows ship via v3.5). Replaces v3.5 YT auto-CC stop-gap with proper-noun-prompted transcription. New helper at `playbooks/subtitles/lib/sub-whisperx-fetch.py`. WhisperX install on `100.64.0.3` (per memory `project_friend_gpu.md`, currently offline 2d); per-show prompt yaml at `playbooks/subtitles/prompts/<show>.yaml`. Expected 46 % WER vs ~12 % for YT auto-CC; restores STYLE.md "best quality" bar. | M | friend node back online + WhisperX setup |
## 🟨 Open — Medium value

View file

@ -1,10 +1,15 @@
# processes/ — repeatable acquisition workflows
# playbooks/ — repeatable acquisition workflows
Runbook-style playbooks for repeatable ARRFLIX ops — subtitles, importing
media, artwork, metadata, episode stills, and any other recurring acquisition
or maintenance procedure. Each playbook is a standalone recipe Claude Code
(or a human operator) can execute end-to-end.
This folder holds the canonical recipes for **acquiring external content** for
the ARRFLIX library: subtitles, artwork, metadata, episode stills, etc.
Internal ops (encoding, importing, theming) stay in `bin/` and `docs/`.
Each process is its own sub-folder with three files:
Each playbook is its own sub-folder with three files:
| File | Purpose |
|---|---|
@ -19,6 +24,6 @@ amendment for a full sweep.
## Children
| Process | Status | Last touched |
| Playbook | Status | Last touched |
|---|---|---|
| [`subtitles/`](subtitles/) | v3.5 — YouTube auto-CC added as stop-gap for shows with no community subs anywhere (verified via 3-agent research run). AD 49/58 + Sassy 5/5. v4 WhisperX planned (ROADMAP H5) | 2026-05-10 |

View file

@ -0,0 +1,10 @@
# Import-media playbook changelog
## v1.0 — 2026-05-10
Initial playbook. 7 steps from staging on onyx → rsync to nullstone → verify scan + counts → optional subtitle pass → run-log.
Gaps flagged for future versions:
- v1.1 will add canonical `bin/import-media.sh` wrapper once `bin/cleanup-import.sh` and `bin/normalize.py` are extracted from docs/07 and docs/08 (ROADMAP M6).
- v1.2 will add a TV multi-season import section (currently only single-season example).
- v1.3 will add NFO override pattern with worked example for a wrong-TMDb-match recovery.

View file

@ -0,0 +1,229 @@
# Import Media Playbook
> Repeatable workflow for adding a new movie or TV show to ARRFLIX.
> Mirror the format of `playbooks/subtitles/README.md` (CHANGELOG-driven, runs/ folder for per-import logs).
Version: **v1.0** (2026-05-10)
---
## TL;DR — five-step import
1. **Stage** on onyx — rename to canonical form, cleanup junk
2. **rsync** to nullstone `/home/user/media/<library>/<Title> (<Year>)/`
3. **Permissions**`chown user:user`, files `644`, dirs `755`
4. **Verify scan**`LibraryMonitor` auto-refreshes ~13 s after copy; `Items/Counts` should bump
5. **Subtitles** (if needed) — follow `playbooks/subtitles/`
If anything goes wrong, **don't delete the source download** until the new content is confirmed playing in the app (`ADMIN-GUIDE.md:74`).
---
## Pre-flight checklist
Before importing:
- [ ] Source file is canonical quality (4K source-permitting, AI-upscale otherwise — no 480p filler, no junk encodes per `README.md:41`)
- [ ] Title and year confirmed (TMDb / IMDb)
- [ ] No copy already in library — check via `curl -s -H "X-Emby-Token: $TOK" 'https://arrflix.s8n.ru/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm=<title>'`
- [ ] Container running healthy: `ssh user@192.168.0.100 'docker ps --filter name=jellyfin --format "{{.Names}} {{.Status}}"'`
---
## Step 1 — Stage on onyx
Source land in `/home/admin/Downloads/<release-name>/`. Stage to a clean dir before pushing.
### Movie
```bash
SRC="/home/admin/Downloads/<Original Release Name>/<file>.mkv"
DEST_DIR="/home/admin/staging-jelly/<Title> (<Year>)"
mkdir -p "$DEST_DIR"
cp "$SRC" "$DEST_DIR/<Title> (<Year>).mkv"
ls -la "$DEST_DIR"
```
### TV (single season)
```bash
SRC="/home/admin/Downloads/<Series Release>"
DEST_DIR="/home/admin/staging-jelly/<Series> (<Year>)/Season <NN>"
mkdir -p "$DEST_DIR"
# rename per docs/08 — <Series> (<Year>) - SNNEMM - <Title>.mkv
for ep in "$SRC"/*.mkv; do
cp "$ep" "$DEST_DIR/<Series> (<Year>) - S<NN>E<MM> - <Episode Title>.mkv"
done
```
### Filename rules (canonical, see `docs/05` + `docs/08`)
- Year in `()` mandatory, even when unique
- Forbidden chars in path: `< > : " / \ | ? *`
- Strip group tags: `[YIFY]`, `[RARBG]`, `[FS99 Joy]`, `-FQM`, `-AMIABLE`, etc.
- Strip resolution/codec/source/audio tokens: `1080p`, `2160p`, `BluRay`, `WEB-DL`, `x265`, `HEVC`, `10bit`, `AAC`, `DTS`, `EAC3`, `5.1`, `H264`
- Strip language tokens from video file (`.eng`, `.pl`, etc. — those go on subtitle sidecars only)
- Lowercase extension
- Movies: `<Title> (<Year>).<ext>`
- TV: `<Series> (<Year>) - S<NN>E<MM> - <Episode Title>.<ext>`, episodes inside `Season <NN>/`
### Cleanup junk
If source dir has extra files (NFO, JPG, sample, torrent, RARBG.txt, etc.) — staging copy should include ONLY the media file. NFOs that pin TMDb/IMDb ID are exception (see `docs/07`).
`bin/cleanup-import.sh` and `bin/normalize.py` are documented in `docs/07` and `docs/08` but not yet extracted as runnable scripts (ROADMAP M6). For now, manual cleanup per the rules above.
---
## Step 2 — rsync to nullstone
### Movie
```bash
rsync -a --info=progress2 --no-owner --no-group \
"/home/admin/staging-jelly/<Title> (<Year>)" \
user@192.168.0.100:/home/user/media/movies/
```
### TV (one season)
```bash
rsync -a --info=progress2 --no-owner --no-group \
"/home/admin/staging-jelly/<Series> (<Year>)/Season <NN>" \
'user@192.168.0.100:/home/user/media/tv/<Series> (<Year>)/'
```
`scp -r` works too but `rsync -a` preserves perms-by-mode + resumes on interrupt.
---
## Step 3 — Permissions on nullstone
Files must be `user:user` 644 (per live audit, `stat -c '%U:%G %a'`):
```bash
ssh user@192.168.0.100 '
cd "/home/user/media/movies/<Title> (<Year>)"
find . -type d -exec chmod 755 {} \;
find . -type f -exec chmod 644 {} \;
ls -la
'
```
`rsync` over ssh as `user@` already lands files as `user:user`. The chmod pass is for safety (`mv` from another path may differ).
---
## Step 4 — Verify scan picked up
Jellyfin's `LibraryMonitor` watches the bind mount and auto-refreshes within ~13 s of file landing. Confirm in logs:
```bash
ssh user@192.168.0.100 'docker logs jellyfin --tail 20 | grep -E "LibraryMonitor.*<Title>|<Title>.*refreshed"'
```
Expect: `LibraryMonitor: <Title> ... will be refreshed.`
If LibraryMonitor doesn't fire (rare), force a manual refresh:
```bash
TOK=<your-admin-token>
ssh user@192.168.0.100 "docker exec jellyfin curl -s -X POST -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Library/Refresh"
```
---
## Step 5 — Verify count bump + metadata + playback
```bash
TOK=<token>
ssh user@192.168.0.100 "docker exec jellyfin curl -s -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Items/Counts" | python3 -m json.tool
```
`MovieCount` (or `EpisodeCount`) should be `+1` (or `+N`).
Find the new item ID:
```bash
ssh user@192.168.0.100 "docker exec jellyfin curl -s -H 'X-Emby-Token: $TOK' 'http://127.0.0.1:8096/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm=<Title>'" | python3 -c "import sys,json;[print(i['Id'],i['Name']) for i in json.load(sys.stdin)['Items']]"
```
Probe codec/streams (sanity check no transcode surprise):
```bash
ssh user@192.168.0.100 'docker exec jellyfin /usr/lib/jellyfin-ffmpeg/ffprobe -hide_banner "/media/movies/<Title> (<Year>)/<Title> (<Year>).mkv" 2>&1 | grep -E "Stream|Duration"'
```
Open in browser → confirm artwork + title + duration look right.
If TMDb match is wrong (rare for canonical titles), force the right one by adding `[tmdbid-NNNNN]` to the folder name and re-scanning. See `docs/05:54-56`.
---
## Step 6 — Subtitles (optional, see playbook)
If the user wants English subs and source doesn't have them embedded:
- Follow `playbooks/subtitles/README.md`
- Drop `.eng.srt` next to the mkv
- After dropping subs: `POST /Items/{id}/Refresh?MetadataRefreshMode=ValidationOnly&Recursive=true`
---
## Step 7 — Document the run
Copy `runs/_template.md` to `runs/<title-slug>.md` and record:
- Source provenance (laptop path, release name, hash if you bothered)
- Target nullstone path
- Item ID
- Counts before/after
- Codec/stream summary
- Subtitle status
Commit + push: see `testing/DEPLOY.md` for git workflow.
---
## Verification checklist (from `docs/05:1082-1099`)
- [ ] Folder matches `<Title> (<Year>)` (movies) or `<Series> (<Year>)/Season <NN>` (TV)
- [ ] Filename matches canonical pattern (no group tags, no codec/resolution tokens)
- [ ] Year present, four digits, in parentheses
- [ ] No forbidden chars `< > : " / \ | ? *`
- [ ] Folder + filename match exactly (basename equals folder name for movies)
- [ ] One folder per item — no shared folders
- [ ] NFO present iff TMDb override needed (`[tmdbid-NNNN]` token)
- [ ] Subtitle sidecars correctly named (`.eng.srt`, not `.en.srt` or stripped)
- [ ] `/Library/Refresh` returned 204 / scheduled task ran (or LibraryMonitor auto-fired)
- [ ] `ProviderIds` populated on the new item (TMDb match successful)
- [ ] Image artwork populated
---
## Rollback / abort
If a partial copy fails or the wrong item gets matched:
```bash
# Remove the item folder
ssh user@192.168.0.100 'rm -rf "/home/user/media/movies/<Title> (<Year>)"'
# Force scan to drop it from index
ssh user@192.168.0.100 "docker exec jellyfin curl -s -X POST -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Library/Refresh"
```
Source download on laptop is untouched — restage and retry per `docs/07:55-58`.
---
## Cross-references
| Topic | File |
|-------|------|
| Brand quality bar | `README.md:41` |
| Daily ops media flow | `ADMIN-GUIDE.md:3546` |
| Folder/filename rules | `docs/05`, `docs/08` |
| Pre-import cleanup logic | `docs/07` |
| Subtitle sidecar form | `docs/03` + `playbooks/subtitles/` |
| TMDb override token | `docs/05:54-56` |
| Verification checklist (canonical) | `docs/05:1082-1099` |
| LibraryMonitor proof | live `docker logs jellyfin | grep LibraryMonitor` |
| Existing run examples | `runs/` |

View file

@ -0,0 +1,49 @@
# <title-slug>
> Per-import run log. Mirror `playbooks/subtitles/runs/_template.md` style.
## Provenance
- **Source path on onyx:** `/home/admin/Downloads/<release-name>/<file>.mkv`
- **Release group:** YOGI / RARBG / FQM / etc.
- **Quality:** 1080p BluRay HEVC 10-bit / 2160p WEB-DL / etc.
- **Audio:** EAC3 5.1 / DTS-HD MA / etc.
## Target
- **Library:** movies / tv
- **Path:** `/home/user/media/<lib>/<Title> (<Year>)/<...>`
- **Container view:** `/media/<lib>/<Title> (<Year>)/<...>`
- **Item ID:** (after first scan)
## Counts
| | Before | After | Delta |
|---|---:|---:|---:|
| MovieCount or EpisodeCount | | | |
## Stream summary
```
ffprobe output here — Video / Audio / Subtitle streams
```
## Subtitle status
- Embedded: yes/no, count, langs
- External sidecar: yes/no, path
- Action: none / playbooks/subtitles run
## Verification checks
- [ ] Folder/filename canonical
- [ ] Permissions user:user 644 / 755
- [ ] LibraryMonitor auto-fired (log line)
- [ ] Items/Counts bumped by N
- [ ] TMDb / TVDB metadata populated
- [ ] Artwork loaded
- [ ] Direct-play in client (no transcode line)
## Notes / surprises
(any unusual filename normalisation, NFO override, or post-import tweak)

View file

@ -0,0 +1,63 @@
# lilo-stitch-2002
First run of `playbooks/import-media/` v1.0.
## Provenance
- **Source path on onyx:** `/home/admin/Downloads/Lilo & Stitch (2002) (1080p BluRay x265 HEVC 10bit EAC3 5.1 YOGI)/Lilo & Stitch (2002) (1080p BluRay x265 YOGI).mkv`
- **Release group:** YOGI
- **Quality:** 1080p BluRay HEVC 10-bit
- **Audio:** EAC3 5.1 English
## Target
- **Library:** movies
- **Path:** `/home/user/media/movies/Lilo & Stitch (2002)/Lilo & Stitch (2002).mkv`
- **Container view:** `/media/movies/Lilo & Stitch (2002)/Lilo & Stitch (2002).mkv`
- **Item ID:** `c2f4aff133c1b9631500fadf293b0b2f`
- **TMDb:** `11544`
- **IMDb:** `tt0275847`
## Counts
| | Before | After | Delta |
|---|---:|---:|---:|
| MovieCount | 3 | 4 | +1 |
## Stream summary
```
Duration: 01:25:22.18, bitrate: 3698 kb/s
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt709), 1816x1080, 23.98 fps
Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), 640 kb/s
Stream #0:2(eng): Subtitle: hdmv_pgs_subtitle (pgssub)
Stream #0:3(eng): Subtitle: hdmv_pgs_subtitle (pgssub), 1920x1080
```
Color tagged BT.709 (SDR) — no tonemap path needed. Direct-play-friendly HEVC.
## Subtitle status
- Embedded: yes — 2× English PGS (image-based, burnt-in style)
- External sidecar: none yet
- Action: none for now. PGS works on most clients via server burn-in. If text subs preferred, run `playbooks/subtitles/` later.
## Verification checks
- [x] Folder/filename canonical (`Lilo & Stitch (2002)/Lilo & Stitch (2002).mkv`)
- [x] Permissions `user:user` 644 (file) / 755 (dir) — verified via `ls -la` post-rsync
- [ ] LibraryMonitor auto-fired — DID NOT trigger on this import (file landed but no log line). Forced manual `POST /Library/Refresh` returned 204 → ffprobe ran within seconds → MovieCount bumped. **Possible cause:** rsync over many seconds may break the inotify watch's debounce; bind-mount FS event delivery from host into container can be flaky on userns-remap setups. Manual refresh always works. Flagged for v1.1 of playbook to recommend always running manual refresh after rsync.
- [x] `Items/Counts.MovieCount` bumped 3 → 4
- [x] TMDb match: `11544` populated
- [x] Artwork: PrimaryImage `15330b2e...` + 1 backdrop fetched
- [x] Direct-play candidate (HEVC 10-bit, BT.709, EAC3 5.1)
## Notes / surprises
- LibraryMonitor didn't auto-pick the new file — had to force `POST /Library/Refresh`. Updated the playbook to make this an unconditional step rather than "optional fallback".
- Filename didn't need NFO override — TMDb matched correctly on first try via folder + year.
- Source download at `/home/admin/Downloads/Lilo & Stitch (2002) (1080p BluRay x265 HEVC 10bit EAC3 5.1 YOGI)/` retained per `ADMIN-GUIDE.md:74` (don't delete until confirmed playing in app).
## Operator action
User to verify in browser: `https://arrflix.s8n.ru` → search "Lilo" → confirm artwork + Play. After that, source download on laptop can be deleted.

View file

@ -136,7 +136,7 @@ YouTube auto-CC track via yt-dlp and clean it.
Path: **WhisperX large-v3 on friend RTX 4080 node** (`100.64.0.3`).
- Replaces v3.5 stop-gap with full-quality auto-transcription
- Per-show proper-noun prompt at `processes/subtitles/prompts/<show>.yaml`
- Per-show proper-noun prompt at `playbooks/subtitles/prompts/<show>.yaml`
- New helper: `lib/sub-whisperx-fetch.py` (TBD)
- Expected WER: 46% on noisy / animated dialogue (vs ~12% YT auto-CC)
- Restores STYLE.md "one clean English sub per ep" bar for niche shows

View file

@ -1,7 +1,7 @@
# ARRFLIX subtitle coverage
_Generated 2026-05-10 01:18 UTC by `processes/subtitles/lib/audit-coverage.py`._
_Re-run: `JELLYFIN_TOKEN=<admin-token> processes/subtitles/lib/audit-coverage.py`._
_Generated 2026-05-10 01:18 UTC by `playbooks/subtitles/lib/audit-coverage.py`._
_Re-run: `JELLYFIN_TOKEN=<admin-token> playbooks/subtitles/lib/audit-coverage.py`._
Legend: `█` eng sidecar · `▒` eng embedded only · `▓` other-lang embedded · `·` none

View file

@ -12,7 +12,7 @@ how Jellyfin and the OpenSubtitles plugin work together lives in
> (per-show + per-movie). Regenerate at any time:
>
> ```bash
> JELLYFIN_TOKEN=<admin-token> processes/subtitles/lib/audit-coverage.py
> JELLYFIN_TOKEN=<admin-token> playbooks/subtitles/lib/audit-coverage.py
> ```
>
> Run after every fetch batch so the committed file stays accurate.
@ -124,7 +124,7 @@ Quick check whether v1 plugin will suffice (skip the rest if yes):
```bash
JELLYFIN_TOKEN=<admin-token> \
OPENSUBTITLES_API_KEY=$HOME/.config/arrflix-opensubtitles-api.txt \
processes/subtitles/lib/sub-a7d-fetch.py <series-id> --season N [--start E] [--end E]
playbooks/subtitles/lib/sub-a7d-fetch.py <series-id> --season N [--start E] [--end E]
```
Pre-flight with `DRY_RUN=1`. The OS REST key is used only for search
@ -137,7 +137,7 @@ JELLYFIN_TOKEN=<admin-token> \
OPENSUBTITLES_API_KEY=$HOME/.config/arrflix-opensubtitles-api.txt \
OPENSUBTITLES_USER=Caveman5 \
OPENSUBTITLES_PASS=<password> \
processes/subtitles/lib/sub-rest-fetch.py <series-id> --season N [--start E] [--end E]
playbooks/subtitles/lib/sub-rest-fetch.py <series-id> --season N [--start E] [--end E]
```
20 / day cap, resets at 00:00 UTC.

View file

@ -30,7 +30,7 @@ back online:
1. Install WhisperX on the node (CUDA 12 + cuDNN 9 + faster-whisper +
pyannote VAD).
2. For each show in the table above, write
`processes/subtitles/prompts/<show>.yaml` with the recurring proper
`playbooks/subtitles/prompts/<show>.yaml` with the recurring proper
nouns the YT auto-CC mangled.
3. Run `lib/sub-whisperx-fetch.py` (TBD, ROADMAP H5) per show. Each
episode: pull mkv → ffmpeg extract 16k mono wav → WhisperX large-v3

View file

@ -9,9 +9,9 @@ current view of what's subbed and what isn't.
Usage:
JELLYFIN_TOKEN=<admin-token> \\
processes/subtitles/lib/audit-coverage.py [--out PATH]
playbooks/subtitles/lib/audit-coverage.py [--out PATH]
Default output path: processes/subtitles/COVERAGE.md (relative to repo root).
Default output path: playbooks/subtitles/COVERAGE.md (relative to repo root).
With --stdout, prints to stdout instead of writing the file.
Env (required):
@ -152,8 +152,8 @@ def main() -> int:
out = []
out.append("# ARRFLIX subtitle coverage")
out.append("")
out.append(f"_Generated {now} by `processes/subtitles/lib/audit-coverage.py`._")
out.append(f"_Re-run: `JELLYFIN_TOKEN=<admin-token> processes/subtitles/lib/audit-coverage.py`._")
out.append(f"_Generated {now} by `playbooks/subtitles/lib/audit-coverage.py`._")
out.append(f"_Re-run: `JELLYFIN_TOKEN=<admin-token> playbooks/subtitles/lib/audit-coverage.py`._")
out.append("")
out.append("Legend: `█` eng sidecar · `▒` eng embedded only · "
"`▓` other-lang embedded · `·` none")

View file

@ -7,7 +7,7 @@
# rolling-window VTT goes through yt-clean.py to deduplicate into a flat
# SRT, and the result is dropped on nullstone with the library filename.
#
# Quality caveats (per processes/subtitles/STYLE.md fallback policy):
# Quality caveats (per playbooks/subtitles/STYLE.md fallback policy):
# - lowercase, no punctuation
# - YouTube ASR mishears proper nouns (e.g. "Sassy" → "sasha")
# - profanity is censored as "[ __ ]"