diff --git a/playbooks/import-media/CHANGELOG.md b/playbooks/import-media/CHANGELOG.md index c577918..bccbf04 100644 --- a/playbooks/import-media/CHANGELOG.md +++ b/playbooks/import-media/CHANGELOG.md @@ -1,10 +1,21 @@ # Import-media playbook changelog +## v1.1 — 2026-05-10 + +Two Jellyfin endpoint bugs found during `archer-s02-2009` run, codified into the playbook: + +- **`POST /Library/Refresh` is a silent no-op** on this Jellyfin build — returns HTTP 204 but the `Scan Media Library` scheduled task does NOT execute. Step 4 rewritten to fetch the scan-task ID from `/ScheduledTasks` and POST to `/ScheduledTasks/Running/` instead. Old endpoint added to a "known broken" table. +- **`/Items/Counts` is scope-cached** and stays stale even after items are indexed (counts stayed at 230 after 13 new eps landed). Step 5 rewritten to use the per-series authoritative query `/Shows//Episodes?Season=` with provider + image-tag verification, plus a per-series `Items//Refresh` recipe for missing metadata. +- LibraryMonitor inotify auto-fire wording removed from Step 4 (failed on both recorded runs — Lilo & Stitch + Archer S02). Manual task trigger is now mandatory. +- Verification checklist updated to reference the task-trigger endpoint and per-series query. +- Rollback section: replaced `/Library/Refresh` invocation with `/ScheduledTasks/Running/`. + ## v1.0 — 2026-05-10 Initial playbook. 7 steps from staging on onyx → rsync to nullstone → verify scan + counts → optional subtitle pass → run-log. Gaps flagged for future versions: -- v1.1 will add canonical `bin/import-media.sh` wrapper once `bin/cleanup-import.sh` and `bin/normalize.py` are extracted from docs/07 and docs/08 (ROADMAP M6). -- v1.2 will add a TV multi-season import section (currently only single-season example). -- v1.3 will add NFO override pattern with worked example for a wrong-TMDb-match recovery. +- v1.2 will add canonical `bin/import-media.sh` wrapper once `bin/cleanup-import.sh` and `bin/normalize.py` are extracted from docs/07 and docs/08 (ROADMAP M6). (Bumped from v1.1 since v1.1 was needed for the Jellyfin endpoint fixes.) +- v1.3 will add a TV multi-season import section (currently only single-season example). +- v1.4 will add NFO override pattern with worked example for a wrong-TMDb-match recovery. +- v1.5: document API-token retrieval (mint a permanent ApiKeys row labelled `import-pipeline` instead of pulling a session token from `Devices` table). diff --git a/playbooks/import-media/README.md b/playbooks/import-media/README.md index 897f1cf..faf6a67 100644 --- a/playbooks/import-media/README.md +++ b/playbooks/import-media/README.md @@ -3,7 +3,7 @@ > Repeatable workflow for adding a new movie or TV show to ARRFLIX. > Mirror the format of `playbooks/subtitles/README.md` (CHANGELOG-driven, runs/ folder for per-import logs). -Version: **v1.0** (2026-05-10) +Version: **v1.1** (2026-05-10) --- @@ -26,7 +26,7 @@ Before importing: - [ ] Source file is canonical quality (4K source-permitting, AI-upscale otherwise — no 480p filler, no junk encodes per `README.md:41`) - [ ] Title and year confirmed (TMDb / IMDb) - [ ] No copy already in library — check via `curl -s -H "X-Emby-Token: $TOK" 'https://arrflix.s8n.ru/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm='` -- [ ] Container running healthy: `ssh user@192.168.0.100 'docker ps --filter name=jellyfin --format "{{.Names}} {{.Status}}"'` +- [ ] Container running healthy: `ssh user@nullstone 'docker ps --filter name=jellyfin --format "{{.Names}} {{.Status}}"'` --- @@ -82,7 +82,7 @@ If source dir has extra files (NFO, JPG, sample, torrent, RARBG.txt, etc.) — s ```bash rsync -a --info=progress2 --no-owner --no-group \ "/home/admin/staging-jelly/<Title> (<Year>)" \ - user@192.168.0.100:/home/user/media/movies/ + user@nullstone:/home/user/media/movies/ ``` ### TV (one season) @@ -90,7 +90,7 @@ rsync -a --info=progress2 --no-owner --no-group \ ```bash rsync -a --info=progress2 --no-owner --no-group \ "/home/admin/staging-jelly/<Series> (<Year>)/Season <NN>" \ - 'user@192.168.0.100:/home/user/media/tv/<Series> (<Year>)/' + 'user@nullstone:/home/user/media/tv/<Series> (<Year>)/' ``` `scp -r` works too but `rsync -a` preserves perms-by-mode + resumes on interrupt. @@ -102,7 +102,7 @@ rsync -a --info=progress2 --no-owner --no-group \ Files must be `user:user` 644 (per live audit, `stat -c '%U:%G %a'`): ```bash -ssh user@192.168.0.100 ' +ssh user@nullstone ' cd "/home/user/media/movies/<Title> (<Year>)" find . -type d -exec chmod 755 {} \; find . -type f -exec chmod 644 {} \; @@ -114,44 +114,92 @@ ssh user@192.168.0.100 ' --- -## Step 4 — Verify scan picked up +## Step 4 — Trigger scan (do not rely on auto-refresh) -Jellyfin's `LibraryMonitor` watches the bind mount and auto-refreshes within ~1–3 s of file landing. Confirm in logs: +> **v1.1 change:** prior versions said `LibraryMonitor` auto-fires within 1–3 s and `POST /Library/Refresh` is the fallback. **Both proved unreliable across consecutive runs (`lilo-stitch-2002`, `archer-s02-2009`).** Always trigger the `Scan Media Library` scheduled task directly. -```bash -ssh user@192.168.0.100 'docker logs jellyfin --tail 20 | grep -E "LibraryMonitor.*<Title>|<Title>.*refreshed"' -``` - -Expect: `LibraryMonitor: <Title> ... will be refreshed.` - -If LibraryMonitor doesn't fire (rare), force a manual refresh: +### 4a. Pull task ID once (cache in shell or env) ```bash TOK=<your-admin-token> -ssh user@192.168.0.100 "docker exec jellyfin curl -s -X POST -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Library/Refresh" +SCAN_TASK_ID=$(ssh user@nullstone \ + "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/ScheduledTasks" \ + | python3 -c 'import sys,json +[print(t["Id"]) for t in json.load(sys.stdin) if t["Name"]=="Scan Media Library"]') +echo "$SCAN_TASK_ID" ``` +### 4b. Trigger the task (returns 204 + actually runs) + +```bash +ssh user@nullstone \ + "docker exec jellyfin curl -sf -X POST -H 'X-Emby-Token: $TOK' \ + http://127.0.0.1:8096/ScheduledTasks/Running/$SCAN_TASK_ID -w 'HTTP:%{http_code}\n'" +``` + +### Known broken endpoints (do NOT use) + +| Endpoint | Why broken | +|---|---| +| `LibraryMonitor` (inotify) auto-fire | Bind-mount + userns-remap inotify events do not propagate reliably; observed silent miss on every recent run. | +| `POST /Library/Refresh` | Returns HTTP 204 but the `Scan Media Library` scheduled task does NOT fire. Silent no-op. Use `/ScheduledTasks/Running/<id>` instead. | + +### 4c. Confirm task actually started + +```bash +ssh user@nullstone \ + "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \ + http://127.0.0.1:8096/ScheduledTasks/$SCAN_TASK_ID" \ + | python3 -c 'import sys,json;d=json.load(sys.stdin);print(d["State"], d.get("CurrentProgressPercentage","-"))' +``` + +Expect: `Running <pct>` while scanning, then `Idle -` when done. `LastExecutionResult.EndTimeUtc` should advance past the trigger time. + --- -## Step 5 — Verify count bump + metadata + playback +## Step 5 — Verify items added + metadata + playback + +> **v1.1 change:** **Do NOT use `/Items/Counts` for verification.** It is scope-cached/user-default-view-cached and stays stale even when the scan completed and items were added (observed in `archer-s02-2009`: counts stayed at 230 long after 13 eps were indexed). Use the per-item or per-series query as authoritative. + +### Movies — search by title ```bash TOK=<token> -ssh user@192.168.0.100 "docker exec jellyfin curl -s -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Items/Counts" | python3 -m json.tool +ssh user@nullstone "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \ + 'http://127.0.0.1:8096/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm=<Title>'" \ + | python3 -c "import sys,json;[print(i['Id'],i['Name']) for i in json.load(sys.stdin)['Items']]" ``` -`MovieCount` (or `EpisodeCount`) should be `+1` (or `+N`). - -Find the new item ID: +### TV — list episodes for the affected series + season (authoritative) ```bash -ssh user@192.168.0.100 "docker exec jellyfin curl -s -H 'X-Emby-Token: $TOK' 'http://127.0.0.1:8096/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm=<Title>'" | python3 -c "import sys,json;[print(i['Id'],i['Name']) for i in json.load(sys.stdin)['Items']]" +SERIES_ID=$(ssh user@nullstone "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \ + 'http://127.0.0.1:8096/Items?Recursive=true&IncludeItemTypes=Series&searchTerm=<Series>'" \ + | python3 -c 'import sys,json;print(json.load(sys.stdin)["Items"][0]["Id"])') + +ssh user@nullstone "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \ + 'http://127.0.0.1:8096/Shows/$SERIES_ID/Episodes?Season=<NN>&fields=Path,ProviderIds,ImageTags'" \ + | python3 -c 'import sys,json +d=json.load(sys.stdin) +print("COUNT:", len(d["Items"])) +for e in d["Items"]: + print(" S{}E{:02d} {} providers={} hasImg={}".format( + e["ParentIndexNumber"], e["IndexNumber"], e["Name"], + list(e.get("ProviderIds",{}).keys()), + "Primary" in e.get("ImageTags",{})))' +``` + +Confirm: count matches files-on-disk, every episode has at least one of `Tvdb`/`Tmdb`/`Imdb` in providers, every episode has `Primary` image. If any row is missing providers/image, run a series-level metadata refresh: + +```bash +ssh user@nullstone "docker exec jellyfin curl -sf -X POST -H 'X-Emby-Token: $TOK' \ + 'http://127.0.0.1:8096/Items/$SERIES_ID/Refresh?MetadataRefreshMode=FullRefresh&ImageRefreshMode=FullRefresh&Recursive=true'" ``` Probe codec/streams (sanity check no transcode surprise): ```bash -ssh user@192.168.0.100 'docker exec jellyfin /usr/lib/jellyfin-ffmpeg/ffprobe -hide_banner "/media/movies/<Title> (<Year>)/<Title> (<Year>).mkv" 2>&1 | grep -E "Stream|Duration"' +ssh user@nullstone 'docker exec jellyfin /usr/lib/jellyfin-ffmpeg/ffprobe -hide_banner "/media/movies/<Title> (<Year>)/<Title> (<Year>).mkv" 2>&1 | grep -E "Stream|Duration"' ``` Open in browser → confirm artwork + title + duration look right. @@ -193,8 +241,9 @@ Commit + push: see `testing/DEPLOY.md` for git workflow. - [ ] One folder per item — no shared folders - [ ] NFO present iff TMDb override needed (`[tmdbid-NNNN]` token) - [ ] Subtitle sidecars correctly named (`.eng.srt`, not `.en.srt` or stripped) -- [ ] `/Library/Refresh` returned 204 / scheduled task ran (or LibraryMonitor auto-fired) -- [ ] `ProviderIds` populated on the new item (TMDb match successful) +- [ ] `Scan Media Library` task triggered via `/ScheduledTasks/Running/<id>` (NOT `/Library/Refresh`) and `LastExecutionResult.EndTimeUtc` advanced +- [ ] Per-series query (`/Shows/<id>/Episodes?Season=N`) shows expected count (do NOT trust `/Items/Counts`) +- [ ] `ProviderIds` populated on the new item (TMDb / TVDB / IMDb match successful) - [ ] Image artwork populated --- @@ -205,9 +254,10 @@ If a partial copy fails or the wrong item gets matched: ```bash # Remove the item folder -ssh user@192.168.0.100 'rm -rf "/home/user/media/movies/<Title> (<Year>)"' -# Force scan to drop it from index -ssh user@192.168.0.100 "docker exec jellyfin curl -s -X POST -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Library/Refresh" +ssh user@nullstone 'rm -rf "/home/user/media/movies/<Title> (<Year>)"' +# Trigger scan (NOT /Library/Refresh — see Step 4 known-broken table) to drop it from index +ssh user@nullstone "docker exec jellyfin curl -sf -X POST -H 'X-Emby-Token: $TOK' \ + http://127.0.0.1:8096/ScheduledTasks/Running/$SCAN_TASK_ID" ``` Source download on laptop is untouched — restage and retry per `docs/07:55-58`.