playbooks/import-media: v1.1 — fix two Jellyfin endpoint bugs + nullstone alias
Some checks are pending
secret-scan / gitleaks (HEAD + history) (push) Waiting to run
secret-scan / detect-secrets (entropy + cross-tool) (push) Waiting to run
secret-scan / summary (push) Blocked by required conditions

Step 4 rewritten: /Library/Refresh is a silent no-op on this build,
must POST to /ScheduledTasks/Running/<scan-task-id> directly. Old
endpoint moved to known-broken table.

Step 5 rewritten: /Items/Counts is scope-cached and stays stale
even after items are indexed. Use /Shows/<id>/Episodes?Season=<NN>
as authoritative verify with provider + image-tag checks.

Both bugs surfaced in archer-s02-2009 run. LibraryMonitor inotify
auto-fire also confirmed broken (failed on lilo-stitch-2002 and
archer-s02-2009 runs).

Replaced user@192.168.0.100 (LAN IP, RFC1918 — flagged by
gitleaks lan-ip-rfc1918) with user@nullstone throughout. SSH config
already aliases nullstone -> 192.168.0.100. Aligns with CLAUDE.md
two-file doc rule: IPs belong in SYSTEM.md, not operational docs.
This commit is contained in:
s8n 2026-05-10 06:49:02 +01:00
parent fcac178882
commit 5b80cfd095
2 changed files with 91 additions and 30 deletions

View file

@ -1,10 +1,21 @@
# Import-media playbook changelog
## v1.1 — 2026-05-10
Two Jellyfin endpoint bugs found during `archer-s02-2009` run, codified into the playbook:
- **`POST /Library/Refresh` is a silent no-op** on this Jellyfin build — returns HTTP 204 but the `Scan Media Library` scheduled task does NOT execute. Step 4 rewritten to fetch the scan-task ID from `/ScheduledTasks` and POST to `/ScheduledTasks/Running/<id>` instead. Old endpoint added to a "known broken" table.
- **`/Items/Counts` is scope-cached** and stays stale even after items are indexed (counts stayed at 230 after 13 new eps landed). Step 5 rewritten to use the per-series authoritative query `/Shows/<id>/Episodes?Season=<NN>` with provider + image-tag verification, plus a per-series `Items/<id>/Refresh` recipe for missing metadata.
- LibraryMonitor inotify auto-fire wording removed from Step 4 (failed on both recorded runs — Lilo & Stitch + Archer S02). Manual task trigger is now mandatory.
- Verification checklist updated to reference the task-trigger endpoint and per-series query.
- Rollback section: replaced `/Library/Refresh` invocation with `/ScheduledTasks/Running/<id>`.
## v1.0 — 2026-05-10
Initial playbook. 7 steps from staging on onyx → rsync to nullstone → verify scan + counts → optional subtitle pass → run-log.
Gaps flagged for future versions:
- v1.1 will add canonical `bin/import-media.sh` wrapper once `bin/cleanup-import.sh` and `bin/normalize.py` are extracted from docs/07 and docs/08 (ROADMAP M6).
- v1.2 will add a TV multi-season import section (currently only single-season example).
- v1.3 will add NFO override pattern with worked example for a wrong-TMDb-match recovery.
- v1.2 will add canonical `bin/import-media.sh` wrapper once `bin/cleanup-import.sh` and `bin/normalize.py` are extracted from docs/07 and docs/08 (ROADMAP M6). (Bumped from v1.1 since v1.1 was needed for the Jellyfin endpoint fixes.)
- v1.3 will add a TV multi-season import section (currently only single-season example).
- v1.4 will add NFO override pattern with worked example for a wrong-TMDb-match recovery.
- v1.5: document API-token retrieval (mint a permanent ApiKeys row labelled `import-pipeline` instead of pulling a session token from `Devices` table).

View file

@ -3,7 +3,7 @@
> Repeatable workflow for adding a new movie or TV show to ARRFLIX.
> Mirror the format of `playbooks/subtitles/README.md` (CHANGELOG-driven, runs/ folder for per-import logs).
Version: **v1.0** (2026-05-10)
Version: **v1.1** (2026-05-10)
---
@ -26,7 +26,7 @@ Before importing:
- [ ] Source file is canonical quality (4K source-permitting, AI-upscale otherwise — no 480p filler, no junk encodes per `README.md:41`)
- [ ] Title and year confirmed (TMDb / IMDb)
- [ ] No copy already in library — check via `curl -s -H "X-Emby-Token: $TOK" 'https://arrflix.s8n.ru/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm=<title>'`
- [ ] Container running healthy: `ssh user@192.168.0.100 'docker ps --filter name=jellyfin --format "{{.Names}} {{.Status}}"'`
- [ ] Container running healthy: `ssh user@nullstone 'docker ps --filter name=jellyfin --format "{{.Names}} {{.Status}}"'`
---
@ -82,7 +82,7 @@ If source dir has extra files (NFO, JPG, sample, torrent, RARBG.txt, etc.) — s
```bash
rsync -a --info=progress2 --no-owner --no-group \
"/home/admin/staging-jelly/<Title> (<Year>)" \
user@192.168.0.100:/home/user/media/movies/
user@nullstone:/home/user/media/movies/
```
### TV (one season)
@ -90,7 +90,7 @@ rsync -a --info=progress2 --no-owner --no-group \
```bash
rsync -a --info=progress2 --no-owner --no-group \
"/home/admin/staging-jelly/<Series> (<Year>)/Season <NN>" \
'user@192.168.0.100:/home/user/media/tv/<Series> (<Year>)/'
'user@nullstone:/home/user/media/tv/<Series> (<Year>)/'
```
`scp -r` works too but `rsync -a` preserves perms-by-mode + resumes on interrupt.
@ -102,7 +102,7 @@ rsync -a --info=progress2 --no-owner --no-group \
Files must be `user:user` 644 (per live audit, `stat -c '%U:%G %a'`):
```bash
ssh user@192.168.0.100 '
ssh user@nullstone '
cd "/home/user/media/movies/<Title> (<Year>)"
find . -type d -exec chmod 755 {} \;
find . -type f -exec chmod 644 {} \;
@ -114,44 +114,92 @@ ssh user@192.168.0.100 '
---
## Step 4 — Verify scan picked up
## Step 4 — Trigger scan (do not rely on auto-refresh)
Jellyfin's `LibraryMonitor` watches the bind mount and auto-refreshes within ~13 s of file landing. Confirm in logs:
> **v1.1 change:** prior versions said `LibraryMonitor` auto-fires within 13 s and `POST /Library/Refresh` is the fallback. **Both proved unreliable across consecutive runs (`lilo-stitch-2002`, `archer-s02-2009`).** Always trigger the `Scan Media Library` scheduled task directly.
```bash
ssh user@192.168.0.100 'docker logs jellyfin --tail 20 | grep -E "LibraryMonitor.*<Title>|<Title>.*refreshed"'
```
Expect: `LibraryMonitor: <Title> ... will be refreshed.`
If LibraryMonitor doesn't fire (rare), force a manual refresh:
### 4a. Pull task ID once (cache in shell or env)
```bash
TOK=<your-admin-token>
ssh user@192.168.0.100 "docker exec jellyfin curl -s -X POST -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Library/Refresh"
SCAN_TASK_ID=$(ssh user@nullstone \
"docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/ScheduledTasks" \
| python3 -c 'import sys,json
[print(t["Id"]) for t in json.load(sys.stdin) if t["Name"]=="Scan Media Library"]')
echo "$SCAN_TASK_ID"
```
### 4b. Trigger the task (returns 204 + actually runs)
```bash
ssh user@nullstone \
"docker exec jellyfin curl -sf -X POST -H 'X-Emby-Token: $TOK' \
http://127.0.0.1:8096/ScheduledTasks/Running/$SCAN_TASK_ID -w 'HTTP:%{http_code}\n'"
```
### Known broken endpoints (do NOT use)
| Endpoint | Why broken |
|---|---|
| `LibraryMonitor` (inotify) auto-fire | Bind-mount + userns-remap inotify events do not propagate reliably; observed silent miss on every recent run. |
| `POST /Library/Refresh` | Returns HTTP 204 but the `Scan Media Library` scheduled task does NOT fire. Silent no-op. Use `/ScheduledTasks/Running/<id>` instead. |
### 4c. Confirm task actually started
```bash
ssh user@nullstone \
"docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \
http://127.0.0.1:8096/ScheduledTasks/$SCAN_TASK_ID" \
| python3 -c 'import sys,json;d=json.load(sys.stdin);print(d["State"], d.get("CurrentProgressPercentage","-"))'
```
Expect: `Running <pct>` while scanning, then `Idle -` when done. `LastExecutionResult.EndTimeUtc` should advance past the trigger time.
---
## Step 5 — Verify count bump + metadata + playback
## Step 5 — Verify items added + metadata + playback
> **v1.1 change:** **Do NOT use `/Items/Counts` for verification.** It is scope-cached/user-default-view-cached and stays stale even when the scan completed and items were added (observed in `archer-s02-2009`: counts stayed at 230 long after 13 eps were indexed). Use the per-item or per-series query as authoritative.
### Movies — search by title
```bash
TOK=<token>
ssh user@192.168.0.100 "docker exec jellyfin curl -s -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Items/Counts" | python3 -m json.tool
ssh user@nullstone "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \
'http://127.0.0.1:8096/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm=<Title>'" \
| python3 -c "import sys,json;[print(i['Id'],i['Name']) for i in json.load(sys.stdin)['Items']]"
```
`MovieCount` (or `EpisodeCount`) should be `+1` (or `+N`).
Find the new item ID:
### TV — list episodes for the affected series + season (authoritative)
```bash
ssh user@192.168.0.100 "docker exec jellyfin curl -s -H 'X-Emby-Token: $TOK' 'http://127.0.0.1:8096/Items?Recursive=true&IncludeItemTypes=Movie&searchTerm=<Title>'" | python3 -c "import sys,json;[print(i['Id'],i['Name']) for i in json.load(sys.stdin)['Items']]"
SERIES_ID=$(ssh user@nullstone "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \
'http://127.0.0.1:8096/Items?Recursive=true&IncludeItemTypes=Series&searchTerm=<Series>'" \
| python3 -c 'import sys,json;print(json.load(sys.stdin)["Items"][0]["Id"])')
ssh user@nullstone "docker exec jellyfin curl -sf -H 'X-Emby-Token: $TOK' \
'http://127.0.0.1:8096/Shows/$SERIES_ID/Episodes?Season=<NN>&fields=Path,ProviderIds,ImageTags'" \
| python3 -c 'import sys,json
d=json.load(sys.stdin)
print("COUNT:", len(d["Items"]))
for e in d["Items"]:
print(" S{}E{:02d} {} providers={} hasImg={}".format(
e["ParentIndexNumber"], e["IndexNumber"], e["Name"],
list(e.get("ProviderIds",{}).keys()),
"Primary" in e.get("ImageTags",{})))'
```
Confirm: count matches files-on-disk, every episode has at least one of `Tvdb`/`Tmdb`/`Imdb` in providers, every episode has `Primary` image. If any row is missing providers/image, run a series-level metadata refresh:
```bash
ssh user@nullstone "docker exec jellyfin curl -sf -X POST -H 'X-Emby-Token: $TOK' \
'http://127.0.0.1:8096/Items/$SERIES_ID/Refresh?MetadataRefreshMode=FullRefresh&ImageRefreshMode=FullRefresh&Recursive=true'"
```
Probe codec/streams (sanity check no transcode surprise):
```bash
ssh user@192.168.0.100 'docker exec jellyfin /usr/lib/jellyfin-ffmpeg/ffprobe -hide_banner "/media/movies/<Title> (<Year>)/<Title> (<Year>).mkv" 2>&1 | grep -E "Stream|Duration"'
ssh user@nullstone 'docker exec jellyfin /usr/lib/jellyfin-ffmpeg/ffprobe -hide_banner "/media/movies/<Title> (<Year>)/<Title> (<Year>).mkv" 2>&1 | grep -E "Stream|Duration"'
```
Open in browser → confirm artwork + title + duration look right.
@ -193,8 +241,9 @@ Commit + push: see `testing/DEPLOY.md` for git workflow.
- [ ] One folder per item — no shared folders
- [ ] NFO present iff TMDb override needed (`[tmdbid-NNNN]` token)
- [ ] Subtitle sidecars correctly named (`.eng.srt`, not `.en.srt` or stripped)
- [ ] `/Library/Refresh` returned 204 / scheduled task ran (or LibraryMonitor auto-fired)
- [ ] `ProviderIds` populated on the new item (TMDb match successful)
- [ ] `Scan Media Library` task triggered via `/ScheduledTasks/Running/<id>` (NOT `/Library/Refresh`) and `LastExecutionResult.EndTimeUtc` advanced
- [ ] Per-series query (`/Shows/<id>/Episodes?Season=N`) shows expected count (do NOT trust `/Items/Counts`)
- [ ] `ProviderIds` populated on the new item (TMDb / TVDB / IMDb match successful)
- [ ] Image artwork populated
---
@ -205,9 +254,10 @@ If a partial copy fails or the wrong item gets matched:
```bash
# Remove the item folder
ssh user@192.168.0.100 'rm -rf "/home/user/media/movies/<Title> (<Year>)"'
# Force scan to drop it from index
ssh user@192.168.0.100 "docker exec jellyfin curl -s -X POST -H 'X-Emby-Token: $TOK' http://127.0.0.1:8096/Library/Refresh"
ssh user@nullstone 'rm -rf "/home/user/media/movies/<Title> (<Year>)"'
# Trigger scan (NOT /Library/Refresh — see Step 4 known-broken table) to drop it from index
ssh user@nullstone "docker exec jellyfin curl -sf -X POST -H 'X-Emby-Token: $TOK' \
http://127.0.0.1:8096/ScheduledTasks/Running/$SCAN_TASK_ID"
```
Source download on laptop is untouched — restage and retry per `docs/07:55-58`.