Seedbox sync stalled / not draining

The seedbox→homelab pull is the single most misleading thing in the stack to diagnose, because the same dataset is mounted under two names and LXC processes show up in the host’s process list. Read this before concluding anything is “broken.” (Reconciled 2026-06-13.)

Symptom
  • Uptime Kuma monitor 57 “Cron: Seedbox Sync” shows DOWN, or a Gotify alert “no successful seedbox sync in Xh”.
  • The seedbox quota isn’t dropping, or /mnt/seedbox “looks empty.”
  • An rsync to …/completed/ → /mnt/seedbox/ appears to be running but moving nothing.
Diagnose

The real sync runs on CT100, not pve. It’s /usr/local/bin/seedbox-sync.sh on CT100, cron */15, pulling delgross@ismene.usbx.me:~/downloads/{completed,transmission,complete}/ into $DEST=/mnt/seedbox.

Two traps that make a healthy sync look broken:

  1. Dual mount name. The ingest ZFS dataset nvmepool/ingest is mounted at /mnt/seedbox inside CT100 and at /nvmepool/ingest on the pve host — same data (~1.3 TB), two names. The pve-host /mnt/seedbox is a dead empty dir (just a stale Music-ImportFailed stub). Looking there and seeing “empty” tells you nothing.
  2. LXC procs in the host ps. CT100’s rsync/cron processes appear in pve’s ps -eo with host PIDs and a /mnt/seedbox target that means the live dataset in CT100’s namespace but a dead dir on the host. Always run cat /proc/<pid>/cgrouplxc/100 = it’s CT100’s process, not pve’s.

Don’t trust “rsync is running” — sample throughput: cat /sys/class/net/eth0/statistics/rx_bytes twice 5s apart inside CT100. A real pull is ~6–13 MB/s; a true stall is ~0.

Note: a 1+ TB backlog legitimately runs up to 6 h per leg, during which the */15 cron just logs “flock held, skipping.” That is normal, not failure.

Fix
  • Never add a second sync on pve. It writes to /nvmepool/ingest = the same dataset CT100 syncs, and both use --remove-source-files → they race and can corrupt mid-transfer. (Claude did exactly this in error on 2026-06-13; reverted.)
  • NZBGet client + remote-path-mapping host must be 192.168.8.100 (the CT100 tunnel), not 192.168.8.221/pve. A wrong host silently fails grabs and breaks import path resolution in Bookshelf/Chaptarr too.
  • The progress heartbeat (added to seedbox-sync.sh 2026-06-13) pushes Kuma 57 every 5 min based on real eth0 RX, so a healthy multi-hour drain stays green and a true stall flags DOWN within the 30-min interval. If 57 is flapping during a normal drain, confirm the heartbeat block is present in the script.
  • If genuinely wedged: check the NZBGet/Transmission tunnels (autossh-* on CT100) and the seedbox daemon, then let the */15 cron pick up — --partial resumes, nothing is lost.
Verify
  • Inside CT100: throughput sample shows MB/s, not KB/s.
  • Seedbox quota -s is dropping; ~/downloads/nzbget/completed/Movies shrinking.
  • Kuma monitor 57 returns UP (it carries an OK moved NNN MB message on success).