Document incremental caption Meili sync

This commit is contained in:
Sleeps
2026-10-07 14:28:32 +01:00
parent 7e8d8648bb
commit dfa1d2fff0
+2 -1
View File
@@ -189,6 +189,7 @@ CAPTION_SPECIALIST_MODEL=aha2025/llama-joycaption-beta-one-hf-llava:Q4_K_M
CAPTION_FORCE_SPECIALIST=false
CAPTION_TIMEOUT_SECONDS=180
CAPTION_MAX_TOKENS=120
CAPTION_MEILI_BATCH_SIZE=500
CAPTION_PROMPT_FILE=./data/caption_prompt.txt
CAPTION_SPECIALIST_PROMPT_FILE=./data/caption_prompt_specialist.txt
@@ -528,7 +529,7 @@ Library removal is targeted. **Apply selection only** (or saving the plugin sett
Use **Apply library selection / index** to apply the same authoritative set and start a backend indexing job. An empty selection is allowed only after an explicit confirmation and removes all Jellyfin-backed Scene Search data. The plugin shows an overall percentage/progress bar across all selected libraries as well as the current library, item and frame progress, and polls until the job completes. When ingestion finishes, the backend automatically rebuilds Meilisearch, so newly indexed vectors are searchable without a separate `sync-meili` command. The completed summary also exposes drill-down buttons for **Failed**, **Removed** and **Indexed** items. Those details are written to `data/last-index-report.json`, are paged on demand instead of being included in the 2-second status polling payload, and survive a dashboard refresh or container restart. Unchanged items are counted but intentionally not persisted individually because large libraries can contain tens of thousands of them.
The caption control is stateful: **Start / resume captioning** immediately changes to **Starting...** while the potentially large SQLite frame-selection/counting pass runs in a background thread, then becomes **Pause** while work is active and **Resume** when paused or a pause is pending. A separate **Stop** button cooperatively ends the job at the next safe boundary. If Ollama is already processing a frame, that frame finishes and is saved first; stopping a paused job wakes it only long enough to terminate. Every successful caption is committed immediately, so Start / resume later continues from the remaining uncaptained frames. Stop deliberately does not begin the expensive final Meilisearch rebuild; the saved captions are incorporated when a later caption run completes. **Force recaption existing frames** deliberately regenerates captions, but reuses saved routing decisions while the routing policy is still current.
The caption control is stateful: **Start / resume captioning** immediately changes to **Starting...** while the potentially large SQLite frame-selection/counting pass runs in a background thread, then becomes **Pause** while work is active and **Resume** when paused or a pause is pending. A separate **Stop** button cooperatively ends the job at the next safe boundary. If Ollama is already processing a frame, that frame finishes and is saved first; stopping a paused job wakes it only long enough to terminate. Every successful caption is committed immediately to SQLite and caption/routing metadata is pushed to Meilisearch incrementally in batches of `CAPTION_MEILI_BATCH_SIZE` (default 500), so captions become searchable during a long run instead of waiting for a multi-million-document rebuild. Stop flushes the current partial batch before ending. SQLite tracks frames whose latest caption metadata has not reached Meilisearch, and later caption runs catch those up automatically. The old full `sync-meili` rebuild remains available as a repair/reconciliation command, but normal caption jobs no longer run it at the end. **Force recaption existing frames** deliberately regenerates captions, but reuses saved routing decisions while the routing policy is still current.
Captioning first runs the local routing classifier, then sends frames to the configured general or specialist Ollama model. Search is blocked only while the local routing GPU is in use and while Meilisearch is rebuilt; it remains available during the long remote Ollama caption phase.