Merge pull request #3 from MrSleeps/feature/ollama-caption-enrichment

Add Ollama caption enrichment and hybrid scene search
2026-11-15 22:41:28 +00:00 · 2026-10-06 14:03:17 +01:00
parent 26e2054dc0 b96a4caa49
commit 3151021c50
9 changed files with 468 additions and 66 deletions
@@ -1,7 +1,13 @@
 MEILI_URL=http://127.0.0.1:7700
 MEILI_MASTER_KEY=change-me-now
 MEILI_INDEX=jellyfin_scenes
+HYBRID_SEMANTIC_RATIO=0.80
+
 MODEL_ID=google/siglip2-base-patch16-224
 DEVICE=cuda
 DATA_DIR=./data
 BATCH_SIZE=8
+
+OLLAMA_URL=http://127.0.0.1:11434
+CAPTION_MODEL=huihui_ai/qwen3-vl-abliterated:4b-instruct-q4_K_M
+CAPTION_TIMEOUT_SECONDS=180
@@ -1,29 +1,39 @@
-# Jellyfin Scene Search — first retrieval prototype
+# Jellyfin Scene Search — retrieval prototype

-This is intentionally **not a Jellyfin plugin yet**. It proves the core pipeline on two short clips:
+Local semantic scene search for Jellyfin-style media libraries.

-`video -> frames -> SigLIP2 image vectors -> SQLite canonical store -> Meilisearch -> natural-language query`
+The current prototype proves this pipeline:

-## Why SQLite *and* Meilisearch?
+`video -> frames -> SigLIP2 visual vectors -> SQLite canonical store -> optional Ollama captions -> Meilisearch hybrid search`

-Meilisearch itself persists its database in `./data/meili`, so an ordinary container restart does **not** lose the index.
+It is intentionally **not a Jellyfin plugin yet**. First we are validating retrieval quality and the persistence/indexing architecture on short clips.

-But Meilisearch is still treated as a **derived search index**. The canonical copy of each expensive GPU-generated embedding is stored in `./data/scenes.sqlite3` as a float16 BLOB. If Meilisearch is deleted, corrupted, upgraded incompatibly, or you simply want a fresh index, run `sync-meili` and it is rebuilt without touching the GPU or re-decoding the videos.
+## Persistence model

-This is the intended long-term split:
+Meilisearch is the fast serving index, but it is not the only copy of expensive AI work.

- Jellyfin: source of truth for media and permissions
- SQLite/PostgreSQL: source of truth for scene embeddings/indexing state
- Meilisearch: disposable, very fast serving index
+```text
+data/
+├── scenes.sqlite3     # canonical vectors, timestamps and captions
+├── thumbs/            # prototype frames used for visual verification/captioning
+└── meili/             # Meilisearch's persistent serving index
+```
+
+If Meilisearch is lost or rebuilt, `sync-meili` repopulates it from SQLite without re-running SigLIP or Ollama.
+
+Long term:
+
+- Jellyfin: source of truth for media, users and permissions
+- SQLite/PostgreSQL: source of truth for scene embeddings/captions/indexing state
+- Meilisearch: disposable, fast hybrid search index

 ## Prerequisites

 - Python 3.12+
 - NVIDIA driver/CUDA-supported PyTorch for GPU use
 - ffmpeg + ffprobe
- Docker / Docker Compose for the included Meilisearch instance
-
-The starter pins Meilisearch `v1.54.3` rather than `latest`.
+- Docker / Docker Compose
+- Ollama for optional stage-2 captions

 ## 1. Start Meilisearch

@@ -33,86 +43,159 @@ cp .env.example .env
 docker compose up -d
 ```

-Persistent Meili data goes to:
-
-```text
-./data/meili/
-```
-
 ## 2. Python environment

-Create and activate a venv, then install a CUDA-compatible PyTorch build for your machine first if required. After that:
+Create and activate a venv, install a CUDA-compatible PyTorch build if required, then:

 ```bash
 pip install -r requirements.txt
 ```

-The first index/search run downloads `google/siglip2-base-patch16-224` from Hugging Face. The checkpoint is about 1.5 GB.
+The first vector index/search run downloads `google/siglip2-base-patch16-224`.

-## 3. Initialise
+## 3. Initialise / migrate the canonical DB

 ```bash
 python -m scene_search.cli init
 ```

-## 4. Index the two test clips
+This is safe on an existing prototype DB. New stage-2 caption columns are added in place.

-For the prototype, use one frame every 2 seconds so we can judge retrieval quality properly:
+## 4. Index test clips
+
+For the prototype we use one frame every 2 seconds:

 ```bash
-python -m scene_search.cli index-file "/path/to/clip-one.mp4" --title "Clip one" --interval 2
-python -m scene_search.cli index-file "/path/to/clip-two.mp4" --title "Clip two" --interval 2
+python -m scene_search.cli index-file "/path/to/clip-one.mp4" --title "Test clip 1" --interval 2
+python -m scene_search.cli index-file "/path/to/clip-two.mp4" --title "Test clip 2" --interval 2
 ```

-The extracted test thumbnails are retained under `./data/thumbs/` so you can inspect returned matches visually.
-
-## 5. Feed/rebuild Meilisearch
+## 5. Vector-only Meilisearch sync

 ```bash
 python -m scene_search.cli sync-meili
 ```

-This can be run again after a Meilisearch rebuild. It reads the vectors from SQLite; it does **not** invoke SigLIP again.
+## 6. Stage 2: persistent visual captions

-## 6. Search
+Captioning is optional and pluggable. The prototype uses an Ollama vision model.
+
+The default is:
+
+```text
+huihui_ai/qwen3-vl-abliterated:4b-instruct-q4_K_M
+```
+
+It was selected for the prototype because the project requirement is content-neutral indexing of legitimate local media libraries, including mature films. It is an unofficial community model and can be replaced with any compatible Ollama vision model:
+
+```env
+OLLAMA_URL=http://127.0.0.1:11434
+CAPTION_MODEL=your-vision-model
+CAPTION_TIMEOUT_SECONDS=180
+```
+
+Pull the configured model before captioning:

 ```bash
-python -m scene_search.cli search "dog rolling in flowers"
-python -m scene_search.cli search "a person walking along a beach"
+ollama pull huihui_ai/qwen3-vl-abliterated:4b-instruct-q4_K_M
 ```

-A result looks like:
+Start with a few frames so you can inspect quality:

-```text
- 1. Clip one @ 00:00:34 score=0.83
-    /.../data/thumbs/<media-id>/frame_000018.jpg
+```bash
+python -m scene_search.cli caption --title "Test clip 1" --limit 5
 ```

-Open the referenced JPEG to see whether the semantic hit is genuinely good.
+Then caption everything not yet captioned:

-## Persistence model
-
-Two independent persistent stores intentionally exist:
-
-```text
-data/
-├── scenes.sqlite3     # canonical vectors + timestamps + indexing metadata
-├── thumbs/            # temporary/prototype visual verification frames
-└── meili/             # Meilisearch's own persistent database
+```bash
+python -m scene_search.cli caption
 ```

-For a full Jellyfin library we will probably stop retaining individual thumbnail files: Jellyfin already owns the Trickplay sheets. The canonical DB will instead store `jellyfin_item_id`, `media_source_id`, timestamp, Trickplay sheet/tile coordinates, model version, and the embedding.
+Or caption and immediately feed the results to Meili:

-## Next milestone after the two clips
+```bash
+python -m scene_search.cli caption --sync
+```

-If retrieval is convincing, replace `frames.py` with a Jellyfin Trickplay frame provider:
+Captions are saved after every successful frame. If captioning is interrupted, rerun the command and it resumes from frames that still need captions.

-1. enumerate Jellyfin media through its API;
-2. fetch Trickplay metadata and JPEG sheets;
-3. split sheets into timestamped tiles;
-4. generate the same SigLIP embeddings;
-5. store Jellyfin item ID + timestamp in SQLite;
-6. feed those records to the same Meilisearch index;
-7. add a play URL / plugin result that jumps to the timestamp.
+Use `--force` to regenerate existing captions with the currently configured model:

-No adult/NSFW classifier is part of this prototype. The stored representation is a content-neutral visual embedding.
+```bash
+python -m scene_search.cli caption --title "Test clip 1" --force
+```
+
+The caption prompt deliberately asks for factual visual descriptions and tells the model not to add labels such as `NSFW`, `adult`, `safe` or `unsafe`. Mature content may be described if clearly visible, but there is no permanent adult-content classifier.
+
+## 7. Hybrid search
+
+After captions exist, sync them:
+
+```bash
+python -m scene_search.cli sync-meili
+```
+
+Search then combines:
+
+- SigLIP2 image/text vector similarity
+- lexical matches against `title` and `caption`
+
+The balance is configurable:
+
+```env
+HYBRID_SEMANTIC_RATIO=0.80
+```
+
+A higher value favours visual semantic similarity; a lower value gives caption text more influence.
+
+```bash
+python -m scene_search.cli search "teenager walking outside"
+python -m scene_search.cli search "people walking down a street"
+```
+
+Search output shows the stored caption as well as the Meilisearch and SigLIP diagnostic scores.
+
+## Status
+
+```bash
+python -m scene_search.cli status
+```
+
+This shows media/frame counts, how many frames have canonical captions, and Meilisearch document/indexing status.
+
+## What is stored in Meilisearch?
+
+A stage-2 document contains fields like:
+
+```json
+{
+  "id": "...",
+  "media_id": "...",
+  "title": "Test clip 1",
+  "timestamp_ms": 42000,
+  "thumb_path": ".../frame_000022.jpg",
+  "caption": "A teenage boy standing outside near a building on a sunny day.",
+  "caption_model": "huihui_ai/qwen3-vl-abliterated:4b-instruct-q4_K_M",
+  "captioned_at": "2026-10-06 12:34:56",
+  "model_id": "google/siglip2-base-patch16-224",
+  "_vectors": {
+    "siglip2": ["..."]
+  }
+}
+```
+
+Meilisearch does not normally display the contents of `_vectors` in its Mini Dashboard.
+
+## Next milestones
+
+If caption + vector retrieval proves useful:
+
+1. group neighbouring frames into scenes/windows to remove duplicate results;
+2. caption one scene/window rather than every frame;
+3. replace local ffmpeg frames with Jellyfin Trickplay tiles;
+4. store Jellyfin item/media-source IDs and Trickplay coordinates;
+5. add subtitles/transcripts as another searchable field;
+6. return Jellyfin results that jump directly to the matched timestamp.
+
+No adult/NSFW classifier is part of the current design.
@@ -0,0 +1,64 @@
+from __future__ import annotations
+
+import base64
+from pathlib import Path
+
+import requests
+
+
+DEFAULT_CAPTION_PROMPT = (
+    "Describe only what is visibly present in this video frame in one concise sentence "
+    "for later visual search. Mention people, broad age group only when visually clear, "
+    "actions, setting, notable objects, clothing, weather, and other distinctive details. "
+    "If nudity, sexual activity, violence, gore, or other mature content is clearly visible, "
+    "describe it factually rather than omitting or euphemising it. Do not infer names, "
+    "identities, relationships, intent, or anything not visible. Do not label the image as "
+    "safe, unsafe, adult, NSFW, or similar. Output the description only."
+)
+
+
+class OllamaCaptioner:
+    def __init__(
+        self,
+        url: str,
+        model: str,
+        timeout_seconds: int = 180,
+        prompt: str = DEFAULT_CAPTION_PROMPT,
+    ) -> None:
+        self.url = url.rstrip("/")
+        self.model = model
+        self.timeout_seconds = timeout_seconds
+        self.prompt = prompt
+        self.session = requests.Session()
+
+    def caption(self, image_path: Path) -> str:
+        image_b64 = base64.b64encode(image_path.read_bytes()).decode("ascii")
+        response = self.session.post(
+            f"{self.url}/api/chat",
+            json={
+                "model": self.model,
+                "stream": False,
+                "messages": [
+                    {
+                        "role": "user",
+                        "content": self.prompt,
+                        "images": [image_b64],
+                    }
+                ],
+                "options": {
+                    "temperature": 0.1,
+                },
+            },
+            timeout=self.timeout_seconds,
+        )
+        if not response.ok:
+            raise RuntimeError(
+                f"Ollama {response.status_code}: {response.text[:500]}"
+            )
+
+        data = response.json()
+        content = (data.get("message") or {}).get("content")
+        if not isinstance(content, str) or not content.strip():
+            raise RuntimeError(f"Ollama returned no caption: {data}")
+
+        return " ".join(content.strip().split())
@@ -6,6 +6,7 @@ from pathlib import Path

 from PIL import Image

+from .captioner import OllamaCaptioner
 from .config import settings
 from .database import connect, init_db, pack_vector, unpack_vector
 from .embedder import SiglipEmbedder
@@ -133,6 +134,7 @@ def cmd_sync_meili(_: argparse.Namespace) -> None:
        cursor = con.execute(
            """
            SELECT f.id, f.media_id, f.timestamp_ms, f.thumb_path, f.embedding_f16,
+                   f.caption, f.caption_model, f.captioned_at,
                   m.title, m.source_path, m.embedding_dim, m.model_id
            FROM frames f
            JOIN media m ON m.id = f.media_id
@@ -156,6 +158,9 @@ def cmd_sync_meili(_: argparse.Namespace) -> None:
                    "timestamp_ms": row["timestamp_ms"],
                    "thumb_path": row["thumb_path"],
                    "model_id": row["model_id"],
+                    "caption": row["caption"],
+                    "caption_model": row["caption_model"],
+                    "captioned_at": row["captioned_at"],
                    "_vectors": {"siglip2": vec.tolist()},
                })
            meili.add_documents(docs)
@@ -166,6 +171,100 @@ def cmd_sync_meili(_: argparse.Namespace) -> None:



+def cmd_caption(args: argparse.Namespace) -> None:
+    init_db(settings.db_path)
+
+    conditions = []
+    params: list[object] = []
+    if not args.force:
+        conditions.append("(f.caption IS NULL OR f.caption_model != ?)")
+        params.append(settings.caption_model)
+    if args.media_id:
+        conditions.append("f.media_id = ?")
+        params.append(args.media_id)
+    if args.title:
+        conditions.append("m.title = ?")
+        params.append(args.title)
+
+    where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
+    limit = ""
+    if args.limit is not None:
+        limit = " LIMIT ?"
+        params.append(args.limit)
+
+    with connect(settings.db_path) as con:
+        rows = con.execute(
+            f"""
+            SELECT f.id, f.media_id, f.timestamp_ms, f.thumb_path, f.caption,
+                   f.caption_model, m.title
+            FROM frames f
+            JOIN media m ON m.id = f.media_id
+            {where}
+            ORDER BY m.title, f.timestamp_ms
+            {limit}
+            """,
+            params,
+        ).fetchall()
+
+    if not rows:
+        print("No frames need captioning.")
+        return
+
+    print(
+        f"Captioning {len(rows)} frame(s) with {settings.caption_model} "
+        f"via {settings.ollama_url}"
+    )
+    captioner = OllamaCaptioner(
+        settings.ollama_url,
+        settings.caption_model,
+        settings.caption_timeout_seconds,
+    )
+
+    completed = 0
+    failed = 0
+    for row in rows:
+        image_path = Path(row["thumb_path"])
+        if not image_path.is_file():
+            failed += 1
+            print(
+                f"SKIP {row['title']} @ {format_ts(int(row['timestamp_ms']))}: "
+                f"missing thumbnail {image_path}"
+            )
+            continue
+
+        try:
+            caption = captioner.caption(image_path)
+        except Exception as exc:
+            failed += 1
+            print(
+                f"FAIL {row['title']} @ {format_ts(int(row['timestamp_ms']))}: {exc}"
+            )
+            continue
+
+        with connect(settings.db_path) as con:
+            con.execute(
+                """
+                UPDATE frames
+                SET caption = ?, caption_model = ?, captioned_at = CURRENT_TIMESTAMP
+                WHERE id = ?
+                """,
+                (caption, settings.caption_model, row["id"]),
+            )
+
+        completed += 1
+        print(
+            f"{completed:4d}/{len(rows)} {row['title']} @ "
+            f"{format_ts(int(row['timestamp_ms']))}: {caption}"
+        )
+
+    print(f"Captioning complete: {completed} saved, {failed} failed.")
+    if completed:
+        if args.sync:
+            cmd_sync_meili(args)
+        else:
+            print("Next: python -m scene_search.cli sync-meili")
+
+
 def cmd_status(_: argparse.Namespace) -> None:
    init_db(settings.db_path)

@@ -174,7 +273,9 @@ def cmd_status(_: argparse.Namespace) -> None:
    with connect(settings.db_path) as con:
        media_rows = con.execute(
            """
-            SELECT m.id, m.title, m.interval_ms, m.model_id, COUNT(f.id) AS frames
+            SELECT m.id, m.title, m.interval_ms, m.model_id,
+                   COUNT(f.id) AS frames,
+                   SUM(CASE WHEN f.caption IS NOT NULL THEN 1 ELSE 0 END) AS captioned
            FROM media m
            LEFT JOIN frames f ON f.media_id = m.id
            GROUP BY m.id, m.title, m.interval_ms, m.model_id
@@ -189,7 +290,8 @@ def cmd_status(_: argparse.Namespace) -> None:
        print()
        for row in media_rows:
            print(
-                f"- {row['title']}: {row['frames']} frames "
+                f"- {row['title']}: {row['frames']} frames, "
+                f"{row['captioned']} captioned "
                f"({row['interval_ms'] / 1000:g}s interval)"
            )

@@ -212,7 +314,12 @@ def cmd_search(args: argparse.Namespace) -> None:
    embedder = build_embedder()
    vector = embedder.embed_text(args.query)
    meili = Meili(settings.meili_url, settings.meili_master_key, settings.meili_index)
-    result = meili.search_vector(vector, args.limit)
+    result = meili.search_vector(
+        vector,
+        args.query,
+        args.limit,
+        settings.hybrid_semantic_ratio,
+    )

    hits = result.get("hits", [])
    if not hits:
@@ -267,6 +374,9 @@ def cmd_search(args: argparse.Namespace) -> None:
            f"{format_ts(int(hit.get('timestamp_ms', 0)))}"
            f"{meili_text}{siglip_text}"
        )
+        caption = hit.get("caption")
+        if caption:
+            print(f"    {caption}")
        print(f"    {hit.get('thumb_path')}")


@@ -287,10 +397,18 @@ def make_parser() -> argparse.ArgumentParser:
    sync = sub.add_parser("sync-meili", help="Rebuild/feed Meilisearch from canonical SQLite vectors")
    sync.set_defaults(func=cmd_sync_meili)

+    caption = sub.add_parser("caption", help="Generate persistent captions with an Ollama vision model")
+    caption.add_argument("--media-id", help="Only caption frames from one media ID")
+    caption.add_argument("--title", help="Only caption frames from one exact media title")
+    caption.add_argument("--limit", type=int, help="Maximum number of frames to caption")
+    caption.add_argument("--force", action="store_true", help="Regenerate captions even if already present")
+    caption.add_argument("--sync", action="store_true", help="Sync captions to Meilisearch when finished")
+    caption.set_defaults(func=cmd_caption)
+
    status = sub.add_parser("status", help="Show canonical SQLite and Meilisearch index status")
    status.set_defaults(func=cmd_status)

-    search = sub.add_parser("search", help="Semantic scene search")
+    search = sub.add_parser("search", help="Hybrid caption + visual scene search")
    search.add_argument("query")
    search.add_argument("--limit", type=int, default=10)
    search.set_defaults(func=cmd_search)
@@ -14,11 +14,21 @@ class Settings:
    meili_url: str = os.getenv("MEILI_URL", "http://127.0.0.1:7700").rstrip("/")
    meili_master_key: str = os.getenv("MEILI_MASTER_KEY", "change-me-now")
    meili_index: str = os.getenv("MEILI_INDEX", "jellyfin_scenes")
+    hybrid_semantic_ratio: float = float(os.getenv("HYBRID_SEMANTIC_RATIO", "0.80"))
+
    model_id: str = os.getenv("MODEL_ID", "google/siglip2-base-patch16-224")
    device: str = os.getenv("DEVICE", "cuda")
-    data_dir: Path = Path(os.getenv("DATA_DIR", "./data")).resolve()
    batch_size: int = int(os.getenv("BATCH_SIZE", "8"))

+    ollama_url: str = os.getenv("OLLAMA_URL", "http://127.0.0.1:11434").rstrip("/")
+    caption_model: str = os.getenv(
+        "CAPTION_MODEL",
+        "huihui_ai/qwen3-vl-abliterated:4b-instruct-q4_K_M",
+    )
+    caption_timeout_seconds: int = int(os.getenv("CAPTION_TIMEOUT_SECONDS", "180"))
+
+    data_dir: Path = Path(os.getenv("DATA_DIR", "./data")).resolve()
+
    @property
    def db_path(self) -> Path:
        return self.data_dir / "scenes.sqlite3"
@@ -32,6 +32,9 @@ CREATE TABLE IF NOT EXISTS frames (
    timestamp_ms INTEGER NOT NULL,
    thumb_path TEXT NOT NULL,
    embedding_f16 BLOB NOT NULL,
+    caption TEXT,
+    caption_model TEXT,
+    captioned_at TEXT,
    UNIQUE(media_id, frame_number)
 );

@@ -52,9 +55,22 @@ def connect(path: Path) -> Iterator[sqlite3.Connection]:
        con.close()


+def _ensure_frame_caption_columns(con: sqlite3.Connection) -> None:
+    columns = {row["name"] for row in con.execute("PRAGMA table_info(frames)")}
+    additions = {
+        "caption": "TEXT",
+        "caption_model": "TEXT",
+        "captioned_at": "TEXT",
+    }
+    for name, sql_type in additions.items():
+        if name not in columns:
+            con.execute(f"ALTER TABLE frames ADD COLUMN {name} {sql_type}")
+
+
 def init_db(path: Path) -> None:
    with connect(path) as con:
        con.executescript(SCHEMA)
+        _ensure_frame_caption_columns(con)


 def pack_vector(vector: np.ndarray) -> bytes:
@@ -42,7 +42,11 @@ class Meili:
        ))
        self._check(self.session.put(
            f"{self.url}/indexes/{self.index}/settings/filterable-attributes",
-            json=["media_id", "title"],
+            json=["media_id", "title", "caption_model"],
+        ))
+        self._check(self.session.put(
+            f"{self.url}/indexes/{self.index}/settings/searchable-attributes",
+            json=["title", "caption"],
        ))

    def add_documents(self, documents: list[dict]) -> None:
@@ -60,10 +64,20 @@ class Meili:
            self.session.get(f"{self.url}/indexes/{self.index}/stats")
        ).json()

-    def search_vector(self, vector: Iterable[float], limit: int = 10) -> dict:
+    def search_vector(
+        self,
+        vector: Iterable[float],
+        query: str,
+        limit: int = 10,
+        semantic_ratio: float = 0.8,
+    ) -> dict:
        payload = {
+            "q": query,
            "vector": self._json_vector(vector),
-            "hybrid": {"embedder": "siglip2", "semanticRatio": 1.0},
+            "hybrid": {
+                "embedder": "siglip2",
+                "semanticRatio": semantic_ratio,
+            },
            "limit": limit,
            "showRankingScore": True,
        }
@@ -0,0 +1,35 @@
+from __future__ import annotations
+
+from pathlib import Path
+import tempfile
+import unittest
+from unittest.mock import Mock
+
+from scene_search.captioner import OllamaCaptioner
+
+
+class OllamaCaptionerTests(unittest.TestCase):
+    def test_caption_sends_image_and_returns_compact_text(self) -> None:
+        captioner = OllamaCaptioner("http://localhost:11434", "vision-test")
+        response = Mock()
+        response.ok = True
+        response.json.return_value = {
+            "message": {"content": "  A person walking\n down a street.  "}
+        }
+        captioner.session.post = Mock(return_value=response)
+
+        with tempfile.TemporaryDirectory() as tmp:
+            image = Path(tmp) / "frame.jpg"
+            image.write_bytes(b"fake-jpeg")
+            result = captioner.caption(image)
+
+        self.assertEqual(result, "A person walking down a street.")
+        call = captioner.session.post.call_args
+        payload = call.kwargs["json"]
+        self.assertEqual(payload["model"], "vision-test")
+        self.assertEqual(payload["messages"][0]["role"], "user")
+        self.assertTrue(payload["messages"][0]["images"][0])
+
+
+if __name__ == "__main__":
+    unittest.main()
@@ -0,0 +1,56 @@
+from __future__ import annotations
+
+import sqlite3
+import tempfile
+from pathlib import Path
+import unittest
+
+from scene_search.database import init_db
+
+
+class DatabaseMigrationTests(unittest.TestCase):
+    def test_existing_frames_table_gains_caption_columns(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            path = Path(tmp) / "scenes.sqlite3"
+            con = sqlite3.connect(path)
+            con.executescript(
+                """
+                CREATE TABLE media (
+                    id TEXT PRIMARY KEY,
+                    source_path TEXT NOT NULL,
+                    title TEXT NOT NULL,
+                    content_sha256 TEXT NOT NULL,
+                    size_bytes INTEGER NOT NULL,
+                    mtime_ns INTEGER NOT NULL,
+                    interval_ms INTEGER NOT NULL,
+                    model_id TEXT NOT NULL,
+                    embedding_dim INTEGER NOT NULL,
+                    created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
+                    updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
+                );
+                CREATE TABLE frames (
+                    id TEXT PRIMARY KEY,
+                    media_id TEXT NOT NULL REFERENCES media(id) ON DELETE CASCADE,
+                    frame_number INTEGER NOT NULL,
+                    timestamp_ms INTEGER NOT NULL,
+                    thumb_path TEXT NOT NULL,
+                    embedding_f16 BLOB NOT NULL,
+                    UNIQUE(media_id, frame_number)
+                );
+                """
+            )
+            con.close()
+
+            init_db(path)
+
+            con = sqlite3.connect(path)
+            columns = {row[1] for row in con.execute("PRAGMA table_info(frames)")}
+            con.close()
+
+            self.assertIn("caption", columns)
+            self.assertIn("caption_model", columns)
+            self.assertIn("captioned_at", columns)
+
+
+if __name__ == "__main__":
+    unittest.main()