SOP — the back-catalogue corpus

Turn delivered videos into style sheets and reconstructed scripts, then index them into the portal. This is what the script-writing agent learns from, and the reason it exists is that the original scripts were lost. The pipeline code is video-pipeline/stages/R*; this file is the procedure and the safety rules.

Where the artefacts live

Per video, work/corpus/<frameio-asset-id>/:

These are the expensive part of the project and they are not gitignored by accident. work/ is ignored, so the copy of record is video-pipeline/corpus/ on branch corpus-backup. Back up with:

cd video-pipeline && bash scripts/backup-corpus.sh   # then commit and push; the script tells you the command

It pulls JSON only, counts every artefact on both sides, and exits non-zero if the repo came back short, because a backup nobody counted is not a backup. Contact sheets stay on the droplet: about 104 MB of media, regenerable, and media does not come to the laptop.

Run it from a git worktree, not the main checkout. Other sessions work in this tree and switch branches; a backup that gets reverted under you is worse than no backup because it reports success.

git worktree add /tmp/backup-wt corpus-backup

Deleting

Delete a corpus artefact only when it is provably wrong, and print what you deleted and why. The one named case is a wordless script. Everything else stays, including artefacts kept as evidence of a fault.

The rules that produced the current state

A script with no words is worse than no script. The classifier once ran ahead of the transcriber and wrote 306 scripts with on-screen text and no spoken words, which is precisely the half that was lost in the first place. There are now two independent refusals, and both must stay:

• R7-classify-worker.mjs refuses to write script-beats.json when the transcript has any words and the reconstruction carries none, and refuses any video with no transcript at all. The style analysis is still saved, because it is valid and already paid for.
• src/lib/video/index-delivered.ts refuses the same thing again at the ingest boundary, and reports the refusal by name. Two layers, because the first one had a threshold and a 4-word transcript walked through it. See ../knowledge/wordless-scripts-need-two-guards.md.

A genuinely silent video is not caught by either: there the script is its on-screen text and B-roll, and that is worth keeping. 29 of the catalogue are like this.

Transcript accuracy beats speed and cost. A library that spells a client's brand wrong teaches the writer to spell it wrong. Do not swap the ASR model for a faster one; buy machines instead. See ../knowledge/asr-whisper-beats-pegasus-on-brand-words.md and ../knowledge/whisper-hears-singlish-as-malay.md.

Indexing into the portal

npx tsx scripts/one-off/index-back-catalogue.ts <corpusDir> --dry   # read first
npx tsx scripts/one-off/index-back-catalogue.ts <corpusDir>

Idempotent on frameio_file_id. Tables delivered_videos (0111) and video_scripts (0112). It refuses scripts that failed the paraphrase check and says which. Reconstructed scripts land as status = 'draft' and angle_id = null: a record of work already delivered is not approved copy, and claiming an angle it was not written from would be a guess.

.env.local points at the real Supabase dev database, not a local copy. Anything run here reads and writes live dev data.

Checking the work rather than trusting it

node stages/R4-style-classify.mjs check      # every classification against the vocabulary
node stages/R4-style-classify.mjs converge   # are traits still being invented, or has it settled
node stages/R4-style-classify.mjs harvest    # add proposed traits, after reading them

converge is the one that matters, and it must be read on a within-client depth series. A breadth sample of one video per client maximises novelty and proves nothing, so the test now refuses to draw a conclusion from one.