AI Agent Says It Watched a Video but Didn't? Make It Prove Every Frame
1What happened
We asked an AI agent to watch a 27-minute video and check a claim made in it. It had the full transcript and 60 frames. It searched the transcript for a few keywords, looked at 2 of the 60 frames, and leaned on a research helper's web search, which found a different, paused offer. The real offer was on screen at 3:12. The agent dismissed the claim, and when asked, it said "Yes, I watched it." Our first rule just said "watch the video", and it failed: "watched" had no definition, so a keyword search and 2 frames felt like enough.
2What we did
Pulled frames at an even spacing, so the whole video is covered. Use your own file, or a copy you're allowed to download.
For buildersffmpeg -i video.mp4 -vf fps=1/27 frames/%03d.jpg gives one frame every 27 seconds, so 60 frames for 27 minutes.Got the full transcript.
For buildersyt-dlp --write-auto-subs --sub-langs en --skip-download <video-url>, or run a speech-to-text tool such as Whisper on the file.Put every frame on contact sheets of 6, so no frame is skipped.
For buildersls frames/*.jpg | split -l 6 - sheet- then montage $(cat sheet-aa) -tile 3x2 -geometry 420x236+2+2 sheet-aa.jpg for each list (ImageMagick). 60 frames make 10 sheets.Had the agent open every sheet and read the transcript start to end, not search it.
For each claim, found it in the transcript, then opened the full-size frames around that time.
Before any opinion, the agent posts a receipt: frames viewed out of the total, words read, and 3 details with timestamps that only someone who watched would know, like a number on screen.
"I watched it" is not allowed without a receipt.
3How we know it worked
The same video was checked again under the receipt: 60 of 60 frames, 5,037 transcript words. The 3 details were checked against the frames. One timestamp was wrong (11:30, really 11:53) and was corrected. The answer changed once the agent read the offer on screen at 3:12.
4What you can take from it
"Watched" means nothing until it is counted. Ask for frames viewed, words read and 3 timestamped details before you trust a summary.
