Find the product moment in a Short: a contact sheet with video-frames

To see when a product is on screen in a Short, ask Sume for frames: POST /v1/video-frames takes an at list of 1 to 24 timestamps and returns one image for each, as JPEG or PNG. Set max_edge between 16 and 2160 to keep the images small, and the source video may be up to 300 seconds. You read the frames, then use the seconds you find as cut points.

The YouTube behaviour behind this is in Tag products in your content (read 2026-10-03), and the Sume side is the video frames doc.

Why look at frames first?

On Shorts, YouTube's help page says a Shopping sticker shows the first product in the tagged list, and that timestamps are not available on Shorts. So nothing times the tag to the product's moment, and the product should be clear on screen long enough for a viewer who taps to see what it is. A contact sheet shows that in seconds instead of scrubbing.

How do I do it?

Pick evenly spaced timestamps, for a 30-second Short every 1.25 seconds gives 24 frames. Frame extraction is not a flat price: the docs say it is billed by its Modal compute, so check the job's usage rather than assuming a number.

import os, requests

API = "https://api.sume.com"
H = {"Authorization": f"Bearer {os.environ['SUME_API_KEY']}", "Idempotency-Key": "frames-short-001"}
body = {
    "video_url": "https://media.sume.com/artifacts/artf_demo/short.mp4",
    "at": [round(i * 1.25, 2) for i in range(24)],
    "format": "jpeg",
    "max_edge": 480,
}

def main():
    r = requests.post(f"{API}/v1/video-frames", headers=H, json=body)
    r.raise_for_status()
    print(r.status_code, r.json())

main()

What do I do with the answer?

If the product is only visible from second 18 to 24, cut that span with video trim ($0.02 a job, start plus one of end or duration) and move it earlier, or open the Short on it. Then make sure the product is on screen for that stretch, because Shorts tags are not timestamped.

What Sume does not do

Sume returns images; it does not tell you which frame shows the product. You or a vision model of your choice decide that. It also does not create or attach YouTube product tags.