Extend an AI clip by hand: last frame to the next clip, in Python

To continue a clip on Sume, take its last frame and start the next job from it. Sume's POST /v1/video-frames returns frame images at timestamps you choose, and POST /v1/videos accepts a first_frame in frame_images on models that list it. Expect some drift in faces and lighting from clip to clip; a longer single generation avoids that when a model allows it.

Steps

Check that your chosen model lists first_frame in supported_frame_images on GET /v1/videos/models.

• Take the completed clip's media.sume.com artifact URL from GET /v1/jobs/{id}/result. The frames endpoint needs a workspace artifact, not the api.sume.com content link.
• POST /v1/video-frames with at set to a time just before the end (duration minus 0.1). Each at value must be at least 0 and less than the duration.
• Poll GET /v1/video-frames/{id} until resource_status is ready, then read frames[0].url.
• Submit the next /v1/videos job with that URL as a first_frame in frame_images.

Code

import asyncio, os, httpx

API = "https://api.sume.com"
H = {"Authorization": f"Bearer {os.environ['SUME_API_KEY']}"}

async def last_frame(c, url, duration):
    r = await c.post(f"{API}/v1/video-frames", headers=H,
                     json={"video_url": url, "at": [duration - 0.1]})
    r.raise_for_status()
    fid = r.json()["id"]
    while True:
        s = (await c.get(f"{API}/v1/video-frames/{fid}", headers=H)).json()
        if s.get("resource_status") == "ready":
            return s["frames"][0]["url"]
        await asyncio.sleep(3)

async def main():
    async with httpx.AsyncClient(timeout=60) as c:
        frame = await last_frame(c, os.environ["CLIP_URL"], float(os.environ["CLIP_SECONDS"]))
        body = {"model": os.environ["VIDEO_MODEL"], "prompt": os.environ["PROMPT"],
                "frame_images": [{"type": "image_url", "image_url": {"url": frame},
                                  "frame_type": "first_frame"}]}
        r = await c.post(f"{API}/v1/videos", headers=H, json=body)
        print(r.status_code, r.json().get("id"))

asyncio.run(main())

Limits to know