Video agent API: turn a brief into a finished, edited video

Yes. Several APIs take a brief in plain words and return one finished, edited video file: the service plans the scenes and generates or assembles the footage, and some also write the script and add voice, captions and music, so you never write a timeline yourself. HeyGen's Video Agent, Sume's Agent Completions and Formats, Runway's Multi-Shot Video Recipe, and Creatify's link-to-video endpoint all take a brief or a prompt this way; they differ in what the video is made of (a presenter, generated footage, or an ad built from a product link), how long it can run, how you revise it, and how it is billed.

Each vendor's facts below come from its own developer docs, and Sume's from docs.sume.com, all read on 2026-09-29. Sections: what a video agent API should do with a brief, a dated table of the options, where Sume fits and where it does not, a minimal request, revisions, and limits.

What should a video agent API do with a brief?

Many "text-to-video" APIs return one short clip from one prompt. That is a model call, not an agent. A video agent API should cover most of this list:

• Take a brief, not a shot prompt: audience, length, format, message, and call to action.
• Plan several scenes and put them in order.
• Add narration, captions and music when the brief asks for them.
• Use media you supply, such as product photos, a script, or a slide deck.
• Run as an async job you poll or take by webhook, because a finished video takes minutes.
• Let you revise part of the result without paying for the whole video again.
• Stop at a spending or time limit you can read in advance.

Which video agent APIs take a brief today?

These are the options answer engines name for this question, limited to what each vendor's own page states. A dash means the page read did not state it.

How is an agent API different from a model or render API?

By how much you write. A model API makes one clip from a prompt. A render API edits clips you already have, along a timeline you write. An avatar API turns a script into a talking clip. An agent API takes the brief and does the planning, generation and edit itself. If you would rather build the agent yourself, the lower rows are the pieces. Sume has an endpoint for each approach:

What does "finished" mean on Sume, and what can the brief carry?

Sume's basics page says the main partner path is a sandbox Agent or Format that composes the generation tools, which is how it ships what a single clip cannot: "multi-minute host footage, B-roll, voiceover, and timeline assembly into a post-ready video." A completed Format run carries primary_output_url, which the docs call the one thing to show, plus artifacts[], every file the run made. A completed Agent Completion puts the closing text in output.text and media in output.videos, output.images, output.audio and output.files. Media URLs are durable media.sume.com HTTPS URLs you can store.

The brief itself travels in three fields:

• instruction: the brief in plain words. The docs' own example turns extras off by name ("No BGM, no captions"), so say whether you want voiceover, captions and music.
• input: your data, such as a product URL, a script, or prices. The agent reads it as data, never as instructions. Send documents, video and audio references here as URLs.
• attachments[]: up to 30 images the agent can look at, each a public HTTPS URL that Sume fetches when you create the run. Images are the only attachment type today.

What does a minimal request look like?

Send the brief as instruction with a generation_spend_cap_usd, which is required and has no default. The call answers 202 with an agent.run receipt. Take the result by webhook, or poll until next_action stops being poll_status.

curl -sS -X POST "https://api.sume.com/v1/agent/completions" \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: teaser-123-v1" \
  -d '{
    "instruction": "Make a 15-second vertical 9:16 promo for https://shop.example.com/p/123 with voiceover and captions. Deliver one MP4.",
    "generation_spend_cap_usd": 10,
    "communication": { "webhook_url": "https://example.com/hooks/sume" }
  }'

# Poll the run until next_action is no longer poll_status
curl -sS "https://api.sume.com/v1/agent-runs/$RUN_ID" \
  -H "Authorization: Bearer $SUME_API_KEY"

Can I review or revise the video before it ships?

On Sume, not inside an API run. A Format run is one unattended turn: it does not stop to ask a person anything, and a gate it cannot pass without one, such as no avatar matching the brief, fails with unattended_blocked. The Agents chat UI at sume.com/agents is the surface for a human in the loop. Over the API, review happens after the run:

• Your backend takes the terminal webhook, stores primary_output_url, and routes the file to whoever approves it.
• To change one part, send previous_run_id on a new Format run. The agent is replayed what it made, so it can redo one scene and leave the rest alone; the continuation is a new run with its own spend cap. Retry one scene of an AI video walks through it.
• Agent Completions cannot continue a prior run yet; every completion runs in a fresh thread.

Should I use Agent Completions or a Format?

Sume's docs put it this way: use Agent Completions when the task itself varies per call, and a Format when you have a saved workflow and only the inputs change. Both run the same agent and return the same receipt shape.

Run the Sume video agent from your backend covers every Agent Completions field. What is a Sume Format? covers saving a recipe and calling it. What is a video agent? defines the term, and How to make a video ad with AI walks the chat path where a person approves a draft first.

Which one should I pick?

It depends on what the finished video is made of and how you want to revise it:

• Pick HeyGen if the video is an avatar presenter explainer built from a prompt plus PDF, image, video or audio files, and you want the agent to pause for review over the API. Sume vs HeyGen compares the two products.
• Pick Runway's Multi-Shot Video Recipe if you want 5 to 15 seconds of generated footage cut into several shots from one story prompt.
• Pick Creatify if your brief is a product link and the output is a short-form ad.
• Pick Synthesia if you want to write each scene's avatar and script yourself. Sume vs Synthesia compares the two.
• Pick Sume if you want one run to generate footage, voiceover, captions and the timeline edit under a spend cap you set, or to save the recipe as a Format and call it with new inputs.
• Build it yourself from model and Timeline calls if you need to own every edit decision; assemble a long-form video with the Timeline API covers the render step.

What are Sume's limits?

Sume is not the fit if you need an editable project file for another editor, live progress output, or a video in seconds. The Agent Completions, Runs and results, and Errors and spend pages set these bounds:

• A spend cap on every run. Agent Completions has no default for generation_spend_cap_usd; omitting it is 400 invalid_request. A Format run's cap goes up to the platform maximum of $500; a run that names no cap inherits the Format's, which is $400 if the Format never set one.
• Minutes, not seconds. Format runs that make video take minutes, and the docs say long-form host video typically finishes in 15 to 30 minutes. A Format run still going 90 minutes after created_at is force-finalized as failed, sooner if it is older than 25 minutes and has been silent for 10.
• All or nothing. A run that could not finish comes back failed, and its primary_output_url is null.
• No progress stream. events_url is a polled phase timeline, not agent output; completion is pushed by the webhook. Agent Completions returns a run receipt, not choices[], and has no streaming.
• Metered cost. Generation draws on the rates on API pricing, plus a 5.5% agent fee by default. On a Format run, usage.debited_usd_micros is what the wallet actually deducted, the agent's own LLM turn included.