Automate faceless short-form videos: AI voiceover, captions, music

To automate faceless short-form videos, chain seven steps and run them from a schedule or a trigger: write a 30–60 second script, turn it into an AI voiceover with text to speech, generate vertical B-roll clips or stills, pick or generate a music bed, assemble everything into one 9:16 video with the music ducked under the voice, burn in captions timed to the voiceover, and deliver the file. You can wire separate tools together with a scheduler, or send the whole brief to one agent call. Keep each video under a minute if you caption it in one pass.

On the Sume API each step is a documented call under one key, or one call to Agent Completions or a Format runs the chain from a brief. The facts below come from the Sume API reference and Sume docs, read on 2026-09-29. Anything called current behavior is read from Sume's code.

What does the automated pipeline look like?

Each paid step is a separate asynchronous job, billed on its own at API pricing rates.

Which tools can run a faceless video pipeline?

It depends on how much of the chain you want one vendor to cover. Model APIs bill each generation and leave the edit and the captions to other tools or your own code; a pipeline API covers the whole chain. Vendor prices below are copied with the unit each vendor's page states.

• Pick Runway if you want its own video models and speech in one credit account and will edit and caption elsewhere. Does Runway have an API? covers its API.
• Pick fal or Replicate if you already run your own assembly code and want to call individual models directly. fal and Replicate alternatives compares the shapes.
• Pick the Sume API if you want voiceover, clips, music, the vertical edit and captions under one key and one wallet.
• Pick a Sume Format or Agent Completions if you want one call per video and accept less control over each step.
• Pick a manual editor if you want to hand-tune each video.

Step 1: Where does the script come from?

From your own code or any text model: the chain only needs text. To have Sume write it, send an Agent Completion with an output_schema, and the run's output comes back in your shape, for example { "script": "…" }. Keep it short enough to caption in one job: step 6 stops at 60 seconds.

Step 2: How do I make the AI voiceover?

Send the script to TTS 1.0 as transcript, up to 20,000 characters, with a voice: the avatar_id or avatar_handle of an avatar whose voice is ready, or a voice.id. Cloning a voice or designing one from a description happens in the Sume app under Voices, not over the API; AI voiceover in your own voice covers it. Text to speech API covers every field; three matter here:

• timestamps: { "words": true } adds words[] with start and end seconds to the completed result. In the current code each entry is { word, start, end }, and the result also carries duration_seconds, the value for the render's audio.duration_seconds.
• language: set it for every non-English script.
• output_format: the default is mp3 at 44,100 Hz. The render inherits its spine's sample rate, and a spine under 32 kHz is reported as audio_spine_low_fidelity.

Step 3: How do I make vertical B-roll?

Generate each shot with POST /v1/videos: a prompt, aspect_ratio: "9:16" for vertical, and a duration the model lists. sume/auto create controls default to 720p and 8 s, with 3–10 s clips at 16:9 or 9:16, so a 24-second voiceover takes three 8-second clips. A pinned model's limits are on GET /v1/videos/models. Poll each job's status until it is terminal, then read the clip's media.sume.com URL from GET /v1/jobs/{id}/result, because Timeline takes only Sume-hosted URLs. A Sume-hosted still in a slot is held as a static frame.

B-roll needs no lip sync: Sume's model docs say video models do not lip-sync to generated TTS. For a speaker on camera, use an Avatar 1.0 talking video or VEED Fabric 1.0 (a talking still plus audio) instead.

Step 4: How do I pick or generate the music bed?

The keyless BGM routes (GET /v1/bgm/catalog, or POST /v1/bgm/pick to score tracks against a mood) list ready tracks; Background music API for video covers them in full. Each track carries a license: sume-original for Sume's own tracks or cc-by-4.0 for the featured ones. In the current code only Sume's own tracks pass as soundtrack.url: the CC BY 4.0 tracks sit at their official URLs off Sume, so Timeline refuses them (unsupported_media_source).

To make a new bed per video, send a prompt to POST /v1/music-router/generate, with sume/music-auto or a pinned id from GET /v1/music-router/models. There is no duration field: steer the length in the prompt, and end with "Instrumental, no vocals." so it sits under a voiceover. Generated outputs are mirrored to Sume media URLs, so the track goes straight into the render.

Step 5: How do I assemble the vertical video?

One Timeline 1.0 render does the edit, and its default output is 1080×1920, so a vertical video needs no output block. audio.url is the voiceover and audio.duration_seconds its length (1–1800). video[] holds the clips: 1–200 slots, the first starting at 0. soundtrack adds the bed at gain_db (default −16 dB, a bed level under the voice), loop repeats it until the voiceover ends, and duck_db (0–20) dips it while the voiceover speaks. The output audio is the voiceover spine with the bed mixed under it; in the current code each clip's own audio is dropped. Check the document first with the unbilled POST /v1/timeline-1.0/plan.

curl -X POST https://api.sume.com/v1/timeline-1.0/render \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: faceless-render-001" \
  -d '{
    "audio": { "url": "https://media.sume.com/artifacts/artf_demo/narration.mp3", "duration_seconds": 24 },
    "video": [
      { "source_url": "https://media.sume.com/artifacts/artf_demo/broll-1.mp4", "start": 0, "duration": 8 },
      { "source_url": "https://media.sume.com/artifacts/artf_demo/broll-2.mp4", "start": 8, "duration": 8,
        "transition": { "type": "fade", "duration": 0.25 } },
      { "source_url": "https://media.sume.com/artifacts/artf_demo/broll-3.mp4", "start": 16, "duration": 8 }
    ],
    "soundtrack": {
      "url": "https://media.sume.com/assets/bgm/minimal-clean-focus.mp3",
      "loop": true,
      "duck_db": 8,
      "fade_out_seconds": 2
    }
  }'

Step 6: How do I caption it from the voiceover timings?

Send the render's video_url to POST /v1/video-captions with words: the TTS words[], each word renamed to text. Sume then skips speech-to-text and burns exactly those words at exactly those times. With the voiceover as the spine from its first second (audio.source_in defaults to 0), the TTS times line up with the render.

• Keep a video you caption to 60 seconds: each words time runs from 0 to 60, and in the current code a longer source fails the caption job, as does a source with no audio stream. For longer videos, split first.
• words takes 1–1,200 entries, each text up to 200 characters, and is mutually exclusive with script_text, cues, and segments.
• Omit style and Latin wording burns in slam, Korean wording in black-outline. For the colour of the emphasised word, design.colors.active overrides it on most styles, but design is not supported on punch or tiktok-green. Customize burned-in caption styles lists the rest.

curl -X POST https://api.sume.com/v1/video-captions \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: faceless-captions-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/faceless.mp4",
    "words": [
      { "text": "Three", "start": 0.12, "end": 0.41 },
      { "text": "ways", "start": 0.41, "end": 0.7 }
    ]
  }'

Step 7: How do I deliver it and run it on a schedule or trigger?

Every paid step is an asynchronous job. Send webhook_url on the submit (POST /v1/videos documents it as callback_url) and Sume POSTs job.completed, job.failed, or job.canceled to your public HTTPS URL, signed with HMAC SHA-256; keep polling status_url as a backup. The caption job's result is the file to post, from your own code: see Automate YouTube Shorts with AI, YouTube Shorts video specs and TikTok video specs.

Because the calls live in your code, your scheduler runs them: a server cron job, a Cloudflare Workers cron, or a trigger such as a new spreadsheet row or a no-code scenario that sends each day's topic. A Sume schedule, authored in the dashboard with a 5-field cron expression in an IANA timezone, instead runs saved instructions as an Agent in a fresh thread; its per-run generation cap is $1.00 when unset. When an event, not a clock, should start a video, the Scheduled docs point to calling a Format from your backend instead.

Can one call make the whole faceless video?

Yes. Sume's basics page describes its agent composing the individual tools (Image, Video, Avatar, TTS, timeline) into a post-ready video with B-roll, voiceover, and timeline assembly. Send the brief (topic, 9:16, length, voiceover, captions, music) to Agent Completions, or save a recipe that works as a Format and call it with each day's topic as input. Video agent API: turn a brief into a finished, edited video covers what that call returns and where it stops.

Where does Sume not fit?

The docs and current code set these limits:

• Caption past 60 seconds in one job, or caption a video with no audio stream (current code).
• Take a file from your disk. Timeline takes only this workspace's media.sume.com URLs, so every input must already be on Sume, such as an earlier Sume job's output.
• Mix the featured CC BY tracks in a render; only Sume's own tracks and Sume-hosted outputs pass as soundtrack.url.
• Synthesize more than 1,200 seconds of audio in one TTS job: that fails with tts_duration_exceeded and captures no credit.
• Post to a platform for you: publishing is your own code.