TTS word timings to burned-in captions: send them as words on Sume
Sume's TTS can return word start and end times; the caption job accepts words with text, start and end and skips transcription. How to wire them together.
You can caption a video made from a Sume voiceover without transcribing it again. Request the voiceover with timestamps: { words: true }, read the words[] timings off the finished job, and send them to the caption job as words, each with text, start and end in seconds. Sume's caption guide says words, cues and segments skip speech-to-text and burn exactly that copy at those times.
The two sides of the hand-off
On the TTS side, the schema says words: true returns monotonic words[] with start and end seconds on the job result. On the caption side, Video captions says authored timing skips transcription. Check the key names in your own job result before you map them, since the caption job wants text, start and end.
| Step | Field | Notes |
|---|---|---|
| TTS request | timestamps.words true | Returns words[] with start and end seconds |
| Caption request | words[] | text, start, end; end must be greater than start |
| Caption limits | Up to 1,200 words | Videos up to 60 seconds |
| Alternative | cues or segments | Phrase-level cards, same skip of transcription |
| Do not combine | script_text, words, cues, segments | Mutually exclusive |
Why bother
Captions made from your own timings carry the exact script wording, so a brand name that a transcriber would misspell stays correct, and you avoid paying a speech-to-text pass. You still pay for the render: a standalone caption job is $0.20 for videos up to 60 seconds under the current fixed estimate.
Timings measure the voiceover alone. If you add a lead-in before the voice starts in the final video, add that offset to every start and end.
Sources
Related posts
More in Developers
- Twilio MMS image limit is 5 MB: compress a Sume image to fit
Twilio's MMS media limit is 5 MB, with image/jpeg, png, gif, heic, heif, tiff and bmp accepted. A 4K PNG may blow past it; recompress to JPEG before sending.
- Two workers, one order: Idempotency-Key from order id and version
Two queue workers pick up the same order and both submit to Sume. Build the key from order id plus version so duplicates collapse and edits still create a run.
- Undo for AI image edits: keep a version chain of every saved result
Generative edits have no undo button. Download each result, hash it, record its parent and prompt, and walk the chain back. Python, no database.
- Unit test a Sume 429 retry loop in Python with a fake opener
Test a Sume rate_limited retry loop with unittest and a fake opener: retry-after is honored, the last try raises, no network, no spend. Stdlib only.
Written by Sume