Vercel AI Gateway's new tools and models, and where Sume media fits

Route the planning model through a gateway if it helps you, and call Sume directly for the paid media jobs, because the two have different failure modes. The Vercel changelog lists AI Gateway additions for Browserbase Search and Fetch tools, Microsoft audio models and a model called Laya that is free through Oct 31, plus Sandbox support for Secure Compute.

This post does not compare Sume with any of those models. It draws a line between the text layer an agent uses to decide and the job layer it uses to make media.

What the changelog lists

Entries from the Sep 30 to Oct 1 window.

Two layers

A planner answers in seconds and its calls are cheap to retry. A Sume generation job is a durable record: a valid submit returns a job id, the job moves through queued, processing and then a terminal state, and the estimated cost is reserved when it is accepted. Retrying the wrong layer in the wrong way is how duplicate charges happen.

Keep the boundary in code

Give the planner a single tool, such as make_clip, that your own server implements. Inside it, call Sume with an Idempotency-Key derived from the task, return the job id, and let a separate step collect the result. The gateway never sees the Sume key, and a retry of the planner's tool call lands on the same Sume job.

async function makeClip(taskId: string, prompt: string) {
  const res = await fetch("https://api.sume.com/v1/video-router/generate", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.SUME_API_KEY}`,
      "Content-Type": "application/json",
      "Idempotency-Key": `clip-${taskId}`,
    },
    body: JSON.stringify({ model: "seedance-2.5", prompt, duration: 5, mode: "async" }),
  });
  return res.json(); // keep the job id, status_url and result_url
}

Waiting without holding the planner

Do not make the planner's turn wait on a render. Sume's sync and subscribe modes are the same bounded wait, capped at 30 seconds, and they are the wrong tool for most video work. Submit with async, store the job id, and let a webhook or a poll loop finish the job.

Where a sandbox fits

If the agent's code runs in a sandbox, treat it as an untrusted client for key purposes. Sume's docs say browser and mobile clients should call your backend, which attaches the API key. Keep the key on the server that implements make_clip, not in the sandbox environment.

Webhook receivers are a separate server concern. Verify the sume-v1 signature on the raw body, store the event keyed by job_id, and answer 2xx before doing more work.

Audio and search tools

The gateway's audio and browsing tools are a separate vendor surface that this post does not evaluate. If you mix them with Sume output, keep each tool's results in its own field, and pass only Sume media URLs, not provider-internal ones, to the next step. Sume results are served from its own media host, and raw provider URLs are not public API outputs.