How to test a webhook URL before a Sume Format run uses it

Call POST /v1/webhooks/test-deliveries with the URL you plan to put on communication.webhook_url. Sume sends one signed dummy webhook.test event to it and returns the status code your server answered, any error, and a fingerprint of the signing secret it used. It needs a key with account:write, and it never starts a job or a Format run, so a test costs nothing in generation.

Do this before the first real run, and again whenever a receiver moves, because a wrong URL or secret otherwise shows up hours later as a webhook that never arrived. It is especially worth doing for unattended setups, such as a scheduled agent whose prompt carries the URL, where nobody is watching the first delivery.

What does the call send and return?

The request body has one field, webhook_url. Sume's docs say localhost, private-network and non-HTTPS URLs are rejected, so a laptop receiver needs a public HTTPS tunnel. The response describes the attempt, not a job.

What is the call?

The script fails loudly when the receiver does not answer with a 2xx, so you can use it as a deploy check.

import os, sys, requests

r = requests.post(
    "https://api.sume.com/v1/webhooks/test-deliveries",
    json={"webhook_url": sys.argv[1]},
    headers={"Authorization": "Bearer " + os.environ["SUME_API_KEY"]},
    timeout=30,
)
r.raise_for_status()
d = r.json()["data"]
print("status:", d["status_code"], "error:", d["error"])
print("secret fingerprint:", d["signing_secret_fingerprint"])
sys.exit(0 if d["status_code"] and 200 <= d["status_code"] < 300 else 1)

What should I check in the result?

Three things, in this order.

• A 2xx status_code means the URL is reachable and your server answered inside Sume's reply window. A null status with an error means Sume could not connect.
• Compare signing_secret_fingerprint with the secret your receiver has loaded. The same fingerprint travels in the x-sume-webhook-secret-fingerprint header on webhook deliveries, so a receiver can log which secret verified the call. Reading the secret itself with GET /v1/webhooks/signing-secret needs account:read.
• Confirm the receiver verified the signature and did not skip it. A receiver that returns 200 without checking x-sume-webhook-signature passes this test and fails in production, so also send a request with a bad signature yourself and expect a 401.

How should the receiver treat webhook.test?

Branch on the event name before any database write. Real Format deliveries are format.run.terminal, and they carry a run id you use as the dedupe key. A webhook.test has no run id, so answer 200 and stop; if your handler tries to insert a row keyed by a missing id, the test fails for the wrong reason.

Keep this separate from redelivery. A test sends a dummy payload to any URL you name. Replaying a real finished run uses POST /v1/format-runs/{run_id}/webhook/redeliver, which only works for a terminal run that had a webhook registered, and it sends the real event with a fresh timestamp and signature.

Does a passing test guarantee delivery?

No. It proves the URL, the secret and a fast 2xx at one moment. Real deliveries are retried up to ten times with backoff when your server is slow or down, deliver only on completed or failed terminal states, and never on cancel or skip. A run that you cancel or that is skipped by on_active_run produces no webhook at all, so keep the receipt URL as a fallback for those.