Rotate the Sume webhook signing secret without dropping a delivery

Upgrade the receiver so that it accepts a header with two signatures, then rotate with POST /v1/webhooks/signing-secret/rotate (a key with account:write) or the Rotate secret button on the Webhooks page, then deploy the new secret within 24 hours. For that window Sume signs each delivery with both secrets, so the rotation is not a cutover and nothing needs to change at the same instant.

What the window does

During the 24 hours after a rotation, x-sume-webhook-signature carries one sume-v1= entry per live secret, newest first and comma separated. A receiver that holds either secret can verify the delivery. After the window, a check with the old secret fails, and rotation.previous_valid_until on both API responses gives the deadline.

The sequence

Do the steps in order, and keep the secret out of shell history and chat. The first step is the one people skip, and a verifier that compares the header with a single expected string will fail every delivery in the window, since the header no longer equals one signature.

# 1. Upgrade the receiver first: it must accept "sume-v1=<new>,sume-v1=<old>".
# 2. Rotate (needs a key with account:write).
curl -s -X POST https://api.sume.com/v1/webhooks/signing-secret/rotate \
  -H "x-api-key: $SUME_API_KEY"
# 3. Read the new secret (needs account:read) and deploy it to the receiver.
curl -s https://api.sume.com/v1/webhooks/signing-secret \
  -H "x-api-key: $SUME_API_KEY"
# 4. Check the fingerprint header on the next delivery, then wait out the 24 h window.

Why the fingerprint helps

The fingerprint is a 12-character value that appears on each delivery and next to the secret in the dashboard. After you rotate, it names the new secret, which tells you which secret to move to. It does not tell you which secrets Sume still accepts. When a signature does not verify, compare fingerprints before you debug anything else, and when you need to ask for help, paste the fingerprint and never the secret.

A leaked secret is a different case

If you think the secret leaked, one rotation is not enough, because the old secret stays valid for 24 hours. Rotating a second time inside the window retires the secret from two rotations back immediately, and that is how you make a leak stop sooner. The cost is that your receiver must hold the secret from the first rotation until you have deployed the second, so deploy after the second rotation, not between the two. In that situation the usual advice to deploy at your own pace does not apply, because every minute the old secret is accepted is a minute a leaked value could be used to forge a delivery.

Treat the key that performs the rotation with the same care. It needs account:write, and scopes are fixed when a key is created, so an older key without that scope gets 403 insufficient_scope and you need a new key.

Confirm it worked

After the deploy, send a test delivery from the Webhooks page or POST /v1/webhooks/test-deliveries, and check that your receiver verifies it. Then redeliver a real recent job with POST /v1/jobs/{id}/webhook/redeliver to see a real terminal event pass through. A redeliver is signed with a fresh timestamp and does not use one of the ten automatic attempts.

When the window closes, watch for 401 responses from your own receiver. If any appear, an instance is still running with the old secret, and the fingerprint on the failing request tells you which secret Sume used.

Write the date and the new fingerprint into your runbook after each rotation. The next engineer who sees a signature failure can then compare one value instead of guessing which environment still has the old secret, and a quarterly rotation turns into a ten-minute task. If several services receive Sume deliveries, list each one in that runbook, because the secret is per workspace and every receiver needs the new value before the window closes.