Lesson 3 of 10

Events

"It didn't fire" and "it fired twice" are the two support tickets every webhook system eventually produces. Both trace back to the same handful of delivery guarantees.

How event delivery actually works

Webhooks trade the simplicity of request/response for the complexity of an asynchronous, retried, unordered channel.

Push, not pull

A webhook is the provider calling your endpoint the moment something happens, instead of you repeatedly polling to ask "did anything change?"

At-least-once delivery

Most platforms retry a webhook until they get a fast 2xx response. That guarantees delivery, but means the same event can arrive more than once.

Idempotency

Because delivery can duplicate, your handler must recognize an event it has already processed — usually by tracking the event ID — and safely no-op the second time.

Ordering isn’t guaranteed

Retries and parallel delivery mean events can arrive out of sequence. Reconstruct order from a timestamp or sequence number in the payload, never from arrival order.

Signature verification

A receiving endpoint should verify a signature (typically an HMAC over the raw body) before trusting a payload — otherwise anyone who finds the URL can post fake events.

Exercises

Look at the sanitized delivery log below, then work through both exercises.

POST https://api.customer.com/webhooks/messaging HTTP/1.1
X-Webhook-Signature: t=1710000000,v1=5257a869e7...
X-Event-Id: evt_8f2c1a
Content-Type: application/json

{
  "event": "message.delivered",
  "message_sid": "SM4d2f91",
  "timestamp": "2026-07-20T14:02:11Z"
}

Delivery attempt 1 → customer endpoint timed out after 15s (no response)
Delivery attempt 2 (retry, +30s) → customer endpoint responded 200 OK

Concept check

Answer in your head first, then click a question to reveal the answer.

Scenario checklist

Work through the sample delivery log and check each item off. Progress is saved in your browser.

Confirm the event ID and how many delivery attempts were loggedCheck the response code and timing your endpoint returned on each attemptVerify your endpoint deduplicates by event ID rather than accepting blindlyCheck whether your handler wrongly assumes strict event orderingConfirm the signature header was validated before processing the payload

Go deeper

These primers are a starting point. Read the primary sources to build real fluency.

webhooks.fyiVendor-neutral webhook best practices: delivery, retries, and design webhooks.fyi: SecuritySignature verification and other techniques to trust inbound events MDN: IdempotentWhat idempotency means and which HTTP methods guarantee it MDN: Retry-After headerHow a server tells a client when it’s safe to retry a request

Training Notes

What building this project taught me.

Webhooks turn distributed-systems problems into support tickets

At-least-once delivery, ordering, and idempotency are classic distributed-systems concerns, but as a TSE you meet them as "the customer says they got the event twice" or "it never arrived." Framing the lesson around delivery guarantees rather than one vendor’s webhook format keeps it portable across products.

A third lesson, the same two exercise shapes

Concept check (useState, throwaway) and scenario checklist (localStorage + useSyncExternalStore, persisted) now repeat for a third topic with its own tse-events-lesson-* storage prefix. At this point the shape is proven enough that a shared component would be reasonable — worth revisiting once RCA and AI as a TSE are built too.