# Customer-Defined Webhook Receivers Playbook **Portable spec for shipping inbound + outbound webhook plumbing in a new SaaS app.** Covers: HMAC signing, SSRF guards, retry with backoff, contract versioning, idempotency, signature verification, dead-letter queues. Built from the BKE / Kompozy implementation. Reference repo: `reference app`. --- ## 0. Use this spec **Inputs:** - List of third-party providers your app must RECEIVE webhooks from (Stripe, Postmark, GHL, etc.) - List of customer-supplied URLs your app must SEND webhooks to (publish destinations, lifecycle events, etc.) - Whether you have Trigger.dev (or equivalent worker queue) for async side-effects - Per-destination secret storage (vault / row-level jsonb column) **Outputs:** - `/api//webhook` inbound routes with verified auth - Outbound webhook publisher with SSRF + HMAC + retry - `webhook_events` idempotency table - `webhook_failures` dead-letter table - Versioned, frozen payload contract --- ## 1. Two flows, one mental model ``` INBOUND (third party → us) OUTBOUND (us → customer URL) ───────────────────────── ──────────────────────────── 1. proxy.ts allowlist 1. Look up destination row 2. Verify auth (sig / basic / hmac) 2. SSRF guard customer URL 3. Idempotency check 3. Build versioned payload 4. Ack 200 quickly 4. HMAC-sign body 5. Fire-and-forget side effects 5. POST with 30s timeout to worker queue 6. Retry 3x with backoff 7. Dead-letter on final fail ``` --- ## 2. Inbound webhooks ### 2.1 Edge auth allowlist (CRITICAL) Every inbound webhook route MUST be added to `PUBLIC_API_PREFIXES` in `src/proxy.ts`. The default-deny posture for `/api/*` 401s any unlisted route at the edge BEFORE the route handler runs — meaning the route's signature verification logic is unreachable and the third party sees a 401 even though they signed correctly. ```ts const PUBLIC_API_PREFIXES = [ '/api/stripe/webhook', '/api/email/inbound', '/api/email/postmark-events', '/api/', // ← add here before anything else ] ``` ### 2.2 Three auth patterns **Pattern A — Signature verification (Stripe).** The provider signs the raw bytes; you verify with their SDK. Reference: `src/app/api/stripe/webhook/route.ts`. ```ts export async function POST(req: Request) { const sig = req.headers.get('stripe-signature') const secret = process.env.STRIPE_WEBHOOK_SECRET if (!sig || !secret) return NextResponse.json({ error: 'missing' }, { status: 400 }) // CRITICAL: raw body, never req.json() first — hash won't match const raw = await req.text() let event: Stripe.Event try { event = stripe.webhooks.constructEvent(raw, sig, secret) } catch (err) { return NextResponse.json({ error: 'invalid signature' }, { status: 400 }) } // ... process } ``` **Pattern B — HTTP Basic in URL (Postmark).** Provider URL contains `https://user:pass@app.com/api/...`. Route reads `Authorization: Basic` and compares against env. Reference: `src/app/api/email/inbound/route.ts`. ```ts function checkBasicAuth(req: Request): boolean { const expectedUser = process.env.POSTMARK_INBOUND_USER const expectedPass = process.env.POSTMARK_INBOUND_PASS if (!expectedUser || !expectedPass) return false const auth = req.headers.get('authorization') || '' if (!auth.startsWith('Basic ')) return false const decoded = Buffer.from(auth.slice(6), 'base64').toString('utf8') const idx = decoded.indexOf(':') if (idx < 0) return false return decoded.slice(0, idx) === expectedUser && decoded.slice(idx + 1) === expectedPass } ``` **Pattern C — HMAC body verification (custom).** Receiver recomputes `hmac_sha256(secret, body)` and timing-safe-compares to a signature header. ```ts const raw = await req.text() const expected = crypto.createHmac('sha256', secret).update(raw).digest('hex') const got = (req.headers.get('x-app-signature') || '').replace(/^sha256=/, '') if (!crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(got))) { return NextResponse.json({ error: 'invalid signature' }, { status: 401 }) } ``` ### 2.3 Idempotency Every provider gives a unique event id (Stripe `event.id`, Postmark `MessageID`, GHL `eventId`). UPSERT into `webhook_events` with `(provider, event_id)` unique constraint. If already processed, return 200 immediately. ```ts const { data: already } = await admin .from('webhook_events') .select('event_id') .eq('provider', 'stripe') .eq('event_id', event.id) .maybeSingle() if (already) return NextResponse.json({ received: true, duplicate: true }) ``` See the Stripe handler's `stripe_webhook_events` pattern in `src/app/api/stripe/webhook/route.ts` — row is written at the END of a successful run so retries that race mid-handler re-enter and finish the same idempotent work. ### 2.4 Sync vs async Stripe retries any webhook that takes >10s. Postmark retries on any non-2xx. Long-running side effects (send confirmation email, run AI pipeline, transcode media) should fire-and-forget to a Trigger.dev worker and return 200 immediately. ```ts // ❌ blocks the webhook response, racing the 10s ceiling await sendWelcomeEmail(userId) await enrollInDripCampaign(userId) return NextResponse.json({ received: true }) // ✅ ack first, defer heavy work await tasks.trigger('post-stripe-checkout', { userId }) return NextResponse.json({ received: true }) ``` ### 2.5 First-event-wins for downstream stamps When the same logical event can arrive multiple times (Postmark Open/Click events deduped on `MessageID`), use first-event-wins semantics on the destination columns — `.is(column, null)` guard on the UPDATE so a duplicate doesn't clobber the original timestamp. See `src/app/api/email/postmark-events/route.ts`. --- ## 3. Outbound webhooks ### 3.1 Schema ```sql -- Destinations: per-tenant, per-provider config create table outbound_destinations ( id uuid primary key default gen_random_uuid(), workspace_id uuid not null references workspaces(id) on delete cascade, provider text not null, -- 'webhook', 'slack', etc. config jsonb not null, -- { endpointUrl, hmacSecret } created_at timestamptz default now() ); -- Idempotency for INBOUND create table webhook_events ( provider text not null, event_id text not null, received_at timestamptz default now(), payload jsonb, processed boolean default false, primary key (provider, event_id) ); -- Dead-letter for OUTBOUND create table webhook_failures ( id uuid primary key default gen_random_uuid(), destination_id uuid references outbound_destinations, payload jsonb, last_attempt_at timestamptz, attempt_count int default 0, last_error text, created_at timestamptz default now() ); ``` ### 3.2 SSRF guard (REQUIRED) Without this, a malicious tenant submits an HTTPS URL pointing at internal infra (AWS IMDS via TLS proxy, Supabase project URL, internal cluster service) and exfiltrates secrets. HTTPS-only does NOT block any of those. Reference: `src/lib/publish/blog/webhook-ssrf.ts`. Two-pass defense: 1. **String-level guard** — reject loopback / `.internal` / `.local` / `.lan` / `.home` / IP-literal-in-private-range URLs without a DNS hop. 2. **DNS-resolve and reject any private-range answer** — catches DNS rebinding where a public hostname resolves to RFC1918. Block list: - IPv4: `10/8`, `127/8`, `172.16/12`, `192.168/16`, `169.254/16` (link-local + IMDS), `100.64/10` (CGNAT), `0/8`, `224+/8` (multicast/reserved) - IPv6: `::1`, `fc00::/7` (ULA), `fe80::/10` (link-local), IPv4-mapped → fall back to IPv4 check - Hostnames: `localhost`, `metadata.google.internal`, anything ending in `.internal` / `.local` / `.lan` / `.home` / `.localhost` Re-run the guard at PUBLISH time, not just at SAVE time — defends against rows modified via direct SQL after passing test. ### 3.3 HTTPS-only ```ts if (!/^https:\/\//i.test(endpointUrl)) { return { ok: false, error: 'http:// rejected — would leak HMAC secret in transit' } } ``` ### 3.4 Versioned payload contract Freeze the schema as `version: "1"`. Bump version + handle both for any breaking change. Reference: `src/lib/publish/blog/publish-webhook.ts`. ```ts const payload = { event: 'blog.publish', version: '1', timestamp: new Date().toISOString(), workspace: { id, name }, post: { id, title, slug, markdown, html, ... }, } ``` ### 3.5 Headers ``` Content-Type: application/json User-Agent: /1.0 X--Event: blog.publish X--Version: 1 X--Workspace: ← lets receivers route by tenant X--Timestamp: ← part of HMAC input, enables freshness window X--Delivery: ← idempotency key for receiver, stable across retries X--Signature: sha256= ``` ### 3.6 HMAC signing Sign `${timestamp}.${body}`, not just body — prevents replay of an old (timestamp, body, sig) tuple with a fresh timestamp. ```ts const tsUnix = String(Math.floor(Date.now() / 1000)) const sig = crypto.createHmac('sha256', hmacSecret) .update(`${tsUnix}.${bodyText}`, 'utf-8') .digest('hex') headers['X-APP-Signature'] = `sha256=${sig}` headers['X-APP-Timestamp'] = tsUnix ``` Receivers verify by recomputing over `${X-APP-Timestamp}.${raw body}` with `crypto.timingSafeEqual` and enforcing a freshness window (e.g. reject sigs > 5 min old). ### 3.7 Timeout + retry - 30s `AbortController` timeout per attempt - Retry 3x with exponential backoff: 1s, 5s, 30s - Dead-letter to `webhook_failures` after final failure - Receivers should ack with 2xx promptly; long-running ingestion belongs in THEIR queue ```ts const controller = new AbortController() const timer = setTimeout(() => controller.abort(), 30_000) try { res = await fetch(endpointUrl, { method: 'POST', headers, body, signal: controller.signal }) } finally { clearTimeout(timer) } ``` ### 3.8 Test-fire at save time Every saved destination gets a synchronous POST with `{ test: true }` body so users see immediate feedback ("✅ webhook reachable" / "❌ SSRF rejected"). Run the SSRF guard on test too so a malicious URL never gets persisted. --- ## 4. Common pitfalls - ❌ **Don't parse the body before signature verification.** Raw bytes are the HMAC input. `req.json()` first → hash mismatch → 100% rejection rate. - ❌ **Don't skip SSRF guards.** Customer-supplied URLs can target internal services. HTTPS-only does not save you. - ❌ **Don't process webhooks synchronously for long side-effects.** Defer to Trigger.dev or equivalent. Stripe retries at >10s; Postmark retries at non-2xx. - ❌ **Don't reuse a single global HMAC secret.** One secret per destination, stored in `config.hmacSecret` jsonb. A leaked secret revokes one tenant, not all. - ❌ **Don't return 5xx for permanent failures.** Providers retry on 5xx. If the failure is permanent (malformed event, deleted user), return 200 and log internally. - ❌ **Don't trust the User-Agent of incoming webhooks.** Anyone can spoof it. Auth via signature only. - ❌ **Don't expose webhook event ids back to the user via API.** Replay-attack vector. - ❌ **Don't forget `PUBLIC_API_PREFIXES`.** New inbound routes 401 at edge before the handler runs. The third party sees auth failures with no log entry on your side. - ❌ **Don't write destructive sync on incoming webhooks.** A Stripe `subscription.updated` event should UPSERT, never DELETE-then-INSERT — the 2026-04-23 destructive-sync incident wiped user data via exactly this pattern. --- ## 5. Kickoff prompt > Execute the Webhook Receivers Playbook at `/templates/b-tier/WEBHOOK-RECEIVER-PLAYBOOK.md` for this new app. Mirror the BKE implementation. Reference codebase at `/src/` — when in doubt about a pattern, read the BKE file rather than re-deriving. > > Inputs: > - Inbound providers: [Stripe / Postmark / GHL / etc.] > - Outbound destinations needed: [list event types — `blog.publish`, `lead.created`, etc.] > - Worker queue: `Trigger.dev / equivalent / none — affects sync vs async) > > Pause after proxy.ts allowlist + first inbound route ships for review. --- **Reference files in BKE:** - [src/lib/publish/blog/publish-webhook.ts` — outbound publisher with HMAC + SSRF + timeout - `src/lib/publish/blog/webhook-ssrf.ts` — two-pass SSRF guard - `src/app/api/stripe/webhook/route.ts` — Pattern A inbound (signature verification) - `src/app/api/email/inbound/route.ts` — Pattern B inbound (HTTP Basic + threading + idempotency) - `src/app/api/email/postmark-events/route.ts` — Pattern B inbound (event stamps, first-event-wins) - `src/proxy.ts` — `PUBLIC_API_PREFIXES` allowlist