Skip to main content
A http drain POSTs batches to any HTTPS endpoint you control. It is the only destination that is signed, and the only one that proves you own the endpoint by making your receiver echo a token back. Use it for a receiver of your own, for a queue in front of a warehouse, or for a vendor that takes arbitrary JSON. Before you begin
  • An HTTPS endpoint that is publicly resolvable. Private, loopback, link-local, and cloud-metadata addresses are refused when the drain is created; plain http:// is refused too.
  • A credential with logs:manage on the organization, or an organization admin role.

What to enter

Output
The secret is not stored anywhere you can read it again. Lose it and you rotate it: see Verify signatures.

The request contract

Every delivery is a single POST. Your receiver must:
  • Answer 2xx within 10 seconds. The request is abandoned at 10 seconds and the batch is retried.
  • Deduplicate on records[].id. Delivery is at-least-once; the same record can arrive twice, in different batches, under different batch ids.
  • Not redirect. A 3xx is treated as a failure and is never followed. Publish the final URL.
  • Return 429 or 503 with a Retry-After header to ask for a pause. The value is honoured, clamped at one hour.
408, 429, 500, 502, 503, and 504 are retried. Every other 4xx is terminal: the batch is dropped and counted, because a 400 means your receiver will reject it again on the next attempt too.

Caps

How ownership verification works

A new drain starts in pending_verification and buffers rather than delivers. Verification proves you control the endpoint before any real log data goes to it.
1

Ask for a verification batch

The platform mints a single-use token, valid for 10 minutes, and sends one batch containing one lua.drain.test record with the token in X-Lua-Verify.
2

Echo the token

Your receiver returns 2xx and copies the value straight back as its own X-Lua-Verify response header.
If the response has no such header, one follow-up GET {origin}/.well-known/lua-drain-verify is made and a body whose trimmed content equals the token is accepted instead. Use that when a proxy strips unknown response headers.
3

Confirm

On success the drain moves to healthy, records verifiedAt, and the buffered backlog starts draining — anything queued in the last six hours still gets delivered. On failure the drain stays in pending_verification and verification.outcome says which check failed: no_2xx, token_not_echoed, ssrf_refused, timeout, or error.
Verification is limited to 5 attempts per drain per hour. Changing the endpoint or the type later sends the drain back to pending_verification, and you verify the new endpoint the same way.

A receiver

A complete receiver: it verifies the signature over the uncompressed body, tolerates a rotation window, echoes the verification token, deduplicates on record id, and answers well inside 10 seconds.
receiver.mjs
Do the slow part — writing to a warehouse, fanning out to a queue — after you answer, or behind a bounded buffer. A receiver that takes longer than 10 seconds is retried, which makes it slower still.

NDJSON

With --format ndjson the body is one JSON object per line and Content-Type is application/x-ndjson. Each line repeats schemaUrl, batchId, and resource so a line is self-contained wherever it ends up:
The signature still covers the whole uncompressed body, all lines together — not line by line.

Test it with curl

Replay a realistic batch at your receiver without waiting for traffic. Substitute your endpoint and the secret you saved.
A receiver that is ready answers 204 and logs one line. Add -H "X-Lua-Verify: probe-token" and the response should carry X-Lua-Verify: probe-token back. Once the drain exists, the same round trip through the real delivery path is one command:
Output
It exits 1 on anything other than a 2xx, so it works as a smoke test in CI.

If it isn’t working

lua drains status <id> --json carries verification.outcome. token_not_echoed means the request reached you and returned 2xx but neither the response header nor /.well-known/lua-drain-verify carried the token — most often a proxy stripping unknown response headers. ssrf_refused means the endpoint resolved to a private or link-local address. no_2xx carries the status code your endpoint actually returned.
You are almost certainly signing over the compressed bytes. The signature covers the body after gunzip. If you use a framework that transparently inflates request bodies, turn that off and gunzip explicitly — otherwise you cannot know which bytes you have.
Expected, and not a bug: delivery is at-least-once. Deduplicate on records[].id and keep the key for at least six hours, the retry horizon. Deduplicating on X-Lua-Batch-Id does not work — a retried record is re-sent under a new batch id with whatever else is due at that moment.
lua drains status prints BACKLOG. A drain that is up but slow builds a queue; a drain that is down buffers for six hours and then starts dropping the oldest records, and tells you it did with a lua.drain.dropped record on recovery. See Delivery guarantees.

Next steps

Verify signatures

Verifiers in Node, Python, and Go, and how rotation works.

Event schema

Every field your receiver will see.

Delivery guarantees

Retry, drops, heartbeat, and health states.

lua drains

Every verb and flag.