datadog drain posts to the Datadog Logs HTTP intake for your site. Records arrive in Log Explorer with service set to the agent’s name, status set to the record’s severity, and env, agent, org, and source as tags — so the standard Datadog facets work with no pipeline of your own.
Before you begin
- A Datadog API key (not an application key) with log-write access.
- The site your Datadog organization is on. It is the domain in your Datadog URL.
What to enter
--endpoint is not accepted for this type. The intake URL is derived from the site:
Only those hosts are allowed. Anything else is refused when the drain is created.
Output
Datadog deliveries are not signed. There is no
X-Lua-Signature header and no signing secret; the DD-API-KEY header authenticates the sender. If you need a signature, use a generic HTTPS drain.How ownership verification works
Datadog uses a test post: one batch containing a singlelua.drain.test record is sent to the intake, and any 2xx is accepted.
403 from the intake means the API key is wrong or is not valid for that site — an API key is site-specific, and a US key against datadoghq.eu fails exactly this way. The drain stays in pending_verification with verification.outcome: "no_2xx" and statusCode: 403, and records keep buffering (for up to six hours) while you fix it. Verification is limited to 5 attempts per drain per hour.
What arrives
Each record becomes one entry in a JSON array:
The complete record under
lua is what makes the drain useful in Datadog: every field is addressable as an attribute. @lua.eventName, @lua.attributes.gen_ai.tool.name, @lua.attributes.lua.primitive.name, and @lua.attributes.lua.execution.id all work in Log Explorer and in monitor queries.
Useful queries
@lua.eventName, @lua.attributes.lua.source, and @lua.attributes.gen_ai.tool.name the first time you use them; the rest (service, status, env, host) are standard Datadog facets already.
Caps
A single record over 1 MiB has its
body cut and suffixed …[truncated for destination] rather than being dropped — you still get the event, its severity, and all of its attributes.
Test the key with curl
Prove the key and the site before creating the drain. This sends exactly the shape a delivery has:202 {} means accepted. Search source:lua in Log Explorer; it should appear within a few seconds. 403 {"status":"error","code":403,...} means the key is wrong for that site.
Then do the same through the real delivery path:
Alert on it
The Datadog monitor pack is four importable monitors built on these field names: the drain’s heartbeat going missing, an error-rate spike per agent, records being dropped, and the runtime truncating an execution’s output.Your first alert, end to end
Ten minutes of work, then one drill you watch. The steps link to the page that has the detail; nothing here is new, it is the order to do it in.1
Prove the API key — 1 minute
Run the curl above against your site.
202 {} means the key and the site agree. Doing this first turns the most common failure — a US key against datadoghq.eu — into a one-line answer instead of a drain stuck in pending_verification.2
Create the drain — 2 minutes
3
Verify — 1 minute
2xx from the intake moves the drain to healthy and releases everything buffered since it was created. Verification is limited to 5 attempts per drain per hour, so read the error rather than retrying in a loop.4
See a record — 2 minutes
source:lua in Log Explorer. If your agents are quiet, send one message to an agent in the drain’s environment — that produces execution and, when a tool runs, skill records within seconds.lua drains deliveries drn_2b7f10d9ac4e83615702ffab --kind batch --json shows the first delivery of real records, as opposed to the synthetic test and verify ones.5
Import the heartbeat monitor — 3 minutes
Take [Lua] Log drain heartbeat missing from the monitor pack and import it first. The other three alert on something being present, so their silence only means anything while the pipeline is known to be up.Add a notification handle to its
message before saving. The pack ships with none — an imported monitor alerts in the UI and pages nobody.6
Prove it fires — one drill, on its own clock
An alert you have not seen fire is a guess. Run the drill below.
Prove the alert fires
Two drills. The first works for every drain and needs no change to your agents; the second is faster but needs traffic you can make fail.- Stop the heartbeat
- Drive an error spike
A drain heartbeats once every five minutes, and only in state Three things to expect, so the drill does not read as a failure:
healthy. Pausing it stops the heartbeat, the monitor’s fifteen-minute window empties, and it fires.- Ten to twenty minutes, not fifteen. The last heartbeat may be up to five minutes old when you pause, and the window is fifteen minutes long, so the earliest possible fire is about ten minutes after the pause. The monitor also carries
notify_no_datawith a twenty-minuteno_data_timeframe, so it fires on that leg by twenty minutes at the latest. Both legs are deliberate: for the one alert every other alert depends on, a duplicate page beats a missed one. - Nothing is lost. A paused drain keeps buffering. Records queue and deliver when you resume.
- Resume lands in
degraded, nothealthy. Heartbeats do not restart the instant you resume — the drain has to succeed a few times first. Watchlua drains statusuntil it readshealthyagain, then confirm the monitor recovered.
Two things not to use as a drill
- Do not point the drain at a broken endpoint to force
failing. It takes fifteen minutes with no success to get there and it teaches everyone watching that a red drain is normal. The pause drill proves the same alert in less time and loses nothing. - Do not push yourself over the daily quota. The degradation ladder drops
debugrecords at 100%,infoat 125%, and pauses the drain at 150% — a real loss of your own records, to test a monitor.
If it isn’t working
403 on every delivery
403 on every delivery
The API key is not valid for the site. Datadog keys are per-organization and per-site; a key minted in a
datadoghq.com organization is refused by datadoghq.eu. Recreate the drain with the right --site, or update the header value with lua drains update <id> --header DD-API-KEY.Logs arrive but service is empty
Logs arrive but service is empty
service is the agent’s name. An agent with no name set shows the id instead. A Datadog log pipeline that remaps service will also override it — check your pipelines before blaming the drain.Attributes are not searchable
Attributes are not searchable
Datadog indexes attributes but only facets the ones you ask it to. Open one record in Log Explorer, expand
lua.attributes, and create a facet on the fields you want to group or alert by. The monitor pack lists the three facets its queries need.429 from the intake
429 from the intake
Retried with backoff, honouring
Retry-After. Sustained rate limiting usually means several drains are pointed at the same Datadog organization; raise --min-severity to warn, or narrow --sources, to cut volume at the source rather than at the intake.Next steps
Datadog monitor pack
Four monitors to import, with the facets they need.
Event schema
What every field under
lua means.Delivery guarantees
Retry, drops, heartbeat, and health states.
lua drains
Every verb and flag.