Ship logs to your stack

A log drain copies your agents’ execution log records to a destination you own — your own HTTPS endpoint, an OpenTelemetry collector, one of seven log vendors, or a bucket in your own cloud account — as they happen. It is a copy, not a move: everything a drain sends is still readable with lua logs and still counts against the platform’s own retention window. For a window that has already passed, rather than everything that happens next, there is an asynchronous export of the same records in the same shape. Drains exist so the logs you already have stop being a place you visit and start being a place that pages you. The monitor packs turn a working drain into a firing alert in a few minutes.

Log drains are switched on for a deployment as a whole, not per organization. While they are off, lua drains and the drain routes answer 404 — the resource does not exist rather than being forbidden, so a caller cannot tell the feature apart from a typo. If every verb answers 404, ask [email protected].

What a drain is

A drain is a rule that lives on the organization, not on one agent. One drain can cover every agent in the organization, including agents created after the drain was, or a named list. That is deliberate: a platform team configures one destination and one credential, and application teams keep shipping agents into it without touching it. Each drain has:

Part	What it is
A destination	A type plus the endpoint, site, bucket, or token that type needs
Selectors	Which sources, severities, agents, and environments are copied
A state	Where the drain is in its lifecycle, from unverified to healthy to paused
Health counters	Delivered, dropped, and rejected counts over the last 24 hours, latency, and backlog
A quota	A daily event and byte allowance, with a degradation ladder before it pauses

Create one with lua drains create, over the REST API, or from the admin dashboard.

lua drains create \
  --name "Datadog prod" \
  --type datadog \
  --site datadoghq.eu \
  --header DD-API-KEY \
  --environments production \
  --min-severity info

What arrives

Every delivery is a batch: one resource object describing the agent, and up to 500 records describing what happened. The shape is versioned and published as a JSON Schema at https://docs.heylua.ai/schemas/logs/1.0; the field-by-field reference is on Event schema.

{
  "schemaUrl": "https://docs.heylua.ai/schemas/logs/1.0",
  "batchId": "01JBW8N2Q4Z6H8Y0KX3R9T5V7M",
  "resource": {
    "service.name": "support-agent",
    "service.namespace": "org_4f2c9a1b",
    "service.instance.id": "agent_1789214224176_2vta8rnyn",
    "deployment.environment.name": "production",
    "lua.org.id": "org_4f2c9a1b",
    "lua.agent.id": "agent_1789214224176_2vta8rnyn"
  },
  "records": [
    {
      "id": "1789217997644-jtoulxxxq",
      "timestamp": "2026-09-12T12:59:57.644Z",
      "observedTimestamp": "2026-09-12T12:59:59.102Z",
      "eventName": "lua.skill.error",
      "severityNumber": 17,
      "severityText": "ERROR",
      "body": "Ticket lookup failed: upstream timeout",
      "attributes": {
        "lua.source": "skill",
        "lua.primitive.name": "tickets",
        "gen_ai.tool.name": "lookup_tickets",
        "gen_ai.operation.name": "execute_tool",
        "exception.type": "Error",
        "exception.message": "upstream timeout"
      }
    }
  ]
}

Consumers dedup on records[].id, never on X-Lua-Batch-Id. Those attributes are the platform’s. Your own code can add its own beside them: log.info('Ticket resolved', { tenant: 'acme', ticket_id: 48219 }) in a skill, a job, a webhook or a trigger puts app.tenant and app.ticket_id on the record, searchable at every destination without a parsing rule. See Emit your own fields.

Selectors

Selectors decide what a drain copies. They are evaluated per record, before anything leaves the platform.

Sources

A source is where a record came from. Twenty-five are selectable: the eighteen agent log sources, execution — which carries the start, completion, and failure of an execution rather than a line your code printed — and six adjacent-record sources, which are the platform’s own records of things that happened next to a turn rather than lines an agent wrote.

The six adjacent-record sources — delivery, trigger-execution, job-execution, workflow-run, voice-call, and workflow-audit — and four lua.execution.* outcome names are available from lua-cli 3.39.0. An earlier CLI refuses those six names.

Source	What it carries	On a new drain
`skill`	Tool calls: input, result, duration, and thrown errors	Yes
`job`	Scheduled job runs	Yes
`webhook`	Webhook handler runs	Yes
`trigger`	Trigger handler runs	Yes
`preprocessor`	Preprocessor decisions and blocks	Yes
`postprocessor`	Postprocessor rewrites	Yes
`device`	Device command handling. Declared, not emitted yet — device work is logged under `device-trigger` today, so selecting this source adds nothing	Yes
`device-trigger`	Device trigger handler runs	Yes
`mcp`	MCP server tool calls, by server and tool name	Yes
`rag`	Knowledge-base retrieval	Yes
`runtime`	The agent runtime’s own lines, including model-provider errors	Yes
`model-resolver`	Which model a turn resolved to, and why	Yes
`workflow`	Workflow run lifecycle	Yes
`workflow-step`	Per-step execution	Yes
`workflow-script`	Script-form workflow output	Yes
`agent_error`	Failures of the message pipeline itself: billing, validation, model errors	Yes
`execution`	Execution start, completion, and failure, with a duration — plus the four outcome names below	Yes
`delivery`	Outbound message status: queued, accepted, sent, delivered, read, failed	Yes
`trigger-execution`	Trigger dispatches — the accept and the outcome, with the tools a turn used	Yes
`job-execution`	Job outcomes only: the seven terminal statuses, with CPU, wall time, and peak memory	Yes
`workflow-run`	Every workflow run event, in the run’s own sequence order	Yes
`voice-call`	Voice call outcomes: status, direction, provider, channel, end reason, duration	No — opt in
`workflow-audit`	The organization’s workflow audit trail	No — opt in
`user_message`	What an end user said	No — content
`agent_response`	What the agent replied	No — content

Four sources are off on a new drain, for two different reasons. user_message and agent_response carry end-user content. They are off unless you turn on --include-content, which prints an acknowledgement you have to accept before the drain is created. Even then, every record still passes the scrubber, and a drain’s redact list drops named attributes — user.id, for instance — before the record is encoded. voice-call and workflow-audit are off simply because you opt into them: voice-call is narrower and more sensitive than a job outcome, and workflow-audit is a compliance stream an operator chooses deliberately. Neither has anything to do with --include-content — name them in --sources and they arrive, with no acknowledgement and no content prompt. (--include-content still governs which of their attributes arrive, as it does for every source with a content half.)

Existing drains keep the sources they were created with. The default set applies when a drain is created and is never re-applied afterwards, so nothing is added to a drain you already configured. The one exception is execution, which has always been a default: a drain that selects it starts receiving four new event names the moment this release ships, with no opt-in.

lua.execution.completed   lua.execution.failed   lua.execution.timeout   lua.execution.killed

They join the lua.execution.start|complete|error names the source already carries, and lua.execution.status tells them apart. New values on an existing source are additive under the schema policy — and this is exactly why a consumer switches on eventName with a default branch.

The adjacent-record sources are observable only through a drain. delivery, trigger-execution, job-execution, workflow-run, voice-call and workflow-audit records are built for the drain from the platform’s own delivery, trigger, job, workflow and audit ledgers; they are not execution-log rows, so lua logs, lua logs --type and the dashboard’s log view never show them. To see them, point a drain at a destination you can read.

The adjacent-record sources have no environment of their own, so every record they produce is read as production. A drain scoped to --environments sandbox alone receives none of them — not a delivery, not a trigger dispatch, not a run event, not an audit entry. Give an adjacent-record drain production in its environment list.

Severity

--min-severity keeps records at or above one of debug, info, warn, error. The default is info, which drops debug lines — the bulk of most agents’ volume. That default is the commonest reason a perfectly healthy drain looks silent. Anything your code prints with console.log is a DEBUG record (severityNumber 5), so a webhook, trigger or tool whose handler only logs that way produces nothing but debug rows and a drain at the default receives none of them. If a drain isn’t delivering walks through it. --sampling debug=0.1 keeps a deterministic tenth of debug records instead of all or none. Sampling is by execution, so a sampled execution keeps all of its debug lines rather than a scattered fraction of them; only debug is sampleable.

Agents and environments

--agents all (the default) covers every agent in the organization, now and later. --agents <id>,<id> pins the drain to a list. --environments production (the default) or --environments production,sandbox decides whether the sandbox is copied too; a sandbox drain is the fastest way to prove a destination works before production traffic reaches it. A sandbox-only drain proves a destination; it does not exercise every source. The six adjacent-record sources record no environment of their own and are read as production, so a drain without production in its list receives none of them however it is otherwise configured. Prove the destination on sandbox, then add production before you rely on the coverage. Pinning --agents to a list has a matching edge on workflow-audit: most audit actions have no agent at all, so an agent-pinned audit drain receives only the run-scoped actions of those agents. Workflow audit records has the three selectors that will quietly deliver you nothing.

Testing a production-scoped drain: chatting to your agent may not exercise it. A turn you drive with lua chat arrives over a development channel, and a development channel is recorded against the sandbox environment. A drain scoped to production alone then sees nothing at all from that kind of testing, and it looks exactly like a broken drain.Where the platform has enabled it, lua chat -e production now says so on the wire: the turn declares environment: "production" on the chat request, its log rows are labelled production, and a production-scoped drain receives them. It may not be enabled on your deployment yet — where it is not, -e production is still recorded as sandbox, exactly as before. Three things hold either way:

A preview stays sandbox. --agent-version <n> runs a snapshot that was never promoted, which is sandbox by every definition the platform uses, so its rows are labelled sandbox even though the flag forces -e production onto the session.
The declaration only ever raises. A turn may declare production and never sandbox, so nothing a caller puts in a request body can move a real production turn out of a drain that selects it.
Not every row of the turn follows. The chat rows themselves and the turn’s tool, MCP, rag, skill, preprocessor, postprocessor and sandbox rows carry the declared environment; runtime rows, which the agent runtime writes from wherever it happens to log, are still labelled sandbox. A drain selecting both environments sees the whole turn; one selecting production alone sees a real part of it, not all of it.

Two ways round it, and the first is better:

Exercise the drain with something that really runs in production — a webhook delivery, a scheduled job, or a trigger. Those produce production records and prove the selector as well as the destination.
Put both environments on the drain while you test (--environments production,sandbox) and take sandbox off when you are done.

lua drains test <id> is the other check worth running, but note what it proves: it sends one synthetic record down the real delivery path, so it proves the destination and the credential, not that your selectors match anything.

Run lua drains create inside a project directory and --agents defaults to that project’s agent rather than to every agent in the organization.

States

A drain moves through six states. Delivery only happens in three of them.

State	Delivering?	Buffering?	Meaning
`pending_verification`	No	Yes	Created, or its endpoint changed. Records queue until ownership is proved
`healthy`	Yes	—	Verified and succeeding
`degraded`	Yes	—	More than 10% of the last five minutes’ attempts failed, or it has just resumed
`failing`	Yes	—	No success for 15 minutes. Still trying
`paused`	No	Yes	Stopped by you, by the quota ladder, or automatically after 24 hours without a success
`disabled`	No	No	Turned off. Nothing is queued while it is off

A drain that stays in failing for 24 hours pauses itself. Resuming always lands in degraded, never straight in healthy: it has to prove itself again. Changing the endpoint or the type sends a drain back to pending_verification, and records buffer while you re-verify. Delivery guarantees has the full transition table.

lua drains status

Output

NAME            TYPE         STATE     ENV         AGENTS  BACKLOG  LAST SUCCESS
Datadog prod    datadog      healthy   production  all     0        2026-09-21T09:14:02Z
Receiver (dev)  http         degraded  sandbox     2       142      2026-09-21T09:08:47Z

lua drains status exits 2 when any drain is failing. That is also the usage exit code, so a guard that must tell a failing drain from a mistyped command reads the state field in --json rather than the exit status.

What never leaves Lua

Three things hold, on every drain, with no way to turn them off:

Credential shapes are masked. Every record body and every string attribute passes a scrubber before it is encoded. It replaces recognizable credentials — bearer and basic headers, JWTs, private-key blocks, connection-string passwords, URL secret parameters, Lua API keys and handoff codes, and well-known vendor keys from Stripe, Slack, GitHub, Twilio, Google, OpenAI, and Anthropic — with a marker such as [redacted:vendor-key]. Your organization can add up to 20 regular expressions of its own, each at most 256 characters, masked as [redacted:org:<rule-id>].
Destination credentials are never readable. Header values and the HMAC signing secret are write-only. A read of a drain returns header names and the last four characters, never a value; the signing secret is shown exactly once, when it is created or rotated.
No request body is stored. The delivery log keeps the status code, latency, byte count, error class, and up to 1 KB of the destination’s response — scrubbed. It has no field that can hold what was sent.

Scrubbing is not a substitute for not logging secrets. Anything your code prints with console.log is stored as printed, and the scrubber only catches shapes it recognizes.

Quotas and limits

Limit	Value
Drains per organization	5; 10 on Enterprise
Records per batch	500; 1,000 for Datadog; 10,000 for Axiom
Uncompressed bytes per batch	1 MiB; 5 MiB for Datadog; 448–768 KiB on the other vendor presets; 50 MB for object storage
Custom headers per drain	10
Organization scrub rules	20, at most 256 characters each
`redact` attribute keys	32, shared between the platform’s keys and the fields your code supplies
Fields per `log.*` call	32 keys, counting the `.count` a list field adds; 1 KiB per value, 8 KiB in total
Verification attempts	5 per drain per hour
Test deliveries	10 per drain per hour
Events per day	1,000,000 on Team; 10,000,000 on Business; unmetered on Enterprise
Bytes per day	1 GiB on Team; 10 GiB on Business; unmetered on Enterprise

Going over the daily quota degrades the drain in steps rather than cutting it off:

Usage	What happens
80%	A notification. Nothing is dropped
100%	`debug` records are dropped
125%	`info` records are dropped too
150%	The drain pauses, with reason `quota`

warn and error records are the last to go: the ladder never drops them before it pauses the drain. Quota counters reset at 00:00 UTC and are visible on lua drains status --json under quota.

Egress allow-listing

Deliveries leave from Lua’s own network, not from your agent’s sandbox, so an IP allow-list needs the platform’s egress address rather than a DNS name. There is one, in eu-west-1:

34.255.23.103/32

It covers every drain in both the production and sandbox agent environments — a drain with --environments production,sandbox needs nothing else allow-listed. The same value is served machine-readably from https://docs.heylua.ai/drains/egress.json, so an allow-list can be refreshed from CI rather than copied by hand. An address changes rarely and never without notice on this page. Every delivery identifies itself with User-Agent: LuaDrain/1.0 (+https://docs.heylua.ai/drains). The X-Lua-* protocol headers ride the generic HTTPS drain only, which is also the only destination whose batches are signed: X-Lua-Batch-Id, X-Lua-Drain-Id, X-Lua-Schema, and X-Lua-Signature (see Verify signatures). An OTLP delivery carries X-Lua-Batch-Id and nothing else of ours. No vendor preset carries any of them: a vendor intake gets the vendor’s own authentication header, and the batch and drain ids travel inside the record instead. Route a shared receiver by X-Lua-Drain-Id only when it is a generic HTTPS drain that reaches it. An object-storage drain sends no X-Lua-* headers either — it writes objects rather than making a request you can inspect, and the batch id is the object’s file name. For one release, a generic HTTPS drain created by the workflow SIEM webhook migration carried two more — X-Lua-Signature-Legacy and X-Lua-Timestamp — so a receiver written against the retired webhook could keep verifying while it moved across. They appeared on no other drain, and both have now been withdrawn: nothing sends either header on any drain, whatever it was created by, and X-Lua-Signature is the only signature a delivery carries.

Pick a destination

`--type`	Where it goes	What you supply	Verified by	`ownership`	Signed
`http`	Your own receiver	The full URL	Token echo	`verified`	Yes
`otlp`	An OTLP/HTTP collector	The full URL	Test post	`reachability-only`	No
`datadog`	Datadog Logs	A site, and `DD-API-KEY`	Test post	`verified`	No
`betterstack`	A Better Stack source	The ingesting host, and `Authorization`	Test post	`reachability-only`	No
`splunk`	Your Splunk HEC	Your HEC host, and `Authorization: Splunk <token>`	Test post	`reachability-only`	No
`loki`	Grafana Cloud or self-hosted Loki	Your Loki host, and `Authorization`	Test post	`reachability-only`	No
`axiom`	An Axiom dataset	The full ingest URL, and `Authorization`	Test post	`verified`	No
`newrelic`	The New Relic Log API	A site, and `Api-Key`	Test post	`verified`	No
`sumologic`	A Sumo Logic HTTP Source	The full Source URL — the URL is the credential	Test post	`verified`	No
`s3`	A bucket you own — S3, R2, or GCS	Bucket, region, prefix, and a key pair or a role	Bucket challenge	`verified`	No

A test post to a host you chose is reachability-checked, not ownership-verified. It proves the host answers and the credential works — not that the destination is yours, because no vendor intake can echo a challenge token back. A drain records which of the two its last verification established, as verification.ownership, and the four reachability-only rows above are the destinations where you name the host yourself. The control that matters is therefore who holds logs:manage; see Protecting your destination.

Destination types are opened per deployment. Where one is not open yet, a create answers 422 DRAIN_TYPE_UNAVAILABLE and the message names the types that are — a shorter list is a rollout state, not a bug. The five vendor presets and object storage may not be enabled on your deployment yet.

Generic HTTPS

Your own receiver. Signed, with a copy-paste Node example.

Object storage

A bucket you own, queryable with Athena or BigQuery.

OpenTelemetry

OTLP/HTTP into a collector, Loki, or anything downstream of one.

Datadog

Log Explorer, with service, status, and env already right.

Next steps

Event schema

Every field, the severity table, and the versioning policy.

Emit your own fields

Attach structured fields from agent code and search on them.

Delivery guarantees

At-least-once, batching, retry, and what a drop looks like.

Protecting your destination

Where a drain may point, what is scrubbed, and what verification proves.

Usage and plan quotas

What is metered, what counts as billable, and the daily allowances.

Drain notices

The six things Lua tells your admins about.

Delivery records

Did the message arrive? Statuses, masking, and what is not reported.

Workflow audit records

The audit trail as a drain, and the SIEM webhook migration.

Log export

A one-off NDJSON export of a window that has already passed.

lua drains

Every verb, flag, and --json shape.

Log drains REST

The routes, bodies, and error codes.

Get started

Concepts

Build

Channels

Integrations

Devices

Ship and operate