Where a drain may point
A destination URL is checked when the drain is created, and the refusal names the rule that fired:
The first four answer
422 DRAIN_ENDPOINT_INVALID; not_allowlisted answers 422 DRAIN_ENDPOINT_NOT_ALLOWED. The message reads endpoint refused by the '<reason>' rule: <detail>.
DNS availability is not a precondition for saving configuration. A host that resolves to nothing is normally accepted at create time and refused later, at delivery — so
unresolvable is the one word on that list you are unlikely to meet on a create. It is what a delivery failure looks like instead, as errorClass: "dns", and unlike every other refusal on this page it is retried: a name that does not resolve now may resolve in a minute, while a policy refusal will not become allowed.::ffff:10.0.0.1 is unwrapped before it is classified — that is the oldest way to smuggle a private address past a check that only looks at the v6 form.
The ranges refused include 0.0.0.0/8 and :: — “this network” and the unspecified address, which some stacks route to a local interface — alongside 10/8, 172.16/12, 192.168/16, 100.64/10, 127/8, 169.254/16, fc00::/7, fe80::/10, ::1, and the well-known instance-metadata addresses.
It is re-checked on every single delivery
Which destination types pin a host
Splunk and Loki are unpinned for the same reason Better Stack is: both are commonly self-hosted on a domain of yours, and an allow-list there would refuse every legitimate drain.
What “verified” actually proves
This is the distinction most worth reading twice, and the drain itself now states which side of it a destination is on. A successful verification writesverification.ownership, and it takes one of two values:
They are not two grades of one claim. They are two different claims, and a UI or a compliance report must not round the second up to the first.
verifiedAt says when, never what. A drain verified before ownership existed carries a timestamp and no ownership at all, and that absence reads as reachability-only — the weaker of the two claims, which is the only safe way to read a field that was not there when the check ran. The value is filled in on the drain’s next successful verification.429 DRAIN_VERIFY_RATE_LIMITED. Changing a drain’s endpoint sends it back to pending_verification and stops delivery until it re-verifies; records buffer meanwhile.
Who agreed to send message content
includeContent — --include-content on the CLI — sends what your end users said and what your agents replied to a third party. Turning it on is a decision somebody makes, so the drain keeps the record of who made it, in contentAck:
The acknowledgement is asked for on the transition, when content starts flowing to a destination. A request that leaves
includeContent on as it was needs no fresh one, turning it off needs none, and turning it back on asks again.
The gate holds for every writer, not only an API call. A drain created by the workflow SIEM webhook migration was already shipping that content under your organization’s SIEM webhook policy, so it inherits that policy’s consent rather than inventing one, and names it:
contentAck was created before this record existed. Because the acknowledgement is a transition, sending includeContentAck at a drain whose content is already on records nothing — turn content off and back on if you want the consent on the row.
What is masked before anything leaves
Every record body and every string attribute passes a scrubber on its way to the wire. It cannot be turned off — only extended. There is no flag, no drain field, and no argument that removes a built-in rule, and an empty rule set still runs every one of them. The mask is visible on purpose —[redacted:builtin], [redacted:lua-token], [redacted:vendor-key], or [redacted:org:<your rule id>] — so a value that was scrubbed is distinguishable from one that was never there, and the marker says which rule fired. Scrubbing an already-scrubbed string is a no-op and adds no hits.
What is not masked
- Message content as such. The scrubber masks credential shapes; it is not a PII engine. End-user text is governed by content sources and
includeContentinstead. - Attribute keys. Only values are scanned — and no field key can carry a credential shape the scrubber masks. The platform’s own keys are a closed set, none of which is secret-bearing. An
app.<key>you supply withlog.*must match^[a-z][a-z0-9_]{0,63}$— lower-case letters, digits and_— which already excludes every shape in the table above that needs an upper-case prefix (AKIA,SK,AIza,xox), a hyphen (sk-ant-,sk-proj-) or a dot (a JWT). Five masked shapes are spellable in lower-case and underscores alone — a legacy Lua API key, a GitHub token, a Stripe key, a Lua handoff code and a Lua scoped key — and a key of any of those five shapes is refused outright at the call, with the rest. So the claim covers the whole record, not just the platform’s half of it: a key that a rule above would have masked as a value cannot be written as a key in the first place. - Numbers and booleans. The scrubber reads strings. No credential shape can be expressed in a number the platform wrote — but an
app.<key>you supply can be a number or a boolean, and those values are not scanned. A secret is not credential-shaped once it is an integer, so keep anything that might be sensitive in a string, where the rules can reach it. See what is not scrubbed. - A credential with no recognisable shape. A password that looks like an ordinary word, printed on its own with no key name, cannot be told from prose. Add a rule of your own.
Scrubbing is not a substitute for not logging secrets. Anything your code prints with
console.log is stored as printed, and the scrubber only catches shapes it recognises.Your own rules
A drain’sscrubRules is a list of { id, pattern, flags? }, compiled once and run after the built-ins.
A rule that fails any of these answers
422 DRAIN_SCRUB_RULE_INVALID at save time. The same function validates and compiles, so a rule that saves is exactly a rule that runs.
Your patterns are held to a linear-time engine
Every pattern you save is compiled with RE2, and a pattern RE2 cannot express is refused — at save time and again when a drain loads its rules, on every deployment. RE2 never backtracks. It visits each input position once, so applying one rule costs time linear in the length of the string, whatever the pattern is. That is the property that makes it safe to run a regular expression you wrote against every string in every record your agents produce. It is also why the refusals below are not negotiable. A static check can rule out the exponential pattern shapes; it cannot rule out the merely polynomial ones.a*a*! satisfies every check on this page, and on a backtracking engine a 5,000-character body never finishes matching it. The 5 ms budget is no help either — it is measured between rules, so it can never interrupt a single slow one. Only the engine bounds one application, and the checks below are the belt to its braces.
Lua’s own built-in rules are unaffected. They are frozen, reviewed, and pinned by a test corpus, and they mask byte-identically whatever engine runs them. Only your rules are held to RE2.
The patterns that are refused
Five constructs are rejected, because each of them either makes a regular expression exponential on the wrong input or cannot be expressed on a non-backtracking engine at all.- Backreferences —
\1,\k<name>. - Lookahead —
(?=…),(?!…). RE2 has no lookaround of any kind, and that is a large part of why it is linear. - Lookbehind —
(?<=…),(?<!…). Named groups,(?<name>…), are fine. - Nested unbounded quantifiers — a quantified group whose body itself contains
*,+, or{n,}:(a+)+,(\s*\w+)*. A bounded outer quantifier is fine:(ab){2,4},(a+)?b. - Unbounded repetition of an alternation —
(a|a)+,(?:x|xy)*,([a-z]|\w){3,}. Repeating branches that can match the same text is the other exponential shape;(a|a)+$needs only 35 characters of input to hang a process for minutes.
- A pattern that matches the empty string —
x*would insert a mask between every character. - A pattern that does not compile in unicode mode. Patterns are compiled with
u, which is stricter than the default: a stray identity escape such as\-outside a character class is a syntax error underualthough it is accepted without it. Write-, or escape it properly.
What a refusal looks like
Every one of these answers the same422, with the reason in message and nothing more machine-readable than that:
code is always the string above — it says a scrub rule was refused, not which check refused it — and field is always scrubRules. So branch on the message, or better, on nothing: fix the rule the message names and send it again. There is no per-rule code to switch on, and the message wording is not a stable interface.
The refusal names the rule id, never the pattern. A pattern is yours, and it is kept out of audit entries and log lines for the same reason a header value is.
Setting them
Scrub rules are part of the drain’s configuration and are sent asscrubRules on the REST API — on POST when you create a drain, or on PATCH to change them afterwards. Sending null clears them.
There is no
lua drains flag for scrub rules. A rule is a regular expression, and neither the create prompts nor any option on lua drains update will set one, so the REST API is the way to add, change, or clear them. lua drains list --json reads them back under scrubRules, and lua drains status <id> reports how often they fired as scrubHits24h.They reach an export as well as a delivery
A rule is configured on a drain, but it is not confined to that drain’s deliveries. A log export applies the union of every active drain’s rules across your organization, because an export is organization-wide and picking one drain’s list could only under-redact. So a rule you add anywhere raises the floor everywhere, andcounts.orgScrubHits on a finished export says how many values it caught. “Active” is every drain that is not disabled and not deleted; a paused drain still contributes. It may not be enabled on your deployment yet — until it is, an export runs the built-in classes alone.
An organization rule is bounded per record by a small time budget, so a very expensive rule set can be cut short on an individual record; the built-in classes are never cut short. An export reports that as counts.scrubBudgetExceeded. Read the budget as a fairness control rather than as the safety property: it is checked between rule applications, so it can never interrupt one. What bounds a single application is the linear-time engine every pattern is compiled on, and the refused patterns are the belt to its braces.
How a destination credential is held
Three separate secrets, all write-only, all stored encrypted and bound to your organization:last4 is an empty string rather than a shortened value when the stored secret is under 8 characters.
A secret is never a command-line flag, because a shell history and a CI log both keep one. --header prompts; --header-from-env and --credentials-from-env name environment variables whose values never reach argv. Either way the value is stored and sent verbatim as the whole header, so a scheme is part of what you supply: for a bearer token the prompt answer, or the variable’s value, is Bearer <token>, not the bare token. A bare token is the usual cause of a 401 on verification.
Rotating the signing secret
--finalize closes the window early; it closes on its own at 24 hours. A second rotation while a window is open answers 409 DRAIN_ROTATION_IN_PROGRESS. The full receiver-side recipe is on Verify signatures.
What a receiver must do
For a generic HTTPS drain, three obligations, and all three matter:- Verify the signature before parsing the body, and compare in constant time. The recipe and four working implementations are on Verify signatures.
- Reject a timestamp more than 300 seconds from now, in either direction. That is the replay window, it is inclusive at exactly 300, and the platform never signs a batch it will not send inside it. Widening it past five minutes gives up the replay protection.
- Cap the decompression, and answer
413. The signature covers the uncompressed body, so you must inflate before you can authenticate — the gunzip runs on bytes from a caller you have not checked yet. Refuse a compressed body over 2 MiB before inflating it, cap the inflated output at 8 MiB, and return413for either. Without a cap, a kilobyte of gzipped zeros expands to a gigabyte and takes your receiver down before the signature is ever checked. The recipe, per language.
What Lua never stores
- No request body. A delivery row holds the status code, latency, byte counts, error class, and at most 1 KB of the destination’s response — scrubbed. It has no field that can hold what was sent.
- No ownership-challenge token. Only its hash. A read of an object-storage drain’s
challengereturnstokenHash: ""and akeyPatterntelling you where to look — never the token, because a control plane that could hand it back would make the challenge prove nothing. - No credential in an error message. Provider errors carry the vendor’s own error code, never the SDK’s message: an SDK message quotes the signed request, and a signed request carries a credential.
- No secret in a notice. A drain notice is built from an allow-list of fields, so a secret-bearing field added to a drain in future is absent from every notice until somebody deliberately adds it there.
Next steps
Verify signatures
The HMAC recipe, four implementations, and rotation.
Egress allow-listing
The address deliveries leave from.
Delivery guarantees
At-least-once, retry, and what a drop looks like.
Log drains REST
Scopes, refusal codes, and the read model.