> ## Documentation Index
> Fetch the complete documentation index at: https://docs.heylua.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Grafana Loki
> Push agent logs into Grafana Cloud or a self-hosted Loki with five stream labels and the whole record as the JSON line
A `loki` drain pushes to Grafana Cloud Loki or a Loki you run yourself. Each record becomes one entry in a stream, labelled with five labels and carrying the whole Lua record as its JSON line — so `| json` reaches every field at query time without a label for it.
Vendor destinations are opened per deployment. Where `loki` is not open yet, a create answers `422 DRAIN_TYPE_UNAVAILABLE` and the message names the types that are. **It may not be enabled on your deployment yet.**
**Before you begin**
* Your Loki host. On Grafana Cloud it is the Loki data source's URL, `https://logs-prod-012.grafana.net`.
* Credentials: on Grafana Cloud, your **instance ID** and an access policy token, used as HTTP Basic. A self-hosted Loki behind a gateway may take a bearer token instead.
* For a multi-tenant Loki: your tenant id, for `X-Scope-OrgID`.
## What to enter
| Field | Flag | Value |
| -------- | ------------------------ | ------------------------------------------------------------------------------------- |
| Name | `--name` | 1–64 characters, unique in the organization |
| Type | `--type loki` | |
| Endpoint | `--endpoint` | Your Loki host — `https://logs-prod-012.grafana.net` |
| Auth | `--header Authorization` | The **complete** header value: `Basic ` or `Bearer ` |
| Tenant | `--header X-Scope-OrgID` | Multi-tenant Loki only |
```bash theme={null}
lua drains create \
--name "Loki prod" \
--type loki \
--endpoint https://logs-prod-012.grafana.net \
--header Authorization \
--environments production \
--min-severity info
```
In CI:
```bash theme={null}
lua drains create --ci --json \
--name "Loki prod" --type loki \
--endpoint https://logs-prod-012.grafana.net \
--header-from-env Authorization=LOKI_AUTH
```
**Store the complete header value**, scheme included — `Basic dXNlcjpwYXNz…` or `Bearer glc_…`. The stored value is sent verbatim; Lua never prefixes a scheme onto it. For Grafana Cloud, the Basic credential is `:`, base64-encoded.
`/loki/api/v1/push` is composed onto whatever host you give, and composition is **idempotent** — paste the bare host or the full push URL, whichever your Grafana UI showed you. A non-default port is kept, which a self-hosted Loki usually needs; a query string or fragment is dropped.
There is **no host allow-list** for this type: Grafana Cloud and a self-hosted stack are equally your host. It is still validated as `https:` and as resolving entirely to public addresses, [re-checked on every send](/drains/protecting-your-destination).
`X-Scope-OrgID` is a **per-customer value**, so it rides as one of the drain's own headers rather than as something the preset sets. Add it with `--header X-Scope-OrgID` (the CLI prompts for the value) only if your Loki is multi-tenant. Grafana Cloud does not want it — the Basic credential already carries the tenant.
## How ownership verification works
**A vendor preset is reachability-checked, not ownership-verified.** `lua drains verify` pushes one batch holding a single `lua.drain.test` record, and any `2xx` from Loki is accepted. It proves the host answers and the credential works. It does **not** prove you own that host — no Loki push endpoint can echo a challenge token back. Only a [generic HTTPS](/drains/generic-https) drain is ownership-verified by a token echo, and only an [object-storage](/drains/object-storage) drain proves ownership by writing into a bucket.
```bash theme={null}
lua drains verify drn_5a72c0e8194bd3f67210cc42
```
Loki answers a successful push with **`204`**, which is a `2xx` and verifies fine. A `401` means the Basic credential is wrong or is missing its scheme; a `403` on Grafana Cloud usually means the access policy lacks `logs:write`. Verification is limited to 5 attempts per drain per hour.
Loki deliveries are **not signed**.
## What arrives
```json theme={null}
{
"streams": [
{
"stream": {
"org": "org_4f2c9a1b",
"agent": "agent_1789214224176_2vta8rnyn",
"environment": "production",
"source": "skill",
"level": "error"
},
"values": [
["1789217997644000000", "{\"id\":\"1789217997644-jtoulxxxq\",\"eventName\":\"lua.skill.error\",\"severityText\":\"ERROR\",\"body\":\"Ticket lookup failed: upstream timeout\",\"attributes\":{\"lua.source\":\"skill\",\"gen_ai.tool.name\":\"lookup_tickets\"}}"]
]
}
]
}
```
Records are grouped into streams by label set, and entries within a stream are sorted ascending by timestamp: an older Loki refuses an out-of-order push outright, and a modern one accepts it only inside its `unordered_writes` window.
### Five labels, and why there are only five
| Label | From |
| ------------- | -------------------------------------------------------------------------------------------- |
| `org` | `resource["lua.org.id"]` |
| `agent` | `resource["lua.agent.id"]` |
| `environment` | `resource["deployment.environment.name"]` |
| `source` | `attributes["lua.source"]` — **omitted** when the record has none, never sent as `undefined` |
| `level` | `severityText`, lower-cased |
Loki indexes label **sets**: every distinct combination is a stream with its own index entry and its own chunks. A label whose values are not a small closed enum is a cost bug in *your* Loki bill, not in ours — which is why `eventName`, tool names, and primitive names are deliberately not labels. They ride inside the JSON line, where `| json` reaches them at query time for free.
Loki's own ceiling is 15 labels. Five leaves room for whatever your own pipeline adds.
### Useful queries
```logql theme={null}
{org="org_4f2c9a1b", environment="production", level="error"}
{org="org_4f2c9a1b", source="skill"} | json | eventName = "lua.skill.error"
{org="org_4f2c9a1b"} | json | attributes_gen_ai_tool_name = "lookup_tickets"
sum by (agent) (rate({org="org_4f2c9a1b", level="error"}[5m]))
```
`| json` flattens nested keys with `_`, so `attributes["gen_ai.tool.name"]` is reachable as `attributes_gen_ai_tool_name`.
### Caps
| Cap | Value |
| --------------------------- | ----------------------------------------------- |
| Records per push | 500 |
| Uncompressed bytes per push | 768 KiB (Loki's default request limit is 1 MiB) |
| Bytes per line | 256 KiB — Loki's own `max_line_size` |
| Compression | `gzip`, always |
| Retried | `408`, `429`, and every `5xx` |
| Terminal | Every `3xx`, and every other `4xx` |
A record over 256 KiB has its `body` cut and suffixed `... [truncated by lua]` rather than being dropped.
### Two terminal 400s worth recognising
Every non-`429` `4xx` is terminal — the batch is dropped and counted under `rejected`, not retried, because the same bytes would be refused again.
* **`line_too_long`** — an entry exceeded your Loki's `max_line_size`. The 256 KiB per-record cap exists to make this unreachable, so if you see it, your Loki's limit is set below the default. Raise `max_line_size`, or narrow the drain's sources so the long records stop being produced.
* **`entry too far behind`** / **`too_far_behind`** — the record is older than your `reject_old_samples_max_age`. Lua only retries a batch for 6 hours, so this normally only bites a drain that was paused for longer and then resumed with a deep backlog. Raise `reject_old_samples_max_age`, or accept the loss of the oldest range.
## Test the credential with curl
```bash theme={null}
LOKI='https://logs-prod-012.grafana.net'
LOKI_AUTH='Basic '
NOW_NS=$(( $(date +%s) * 1000000000 ))
curl -sS -i -X POST "$LOKI/loki/api/v1/push" \
-H "Authorization: $LOKI_AUTH" \
-H 'Content-Type: application/json' \
-d "{\"streams\":[{\"stream\":{\"org\":\"org_4f2c9a1b\",\"agent\":\"agt_probe\",\"environment\":\"production\",\"source\":\"drain\",\"level\":\"info\"},\"values\":[[\"$NOW_NS\",\"{\\\"eventName\\\":\\\"lua.drain.test\\\"}\"]]}]}"
```
`204` with an empty body means accepted. Then query it back:
```bash theme={null}
curl -sS -G "$LOKI/loki/api/v1/query_range" \
-H "Authorization: $LOKI_AUTH" \
--data-urlencode 'query={org="org_4f2c9a1b"}' | jq .
```
Then do the same through the real delivery path:
```bash theme={null}
lua drains test drn_5a72c0e8194bd3f67210cc42
```
## If it isn't working
The stored `Authorization` value is incomplete. It must be the whole header value — `Basic ` or `Bearer ` — with the scheme. Re-enter it with `lua drains update --header Authorization`.
Your Loki is multi-tenant and wants `X-Scope-OrgID`. Add it: `lua drains update --header X-Scope-OrgID`.
`healthy` means Loki accepted the push. Check the label selector first — every stream carries `org`, `agent`, and `environment`, so `{org="…"}` alone should match. Then check the time range: Loki files entries at the record's own timestamp, not at ingest.
Not from these labels — five is deliberately few. Check whether a pipeline of yours is promoting fields out of the JSON line into labels. `eventName` and tool names have high cardinality and belong in the line.
## Next steps
An importable dashboard built on these fields.
Every field inside the JSON line.
What verification does and does not prove.
Every verb and flag.