> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# Observability

> The trace attributes, response headers and Prometheus metrics the LangWatch AI Gateway emits for every request.

## Where each signal lands

Every request through `/v1` produces three things you can read. The one exception is on the trace span: a successful chat or messages call that returns no output, no cost and no completion tokens is dropped before export.

| Signal | Where you read it |
| - | - |
| Trace span | Trace Explorer, in the project the virtual key writes its traces to |
| Response headers | On the HTTP response your client already receives |
| Prometheus metrics | `GET /metrics` on the metrics port (`9464` by default), when enabled. See [Metrics](#metrics-prometheus) |

The gateway starts one span per dispatched request and exports it to the project the virtual key names. The span is named `gen_ai.<operation>`, for example `gen_ai.chat`.

## Trace attributes

The table lists every attribute the gateway sets. An attribute whose value the gateway does not have is left off the span.

| Attribute | Meaning |
| - | - |
| `gen_ai.operation.name` | Request type: `chat`, `messages`, `responses`, `embeddings`, `speech`, `transcription`, `passthrough` or `realtime_session` |
| `gen_ai.provider.name` | Provider the request was dispatched to |
| `gen_ai.request.model` | Model dispatched, in the provider-prefixed spelling (`anthropic/claude-haiku-4-5`) |
| `langwatch.requested_model` | Model name the client sent. Set only when a routing policy rewrote it |
| `gen_ai.input.messages` | Request messages |
| `gen_ai.output.messages` | Response messages |
| `gen_ai.usage.input_tokens` | Input tokens the provider billed fresh, with cache reads and cache writes excluded |
| `gen_ai.usage.output_tokens` | Completion tokens |
| `gen_ai.usage.total_tokens` | Total the provider reported |
| `gen_ai.usage.cache_read.input_tokens` | Input tokens served from the provider's prompt cache |
| `gen_ai.usage.cache_creation.input_tokens` | Input tokens written to the provider's prompt cache |
| `gen_ai.usage.cache_creation_1h.input_tokens` | The part of the cache writes that bought a one-hour entry, when the provider reports it |
| `gen_ai.usage.input_audio_tokens` | Audio input tokens, counted apart from `gen_ai.usage.input_tokens` |
| `gen_ai.usage.output_audio_tokens` | Audio output tokens, counted apart from `gen_ai.usage.output_tokens` |
| `gen_ai.usage.input_chars` | Characters a text-to-speech call synthesized |
| `gen_ai.usage.audio_seconds` | Seconds of audio a speech-to-text call transcribed |
| `gen_ai.usage.cost` | Cost the gateway computed for the call |
| `gen_ai.conversation.id` | Session id the client sent, so multi-turn calls group into one thread |
| `langwatch.project_id` | Project the span is filed under |
| `langwatch.virtual_key_id` | Virtual key that authenticated the request |
| `langwatch.gateway_request_id` | Same value as the `X-LangWatch-Gateway-Request-Id` response header |
| `langwatch.model_provider_id` | Model provider row the request was dispatched to. Provider-filtered budgets match on this id |
| `langwatch.labels` | The virtual key's tags. Trace Explorer filters on them as **Label** |
| `langwatch.end_user_id` | End-user id from `X-LangWatch-End-User-Id`, its LiteLLM alias, or the `user` body field |
| `langwatch.reserved.request_metadata` | The `X-LangWatch-Metadata` JSON object, echoed back verbatim |
| `http.response.status_code` | Upstream status, set when the provider answered 400 or above |
| `error.type` | Error class on a failed request: `rate_limited`, `provider_timeout`, `provider_error`, `not_found`, `bad_request`, or the gateway error code that refused the call |

> [!NOTE]
>
> A successful chat or messages call that returns no output, no cost and no completion tokens is
> dropped before export. These are the empty probe calls coding agents send, and they would
> otherwise fill the trace list with rows that carry no data.

## Response headers

| Header | Value | Set when |
| - | - | - |
| `X-LangWatch-Gateway-Request-Id` | `req_` plus 30 hex characters, or the `X-Request-Id` you sent | Every dispatched request |
| `X-LangWatch-Trace-Id` | 32 hex characters | The gateway exports its own spans (`OTEL_OTLP_ENDPOINT` is set). Absent otherwise |
| `X-LangWatch-Span-Id` | 16 hex characters | Same condition as `X-LangWatch-Trace-Id` |
| `Traceparent` | W3C traceparent for the span written to your project | Every dispatched request |
| `X-LangWatch-Gateway-Version` | Gateway version string | Every request, when the deployment sets a version |
| `X-LangWatch-Fallback-Count` | Number of failed attempts before the one that served the request | One or more attempts failed |
| `X-LangWatch-Cache-Mode` | `respect`, `force`, `disable` or `auto` | Cache control resolved a mode |
| `X-LangWatch-Budget-Warning` | Comma-separated `<scope>:<percent>` entries, for example `project:95`. A provider-filtered budget reads `<scope>/<modelProviderId>:<percent>` | A budget crossed its warning threshold |
| `X-LangWatch-Params-Dropped` | Comma-separated parameter names the parameter policy removed | A translated request dropped parameters |
| `X-LangWatch-Guardrails-Not-Applied` | Reason the key's guardrails did not run, for example `realtime_session` | Guardrails were skipped |
| `X-LangWatch-Session-Id` | LangWatch id of the voice session a mint opened | Realtime session mints |
| `X-LangWatch-Provider` | Provider that returned the error | The gateway forwards a provider error |
| `X-Langwatch-Models-Discovery-Incomplete` | Providers that contributed no models, with the reason | `GET /v1/models` returned a partial list |

## Debugging a single request

Read `X-LangWatch-Gateway-Request-Id` off the response or the client's error, then search that value in Trace Explorer. That span shows the model, the provider, the token counts, the cost and the upstream status.

```bash theme={null}
curl -sD- -o /dev/null \
  -H "Authorization: Bearer $LANGWATCH_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -X POST https://gateway.langwatch.ai/v1/chat/completions \
  -d '{"model":"openai/gpt-6-luna","messages":[{"role":"user","content":"ping"}],"max_completion_tokens":16,"reasoning_effort":"none"}' \
  | grep -i '^x-langwatch-'
```

## Trace ID propagation

Send a `Traceparent` request header and the span written to your project becomes a child of your trace, under your trace id. Send no header and the span starts a trace of its own.

The `Traceparent` response header names that span. Forward it to the next hop in your system and everything stitches into one trace in Trace Explorer.

`X-LangWatch-Trace-Id` and `X-LangWatch-Span-Id` name the gateway's own operational span, which stays separate from your trace. They are present only on a deployment that exports the gateway's own spans; LangWatch Cloud responses carry `Traceparent` and `X-LangWatch-Gateway-Request-Id` only. Use them when you report a problem to LangWatch support.

## Metrics (Prometheus)

The gateway serves `GET /metrics` on its own listener, separate from the API port. Set `OTEL_METRICS_EXPORTER=otlp,prometheus` and `METRICS_API_KEY` to turn it on. The listener binds `OTEL_EXPORTER_PROMETHEUS_HOST:OTEL_EXPORTER_PROMETHEUS_PORT`, which default to every interface and `9464`. Each scrape sends `Authorization: Bearer <METRICS_API_KEY>`. Without `METRICS_API_KEY` the listener does not start and the gateway logs `aigateway_metrics_door_unmounted`. In the Helm chart, set `metrics.enabled=true` and keep the key in a Secret. See [Self-hosting: Helm](https://langwatch.ai/docs/ai-gateway/self-hosting/helm.md).

### Traffic and lifecycle

| Metric | Type | Labels | Meaning |
| - | - | - | - |
| `gateway_http_requests_total` | counter | `route`, `status`, `provider`, `model` | Requests served. `route` is the route pattern (`/v1/chat/completions`), `status` the HTTP status as a string |
| `gateway_http_request_duration_seconds` | histogram | `route`, `provider` | End-to-end latency, including auth, guardrails and the provider round trip |
| `gateway_in_flight_requests` | gauge | none | Requests this pod is handling now |
| `gateway_draining` | gauge | none | `1` while the pod drains after SIGTERM, `0` otherwise |
| `gateway_streaming_active` | gauge | none | Streaming responses open on this pod |
| `gateway_streaming_usage_missing_total` | counter | `provider`, `model` | Streams that closed without the provider reporting usage. They debit no budget, so alert on any positive rate |

### Provider dispatch and fallback

| Metric | Type | Labels | Meaning |
| - | - | - | - |
| `gateway_provider_duration_seconds` | histogram | `provider`, `model` | Upstream round trip per attempt. Subtract it from the request duration to isolate gateway overhead |
| `gateway_provider_attempts_total` | counter | `credential_id`, `outcome` | One increment per dispatch attempt against a credential |
| `gateway_fallback_events_total` | counter | `from_credential`, `to_credential` | Failovers, by the credential that failed and the one that served the request |
| `gateway_circuit_state` | gauge | `credential_id` | Circuit breaker state: `0` closed, `1` open, `2` half-open |

`outcome` is one of `success`, `fallback_success`, `retryable_5xx`, `not_found`, `rate_limit`, `timeout`, `network`, `circuit_open`, `non_retryable`, `chain_exhausted`, `context_done`.

### Authentication cache

| Metric | Type | Labels | Meaning |
| - | - | - | - |
| `gateway_auth_cache_hits_total` | counter | `tier` | Virtual-key resolutions served from cache |
| `gateway_auth_cache_misses_total` | counter | `tier` | Resolutions that fell through to the control plane |
| `gateway_auth_cache_lookups_total` | counter | none | Resolutions attempted. The denominator for the hit rate |
| `gateway_auth_cache_size` | gauge | `tier` | Virtual keys held in the cache |

`tier` always reads `l1`. The cache is an in-process LRU, private to each pod.

### Governance

| Metric | Type | Labels | Meaning |
| - | - | - | - |
| `gateway_budget_blocks_total` | counter | `scope` | Requests rejected by the budget precheck, by the scope whose limit was breached |
| `gateway_cache_hits_total` | counter | `outcome` | Prompt-cache result: `hit` when the provider reported cache-read tokens, `miss` otherwise |
| `gateway_cache_rule_hits_total` | counter | `rule_id`, `mode_applied` | Cache-control rule matches. `mode_applied` is `RESPECT`, `FORCE` or `DISABLE` |
| `gateway_guardrail_verdicts_total` | counter | `direction`, `verdict` | Guardrail decisions. `direction` is `request`, `response` or `stream_chunk`; `verdict` is `allow`, `block`, `modify` or `fail_open`. `fail_open` means the guardrail service could not be reached and the traffic passed unchecked |
| `gateway_rate_limit_denied_total` | counter | `dimension`, `vk_id` | Requests denied by a rate limit. `dimension` is `rpm` or `rpd` |
| `gateway_client_rejects_total` | counter | `code`, `vk_id` | Requests the gateway itself rejected as the caller's fault after authentication, by gateway error code and virtual key |

`gateway_client_rejects_total` counts only what the gateway refused. Three cases stay off it on purpose, because each would make a per-key alert fire for a key doing its job:

* Authentication failures, which have no key to attribute yet. Watch `gateway_http_requests_total{status="401"}`.
* Provider rejections, such as an upstream 429. Watch `gateway_provider_attempts_total{outcome="rate_limit"}`.
* Rate limiting, which a key at its ceiling would pin. Watch `gateway_rate_limit_denied_total`.

To find a client stuck in a rejection loop, rank the keys first:

```promql theme={null}
topk(5, sum by (vk_id) (rate(gateway_client_rejects_total{code!=""}[15m])))
```

Then break the top key down by code, with the full key id in place of `<vk-id>`:

```promql theme={null}
sum by (code) (rate(gateway_client_rejects_total{vk_id="<vk-id>"}[15m]))
```

### Realtime voice

| Metric | Type | Labels | Meaning |
| - | - | - | - |
| `gateway_realtime_mints_total` | counter | `vendor`, `outcome` | Voice session mint attempts. `outcome` is `minted`, `session_limit`, `registry_unavailable` or `provider_error`. `minted` counts sessions admitted at mint time; the vendor's later report decides what a session cost |
| `gateway_realtime_session_limit_blocks_total` | counter | none | Mints refused because a key already held its maximum open sessions. A rise means the cap is too low for the key's traffic, or its sessions stay open. The key is in the trace and the log line |
| `gateway_realtime_registry_errors_total` | counter | `operation` | Failed calls to the voice-session record, by `reserve`, `correlate`, `release` or `usage`. A `reserve` failure refuses the mint; a `correlate` failure costs the session its exact join to the vendor's report |
| `gateway_voice_sessions_supervised` | gauge | `kind` | Brokered voice calls this pod meters from its own server-side socket, by `live` or `realtime`. Media does not pass through the pod |
| `gateway_voice_sessions_ended_total` | counter | `kind`, `reason` | Supervised calls that ended. `reason` is `vendor_closed`, `budget_exceeded`, `key_revoked`, `sideband_lost`, `drain` or `session_closed`. Every reason but `vendor_closed` is the gateway ending the call |
| `gateway_voice_usage_reports_total` | counter | `outcome` | Usage reports sent for supervised calls. `outcome` is `recorded`, `duplicate`, `closed`, `already_closed`, `error` or `dropped`. `error` is retried; `dropped` is usage lost because one call's retry queue was full |

### Control plane

| Metric | Type | Labels | Meaning |
| - | - | - | - |
| `gateway_internal_rtt_seconds` | histogram | `endpoint` | Round trip to the control plane. Separates a slow control plane from a slow provider |
| `gateway_control_plane_requests_total` | counter | `endpoint`, `status` | Control-plane calls by outcome. `status` reads `error` when the call got no response |

`endpoint` is one of `resolve-key`, `guardrail-check`, `config`, `codex-refresh`. The change-feed long poll is excluded, because it blocks until an event or its own timeout and its timing says little about control-plane health.

### Spend emission

Every request writes a spend record to a bounded on-disk spool, and a background drainer ships the spool to the control plane. The control plane debits budgets from those records, so a record the spool drops is spend that is never billed and never enforced against.

| Metric | Type | Labels | Meaning |
| - | - | - | - |
| `gateway_spend_spool_appended_total` | counter | none | Spend records written to the spool. The denominator for the drop counter |
| `gateway_spend_spool_dropped_total` | counter | `reason` | Records lost before they could ship |

`reason` is `intake` when the spool's writer could not take the record, and `overflow` when the oldest sealed segments were deleted to keep the spool inside its 64 MiB bound. Alert on any sustained positive rate:

```promql theme={null}
sum by (reason) (rate(gateway_spend_spool_dropped_total[10m])) > 0
```

Both counters read `0` on a pod whose spool failed to open. That pod serves traffic and emits no spend at all. It logs the failure at startup, and you can spot it from a scrape by reading these two counters against a live `gateway_http_requests_total`.

The spool is pod-local and stays off a volume. A pod deleted with records still unshipped loses them, bounded by the drain lag of about a second of traffic.

The registry also carries the standard Go runtime and process collectors (`go_*`, `process_*`).

**Also check:** [Troubleshooting](https://langwatch.ai/docs/ai-gateway/troubleshooting.md) for reading a failed request end to end, [Security](https://langwatch.ai/docs/ai-gateway/security.md) for what the gateway records and what it never logs, [Self-hosting: Configuration](https://langwatch.ai/docs/ai-gateway/self-hosting/config.md) for the environment variables.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.