> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# Troubleshooting

> Symptoms you can search for on the LangWatch AI Gateway, with the check that confirms each cause and the fix.

Search this page for the status code or the error code your client printed. Each section carries the check to run and the change to make.

## Start with the request id

Every gateway response carries `X-LangWatch-Gateway-Request-Id`. Read it from your client's output and search for it in [Trace Explorer](https://langwatch.ai/docs/ai-gateway/observability.md). The trace holds the resolved provider, the dispatch attempts, the upstream latency and the guardrail verdicts for that one request.

Error responses share one body shape:

```json theme={null}
{
  "error": {
    "type": "rate_limited",
    "code": "rate_limited",
    "message": "rate_limited",
    "fault": "customer"
  }
}
```

The `code` field is the stable identifier. Match your client code on `code`; the `message` is copy and it changes.

## 401 on every request

The gateway answers `401 invalid_api_key` when it cannot resolve the bearer token to a virtual key.

**Confirm it:**

```bash theme={null}
curl -si https://gateway.acme.com/v1/models \
  -H "Authorization: Bearer $LW_VK" | head -20
```

A `401` with an empty `Authorization` value in your own shell means the variable never expanded in the running process. That is the most common cause.

**Causes and fixes:**

| Cause | Fix |
| - | - |
| The environment variable is empty in the client process | Print the variable from inside the running container |
| The client holds a secret that was rotated more than 24 hours ago | Copy the new secret from the rotate response into your secret store |
| The value is the display prefix, not the secret | A secret is `vk-lw-` plus 26 characters, 32 in total. The 13-character prefix shown in the UI is not a key |
| The virtual key was deleted or belongs to another organization | Create a key in the organization you are calling |

A revoked or disabled key answers `403`, not `401`. See the next section.

## 403 `virtual_key_revoked` or `virtual_key_disabled`

Both are deliberate stops on the key, and they are different states.

| Code | Status | Meaning |
| - | - | - |
| `virtual_key_revoked` | 403 | The key is dead. Revocation is terminal |
| `virtual_key_disabled` | 403 | The key is stopped and an administrator can turn it back on |
| `virtual_key_expired` | 403 | The key passed its expiration date |

**Fix:** re-enable a disabled key in **AI Gateway → Virtual Keys**, or with `langwatch virtual-keys enable <id>`. A revoked key cannot come back. Create a new key and move the callers to it.

## 429 with `rate_limited`

The virtual key hit its own requests-per-minute or requests-per-day ceiling. The gateway checks the ceiling before it calls any provider, so a rejected request burns no upstream tokens.

**Confirm it:** the response body carries `"code":"rate_limited"` and no `X-LangWatch-Provider` header. A `429` from the provider instead carries `X-LangWatch-Provider` and forwards the provider's own `Retry-After`.

The response does not say which dimension fired. The metric does:

```promql theme={null}
sum by (dimension) (rate(gateway_rate_limit_denied_total[5m]))
```

`dimension` is `rpm` or `rpd`.

**Fix:** raise or clear the ceiling on the key.

```bash theme={null}
langwatch virtual-keys update <id> --config-json '{"rate_limits":{"rpm":600,"rpd":100000}}'
```

The limiter runs in each pod against the key's cached configuration, so the ceiling applies per pod. Size it against one pod's share of your traffic. See [Rate limits](https://langwatch.ai/docs/ai-gateway/rate-limits.md).

For a provider 429, add a fallback credential to the key's chain, or ask the provider for more quota. See [Fallback chains](https://langwatch.ai/docs/ai-gateway/providers/fallback-chains.md).

## 400 `model_not_recognized`

The gateway resolved the virtual key but could not turn the `model` string into a provider and a model id.

**Confirm it:** list what the key can actually route to.

```bash theme={null}
curl -sS https://gateway.acme.com/v1/models \
  -H "Authorization: Bearer $LW_VK" | jq -r '.data[].id' | sort
```

**Causes and fixes:**

| Cause | Fix |
| - | - |
| The model has no provider prefix and no prefix rule matches it | Write `provider/model`, for example `openai/gpt-6-luna` or `anthropic/claude-sonnet-4-5` |
| The first segment is not a provider the key can reach | Use a provider id from the list above, or a routing handle defined on the key's provider row |
| An Azure deployment name is not in the deployment map | Add the deployment name on the provider credential, or send `azure/<deployment name>` |

Two neighbouring codes mean something else. `400 model_not_allowed` means the model is real but the key's allowlist refuses it. `400 model_provider_not_bound` means no credential in the key's scope serves that provider. See [Model naming](https://langwatch.ai/docs/ai-gateway/providers/overview.md).

## 402 `budget_exceeded`

A budget scope on the key breached its limit. The message names the scope.

**Fix:** raise the limit, move the key to a scope with headroom, or change the scope's breach action from block to warn. See [Budgets](https://langwatch.ai/docs/ai-gateway/budgets.md).

Spend reaches the budget through the spend records the gateway ships. If the ledger is not moving at all, read [Spend does not appear on the key](#spend-does-not-appear-on-the-key) instead.

## 502 or 503 during a rollout

A pod stopped serving before the load balancer removed it, or the drain window cut requests that were still running.

**Confirm it:** during a healthy drain, `gateway_draining` goes to 1 on the leaving pod and `gateway_in_flight_requests` falls to 0 before the pod exits. A flat in-flight count past the drain window is a handler that never returned.

```promql theme={null}
sum by (pod) (gateway_draining)
sum by (pod) (gateway_in_flight_requests)
```

**How the drain works.** On `SIGTERM` the gateway flips `/readyz` to `503 {"status":"draining"}`, waits `shutdown.preDrainWaitSeconds` (default 5) so the endpoint controller removes the pod, then closes the listener and gives in-flight requests `shutdown.timeoutSeconds` (default 60).

**Fix, 502 from the load balancer:** raise `shutdown.preDrainWaitSeconds` so the removal has time to propagate, and raise `terminationGracePeriodSeconds` by the same amount. The chart requires `terminationGracePeriodSeconds` to be at least `preDrainWaitSeconds + timeoutSeconds + 10`, and refuses to render otherwise.

**Fix, cut streams:** a provider call can run up to 14 minutes, so the default 60 second window cuts long streams on every rollout. The gateway says so at boot with `graceful_shutdown_shorter_than_max_stream_duration`. To finish long streams:

```yaml theme={null}
gateway:
  shutdown:
    preDrainWaitSeconds: 5
    timeoutSeconds: 840
  terminationGracePeriodSeconds: 855
```

A `preStop` sleep does not help. It delays the `SIGTERM`, so the pod still reports ready while the sleep runs and the load balancer has no signal to act on.

## A stream stops mid-way

The client received part of a response and then silence.

**Confirm it:** the gateway never switches providers mid-stream. When an upstream stream fails after the first chunk, the gateway writes one terminal SSE frame and closes:

```text theme={null}
data: {"type":"error","error":{"type":"provider_error","message":"..."}}
```

If your client shows silence instead of an error, it is not reading that frame. Log the raw SSE bytes and check.

**Streams that end with no error** are usually the provider closing early. Open the request's trace and compare the token counts against the response you got.

**A stream that reports no usage** records zero cost, so it bypasses budget enforcement:

```promql theme={null}
sum(rate(gateway_streaming_usage_missing_total[10m]))
```

Send `stream_options: {"include_usage": true}` on OpenAI-style requests. See [Streaming](https://langwatch.ai/docs/ai-gateway/streaming.md).

## A key change is not applied

You changed a limit, an allowlist or a credential, and requests still behave the old way.

**How a change reaches a pod.** Each pod holds the resolved key in memory and refreshes it three ways:

| Path | Timing |
| - | - |
| The change feed, a long poll against the control plane | About 10 seconds |
| The configuration re-fetch, for changes that emit no event | `LW_GATEWAY_AUTH_CACHE_CONFIG_TTL_SECONDS`, default 60 |
| The key's own token expiring | 15 minutes |

Rotations, revocations, budget edits, routing changes and cache-rule edits all travel on the change feed, so 10 seconds is the normal case.

**Confirm the feed is running:** the pod log carries an `auth_cache_change_evict` line with the change kind and the key id when a change lands. A `change_feed_poll_failed` line instead means the pod cannot reach the control plane, and every cached key stays as it is until the poll recovers.

**Fix:** restore the route to the control plane. `curl http://langwatch-gateway/debug/control-plane` inside the cluster prints the URL the pod resolved. A rolling restart clears every cache at once when you need the change now.

## Spend does not appear on the key

Traffic is flowing, and the key's spend and its budgets stay flat.

**How spend travels.** The gateway writes one record per request to a spool on disk. A background drainer ships the spool to the control plane, which prices the records and writes the ledger. Budget enforcement in the gateway is a local check against the budget snapshot in the cached key, so spend only moves when the whole chain runs.

**Confirm it, in order:**

| Check | Query | If it fails |
| - | - | - |
| Records are written | `gateway_spend_spool_appended_total` is climbing | Spend emission is off. Set `spend.enabled: true` |
| No records are lost | `gateway_spend_spool_dropped_total` is flat | `reason="intake"` means the pod outran the spool; `reason="overflow"` means the drainer is behind and the oldest segments were deleted |
| The control plane is answering | `gateway_control_plane_requests_total{status="error"}` is flat | The drainer cannot reach the control plane |

Every dropped record is spend that is never billed and never enforced against, so alert on the drop counter.

**A budget that should block but does not** is the same chain seen from the other end. The precheck is local and permissive, so a stale snapshot lets requests through rather than rejecting them. It reconciles once the ledger catches up.

## `503 auth_upstream_unavailable`

The gateway cannot reach the control plane and has no cached bundle for the key.

**What still works.** A key a pod has already resolved keeps serving. When its token expires while the control plane is unreachable, the pod extends the entry by `LW_GATEWAY_AUTH_CACHE_SOFT_BUMP_SECONDS` (default 300) on each failed refresh, up to `LW_GATEWAY_AUTH_CACHE_HARD_GRACE_SECONDS` past the token expiry (default 21600, six hours). Past that cap the entry is evicted and the key answers 503.

A key evicted by a change-feed event, such as the budget update every debit emits, keeps serving its last known config if the refetch fails for a transport reason (timeout, 5xx, network error). The fallback lasts up to one hour after the config was last confirmed by the control plane. Revoked, disabled, rotated, deleted and expired keys get no fallback.

A key the pod has never seen has no cached bundle and fails at once. There is no cache warm-up on boot, so a pod added during an outage starts cold, and so does every pod after a rolling deploy.

A `401`, `403` or `404` from the control plane about the key itself is not an outage. It evicts the entry immediately, with no grace.

**Read the ladder in the pod log:**

| Line | Level | Meaning |
| - | - | - |
| `auth_cache_serve_stale` | info | The grace window is active and keys are still being served |
| `auth_cache_refresh_transport_failure` | warn | Each refresh is failing. This is the outage signal, and cached keys are still serving |
| `auth_cache_hard_evict` with `reason=hard_cap_exceeded` | error | The grace window is spent and requests are now refused |

**Fix:** restore the control plane. Raise `LW_GATEWAY_AUTH_CACHE_HARD_GRACE_SECONDS` while an outage is still running if you would rather extend than refuse. Set it to a negative value to switch stale serving off entirely, which is what a strict revocation policy wants.

## First request to a new pod is slow

A pod resolves each virtual key from the control plane the first time it sees it, which adds one round trip. Every request after that reads the in-process cache. The cache is per pod and holds 10,000 keys.

Expect the extra round trip after a scale-up and after every rolling deploy.

## `413 payload_too_large`

The request body passed the cap, which the gateway applies at the edge before authentication.

**Fix:** shrink the body first. A 4K image is about 6 MB once base64 encoded, so a few full-resolution images plus a long system prompt reach the 32 MiB default. Raise `security.maxRequestBodyBytes` in the chart only when your traffic genuinely needs it, and write it as an integer literal. Never remove the cap on a public endpoint.

The audio upload routes have their own fixed cap of 26 MB.

## The gateway refuses to start

| Message | Cause | Fix |
| - | - | - |
| `LW_GATEWAY_AUTH_CACHE_... is no longer read` | A retired duration-string variable is still set | Set the `_SECONDS` variable the message names, as a plain number of seconds |
| `hosted gateway requires BLOCK_LOCAL_HTTP_CALLS=true` | `ENVIRONMENT` is set to anything but `local` | Set `BLOCK_LOCAL_HTTP_CALLS` and `REQUIRE_HTTPS_CUSTOM_ENDPOINTS` to `true` |
| A validation error naming `LW_GATEWAY_BASE_URL`, `LW_GATEWAY_INTERNAL_SECRET` or `LW_GATEWAY_JWT_SECRET` | One of the three required variables is empty | Set all three. See [Config](https://langwatch.ai/docs/ai-gateway/self-hosting/config.md) |
| `... is outside the supported range` | A seconds field holds a millisecond or nanosecond count | Write the value in seconds |

## See also

* [Errors](https://langwatch.ai/docs/ai-gateway/api/errors.md): every error code and its HTTP status.
* [Observability](https://langwatch.ai/docs/ai-gateway/observability.md): metric names, labels and span attributes.
* [Health checks](https://langwatch.ai/docs/ai-gateway/self-hosting/health-checks.md): what each probe reports.
* [CI smoke test](https://langwatch.ai/docs/ai-gateway/cookbooks/ci-smoke-test.md): catch these failures before your callers do.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.