> ## Documentation Index > Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details. # Fallback Chains > When a provider fails, the LangWatch AI Gateway retries the request on the next provider the virtual key can reach. ## What a fallback chain is A fallback chain is the ordered list of model providers a request may be sent to. The gateway sends the request to the first provider in the chain. If that provider fails in a way that is worth another try, the gateway sends the same request to the next one, up to the key's attempt limit. The chain for a request is built from three things: 1. **The key's routing setting**: no fallback, fall back to all eligible providers, or a routing policy. 2. **The requested model**: only providers that can serve the model stay in the chain. A request for `claude-sonnet-4-5` is never sent to an OpenAI credential. 3. **The key's provider access and budgets**: a provider the key is not allowed to use, or whose provider-scoped budget is exhausted, is removed. ## Routing settings on a key Open the key under **AI Gateway, Virtual Keys** and click **Edit**. The **Routing** section lists two fixed choices, then one row per routing policy the key can reach: | Choice | Chain | | - | - | | Default: no fallback | One attempt. The provider's error reaches the caller unchanged. | | Fall back to all eligible providers | Every provider the key's scope can reach, ordered by each provider's **Fallback priority** (Settings, Model Providers, Advanced), lowest first, then by creation date. | | A routing policy | The providers the policy lists, in the policy's order. See [Routing policies](https://langwatch.ai/docs/ai-gateway/governance/routing-policies.md). | The attempt limit is 3 by default and is part of the key's configuration (`fallback.maxAttempts` on the management API). With no fallback the limit is 1. ## What triggers a fallback The gateway decides from the provider's real answer, and this is not configurable. | Provider outcome | Next provider? | | - | - | | HTTP 5xx | Yes | | HTTP 429 | Yes | | HTTP 404 | Yes. In a multi-provider chain a 404 usually means this provider does not serve that model. | | Timeout, connection failure | Yes | | Any other 4xx (400, 401, 402, 403) | No. The error reaches the caller. | | Credential or provider configuration refused before the call | No. Retrying gives the same answer. | | The caller disconnected | No | When every attempt fails, the caller gets the error of the last provider tried, with that provider's status and body. ## Streaming For a streaming request the gateway falls back only while opening the stream. Once the first bytes reach the caller, a provider failure ends the stream with an `event: error` frame. Chunks from two providers are never spliced into one response. ## What the caller sees The response carries `X-LangWatch-Fallback-Count` with the number of failovers that happened, and only when it is at least 1. A request served by the first provider has no header. ```http theme={null} HTTP/1.1 200 OK X-LangWatch-Fallback-Count: 1 X-LangWatch-Gateway-Request-Id: req_3e8385c4e62cbe4e31ca9c410c5891 ``` The body is the provider's normal response and does not name the skipped provider. The span records the provider that served the request. Prometheus counts every attempt in `gateway_provider_attempts_total{credential_id, outcome}` and every failover in `gateway_fallback_events_total{from_credential, to_credential}`. ## Circuit breaker Each provider credential has a circuit breaker. After 10 failures inside a 30 second window the breaker opens and the gateway skips that credential for 60 seconds, then lets one probe request through. A 5xx, a timeout or a connection failure counts as a failure; a 429 or a terminal 4xx does not. The window decides: the same ten failures spread over more than 30 seconds leave the breaker closed. A skipped credential does not consume an attempt. When the breaker is open on every credential in the chain the caller gets `503 circuit_open` with the message `provider temporarily unavailable: repeated upstream failures opened the circuit breaker, retry shortly`. The window, the threshold and the open duration are gateway settings: `LW_GATEWAY_CIRCUIT_WINDOW_S`, `LW_GATEWAY_CIRCUIT_THRESHOLD`, `LW_GATEWAY_CIRCUIT_COOLDOWN_S`. `gateway_circuit_state{credential_id}` reports 0 closed, 1 open, 2 half-open, and is the only view of the breaker state. A skipped attempt is counted as `outcome="circuit_open"` in `gateway_provider_attempts_total`. ## Errors before the first attempt | Code | Status | Cause | | - | - | - | | `model_not_allowed` | 400 | The model resolves to a provider the routing policy does not list. The message names the provider and the policy. | | `policy_violation` | 403 | A restriction on the policy refuses the model, or the model is not in the policy's allow-only list. The message names the model: `model "gemini-2.5-flash" is blocked by policy`, or `model "gemini-2.5-flash" is not in allowlist`. See [Policy rules](https://langwatch.ai/docs/ai-gateway/policy-rules.md). | | `model_not_allowed` | 400 | The model is outside the key's own model allowlist, or resolves to a provider the key's provider access excludes. | | `no_provider_configured` | 400 | The key reaches no provider at all. | | `model_not_recognized` | 400 | No provider on the key declares the model. See [Model naming](https://langwatch.ai/docs/ai-gateway/model-naming.md). | ## Boundaries * The gateway sends each attempt to one credential. Requests are not spread across several credentials of the same provider. * Which failures trigger a fallback is fixed. A per-key trigger list does not exist. * A fallback to a different vendor sends the prompt to that vendor. Use a routing policy or provider access to keep a key inside one vendor. **Also check:** [Routing policies](https://langwatch.ai/docs/ai-gateway/governance/routing-policies.md), [Model naming](https://langwatch.ai/docs/ai-gateway/model-naming.md), [Troubleshooting](https://langwatch.ai/docs/ai-gateway/troubleshooting.md). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.