Providers

Every LLM provider the LangWatch AI Gateway can dispatch to.

The gateway delegates provider-specific dispatch to bifrost/core, which LangWatch embeds as a Go library. This page lists the providers the gateway can talk to today and the path to configuring each.

Configure a provider credential once, use from many VKs

Every provider credential lives in the LangWatch Model Providers table (under Settings → Model Providers). It’s the same surface that powers evaluators and the playground, no separate “gateway providers” store.

Gateway-only settings (rate limits, fallback priority, providerConfig overrides) live directly on the ModelProvider itself, on the Advanced (Gateway) tab of the editor. There is no separate binding wrapper. VKs reference ModelProviders by id; the order in the bound routing policy drives primary + fallback selection.

ModelProvider (raw credential + Advanced/Gateway settings)
  └── RoutingPolicy.modelProviderIds[] (ordered primary + fallback)
       └── VirtualKey.routingPolicyId (or default policy from the org)

This means: rotating the raw OpenAI key rotates it everywhere; changing a rate limit only affects gateway traffic (it’s on the Advanced tab, not the basic credential); changing a VK’s provider mix is one edit to the routing policy it references.

Supported providers (v1)

Provider	Routes supported	Auth methods	Caching passthrough
OpenAI	chat, responses, embeddings, images, audio, moderations	API key	auto-prefix
Anthropic	messages, chat (translated), streaming	API key	cache_control blocks
Azure OpenAI	chat, responses, embeddings, images, audio	API key, AAD	auto-prefix
AWS Bedrock	messages, chat, embeddings (Titan)	AWS SigV4, IRSA	cachePoint
Google Vertex AI	messages, chat	GCP ADC, SA JSON	implicit context cache
Google Gemini	chat	API key	implicit context cache
Custom OpenAI-compatible	chat, embeddings (depends on upstream)	Bearer, custom header	opaque

Behind the scenes bifrost/core also supports Groq, Cohere, Mistral, Ollama, vLLM, SGLang, Perplexity, ElevenLabs, and others. Each appears in the UI once LangWatch ships its configuration form.

Picking a primary + fallback

Pick one or two fallback providers. Guidelines:

• Anthropic-first with Bedrock-Anthropic fallback. Same models on both sides; Bedrock is a warm backup during Anthropic direct outages.
• OpenAI-first with Anthropic fallback. Different model families but Claude Haiku can serve gpt-5-mini traffic acceptably for coding tasks.
• Azure-first with OpenAI direct fallback. Keeps traffic inside Azure data-residency until a configured fallback trigger (by default 5xx, timeout, rate_limit) moves it to OpenAI direct.

See Fallback Chains for trigger semantics.

Multi-region, multi-account

Each ModelProvider row pins a region (Azure endpoint, Bedrock region, Vertex project). If an organisation spans multiple regions, create sibling ModelProvider rows of the same family with a naming convention (e.g. openai-eu, openai-us) and use VK model_aliases to route gpt-5-mini-eu vs gpt-5-mini-us to the right ModelProvider id.