Migrate from LiteLLM

Map LiteLLM proxy keys, model lists, budgets, end-user ids and fallbacks to their LangWatch AI Gateway equivalents and move an application over with a base URL change.

What maps to what

LiteLLM proxy	LangWatch AI Gateway
Provider keys in config.yaml or the environment	A row per provider under Settings > Model Providers. See Providers.
A virtual key from /key/generate (sk-...)	A virtual key (vk-lw-...), sent as Authorization: Bearer.
max_budget and budget_duration on a key	A budget on the key: a limit in USD per day, week or month, block or warn. See Budgets.
team_id and team budgets	A key with Ownership on the team, and a budget on the team.
models on a key	Provider access on the key: the providers and models it may call.
rpm_limit, tpm_limit on a key	Rate limits on the key: requests per minute and per day. Tokens per minute are not enforced.
model_list entries: model_name to litellm_params.model	The provider/model name on the request, or a model alias on the routing policy when the public name differs. See Model naming and Model aliases.
fallbacks in router_settings	A routing policy: the provider rows a key tries, in order.
x-litellm-end-user-id header, user body field	Both accepted as is. X-LangWatch-End-User-Id is the native header.
metadata body field	X-LangWatch-Metadata, a request header with a JSON object, copied onto the spend event. The gateway does not read a metadata body field.
Spend logs, /spend/logs	A spend event per request, by webhook or over REST, and a trace per request in Trace Explorer.

Several model_list entries under one model_name, which LiteLLM balances across, have no equivalent: the gateway tries the rows of a routing policy in order.

1. Move the credentials

Add a row under Settings > Model Providers for each provider in your LiteLLM config. The LiteLLM model spellings carry over: azure/<deployment> and bedrock/<model id> are the same, and vertex_ai/gemini-2.5-flash is accepted as an alias of vertex/gemini-2.5-flash.

2. Create the keys and budgets

For each LiteLLM key, create a virtual key with the same budget and model list. From the CLI:

langwatch vk create \
  --name acme-support-agent \
  --scope team:<team_id> --trace-project <project_id> \
  --budget-limit 100 --budget-window month --budget-breach block

A LiteLLM budget on a team becomes a budget with Target set to the team, under AI Gateway > Budgets. A per-end-user cap becomes an attributed_user budget. See Budgets.

3. Turn fallbacks into a routing policy

# LiteLLM
router_settings:
  fallbacks:
    - gpt-5-mini: ["azure-mini-eu"]

Create a routing policy under AI Gateway > Routing policies with the OpenAI row first and the Azure row second, map gpt-5-mini to azure/acme-mini-eu in its model aliases, and pick the policy under Routing on the key.

4. Change the client

Before

client = OpenAI(
    base_url="http://litellm.acme.internal:4000",
    api_key="sk-...",
)
client.chat.completions.create(
    model="gpt-5-mini",
    messages=[...],
    user="cust_8841",
    extra_body={"metadata": {"plan": "pro"}},
)

An application that sends x-litellm-end-user-id keeps working without a change. See Python and TypeScript.

5. Verify

Send one request and check the response for X-LangWatch-Gateway-Request-Id. Open Trace Explorer in the key’s project for the trace and AI Gateway > Usage for the spend.

Keep a LiteLLM proxy behind the gateway

To move in stages, add the LiteLLM proxy as a custom OpenAI-compatible provider with its URL as the base URL and a LiteLLM key as the API key. Keys created in LangWatch then reach it as custom/<model_name>, with LangWatch budgets, traces and spend events in front of it. Move providers to their own rows one at a time.

On LangWatch Cloud that base URL must use https and must not resolve to a private address. A self-hosted gateway accepts http while REQUIRE_HTTPS_CUSTOM_ENDPOINTS stays false.