Routing policies

An admin decides once which providers and models a key can reach. Virtual keys point at that decision instead of carrying their own copy.

A routing policy is one decision, written down. It says which providers serve a request, and in which order. It says which model names a client can send.

An admin writes the policy one time. Each virtual key points to the policy. To change the routing of many keys, change the policy. Do not change each key.

Shape

RoutingPolicy {
  id:               string
  organizationId:   string
  name:             string
  description?:     string

  // The providers this policy routes through, in order. The gateway
  // tries the first provider. If the request fails in a way that a
  // different provider can answer, the gateway tries the next one.
  modelProviderIds: string[]

  // Model name mapping. The key is the name the client sends. The
  // value is the model that serves the request. The three tier names
  // are entries in this map.
  modelAliases:     Record<string, string>

  // The model that serves a tier when the map has no entry for it.
  // Applies only to the three tier names.
  defaultModel?:    string

  // What a request can reach. Four groups: tools, mcp, urls, models.
  // Each group has a deny list and an allow list.
  policyRules:      { tools: {...}, mcp: {...}, urls: {...}, models: {...} }

  isDefault:        boolean
  scopes:           { scopeType: 'ORGANIZATION' | 'TEAM' | 'PROJECT', scopeId: string }[]
}

A policy can apply in more than one place. Each entry in scopes adds one organization, one team, or one project.

Model tiers

A client can send a tier name in the model field. The policy maps the tier name to a model. When you move to a newer model, you change the policy. The client code does not change.

The gateway knows three tier names:

Tier	Use
complex	The strongest model you have. Use it when quality is more important than cost.
reasoning	A model that thinks before it answers. Use it for plans, analysis, and difficult debugging.
fast	A small model for high volumes. Use it for classification, extraction, routing, and short replies.

The client sends the tier name where a model name goes:

{
  "model": "complex"
}

Set a model for each tier in the policy. A tier without a model uses defaultModel. A tier without a model and without defaultModel is not a known name. The gateway refuses a request that sends it.

defaultModel applies only to the three tier names. A model name with a typo in it still gets a refusal. The gateway does not put the default model in its place.

Write a full model id, for example openai/gpt-5-mini. Do not write a name that changes over time, for example openai/latest. The gateway reads the value as one literal model name, so it would call a model named latest. The app refuses to save such a name.

GET /v1/models lists each tier that has a model. The tier shows in a model picker with the other names. If you set defaultModel, the list has all three tiers. A tier without a model and without defaultModel is not in the list, because the gateway refuses it.

Model name mapping

modelAliases also maps usual model names. Use it to point an old name to its replacement. Use it to give a model the name your teams already use.

{
  "gpt-4o": "openai/gpt-5-mini",
  "coding": "anthropic/claude-haiku-4-5"
}

A mapping cannot reach a model that the key is not permitted to use. The gateway examines the model that the mapping points to, not the name that the client sent.

Restrictions

policyRules has four groups of patterns. Each group has a deny list and an allow list.

• tools: the name of each tool in the request.
• mcp: the name and the address of each MCP server in the request.
• urls: each address in the request. This includes messages, system prompts, and tool arguments.
• models: the model that the request resolves to.

The gateway refuses a request that matches a deny pattern. The refusal comes before the request costs money. If a group has one or more allow patterns, a request must match one of them. Deny wins over allow.

The gateway applies the models group after the name mapping. A mapping cannot go around it.

Provider order

modelProviderIds is an ordered list. The gateway tries the first provider. If the request fails in a way that a different provider can answer, the gateway tries the next one.

The gateway does not call a provider that cannot serve the model. A chain across two vendors only moves over for models that the two vendors serve. To give one model a backup, map its name to a model from each vendor, or point a tier to a model from each vendor.

The gateway also removes a provider that the key cannot reach. This occurs, for example, when an admin removes the provider from the key’s scope.

A model name that contains a provider, for example openai/gpt-5-mini, runs only on that provider. If the key cannot reach the provider, the gateway refuses the request. It does not send the request to a different vendor. A tier or a mapping that points to a removed provider gets the same refusal, and GET /v1/models stops listing it.

Defaults

One policy in each scope can be the default. When you make a policy the default, the app clears the flag on the other policies in the same scope. This occurs in one step. Two policies are never both the default for one scope.

A key routes through the policy that it names. The default policy is the one the app offers first when someone makes a key in that scope.

Manage in the app

Open AI Gateway → Routing Policies. The page lists your policies by where they apply. Select New policy, or open the actions menu on a policy and select Edit.

In the editor you can:

• Give the policy a name and a description.
• Select where it applies.
• Select the providers and put them in order.
• Select a model for each tier, and a default model.
• Map other model names.
• Add restrictions.

The editor has its own address. A link to a policy opens that policy.

Manage with tRPC

// List the policies in an organization.
trpc.routingPolicy.list.useQuery({ organizationId });

// List only the policies a scope can select.
trpc.routingPolicy.list.useQuery({
  organizationId,
  selectableForScope: { scopeType: "TEAM", scopeId: teamId },
});

// Read one policy.
trpc.routingPolicy.get.useQuery({ organizationId, id });

// Suggest models for a tier.
trpc.routingPolicy.tierSuggestions.useQuery({
  organizationId,
  tier: "complex",
  boundProviderTypes: ["openai", "anthropic"],
});

trpc.routingPolicy.create.useMutation();
trpc.routingPolicy.update.useMutation();
trpc.routingPolicy.setDefault.useMutation();
trpc.routingPolicy.delete.useMutation();

create, update, setDefault, and delete need the routingPolicies:manage permission. list, get, and tierSuggestions need routingPolicies:view.

Keys that name no policy

A key that names no policy uses its own routing mode. NONE sends the request to one provider and returns that provider’s answer. FALLBACK_ALL tries the providers that the key can reach, in a set order.

To move a key to a policy:

1. Make a policy with the providers that the key uses.
2. Set the key’s routing policy.
3. Send a request. Make sure that it reaches the provider you expect.

Point new keys to a policy. Then the routing decision stays in one place.