> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# Custom OpenAI-compatible

> Add any OpenAI-compatible endpoint, for example vLLM, Ollama or an internal inference server, under Settings, Model Providers and call it through the LangWatch AI Gateway.

## Add the credential

Open **Settings > Model Providers**, click **Add Model Provider** and select **Custom (OpenAI-compatible)**. Give the row a name, pick the organization, team or project it belongs to, and fill in:

| Field | Value |
| - | - |
| `CUSTOM_BASE_URL` | The endpoint, for example `https://inference.acme.internal/v1`. A trailing `/v1` is accepted. |
| `CUSTOM_API_KEY` | Optional. Sent as `Authorization: Bearer` when set. |

Save the row. Every virtual key whose scope can see the row can now dispatch to it, unless the key restricts its providers. See [Virtual keys](https://langwatch.ai/docs/ai-gateway/virtual-keys.md#create-a-virtual-key).

![The Custom OpenAI-compatible model provider drawer with CUSTOM\_API\_KEY, CUSTOM\_BASE\_URL, an Add Header button and the routing handle custom](https://mintcdn.com/langwatch/6EHh-4eZyYKZhosJ/images/ai-gateway/model-provider-custom-form.png?fit=max\&auto=format\&n=6EHh-4eZyYKZhosJ\&q=85\&s=1384249425c0917c7078464244807073)

The Custom (OpenAI-compatible) row: the optional CUSTOM\_API\_KEY, the required CUSTOM\_BASE\_URL, Add Header and the routing handle custom.

Add the models the endpoint serves under **Custom Models** on the row. `GET /v1/models` lists those names.

The endpoint must be reachable from the gateway. The gateway always refuses link-local and cloud metadata addresses, and refuses private and loopback addresses when it runs with `BLOCK_LOCAL_HTTP_CALLS=true`.

On LangWatch Cloud the endpoint must use `https`: a hosted gateway refuses to start without `REQUIRE_HTTPS_CUSTOM_ENDPOINTS=true`, and with it an `http` endpoint is refused. A self-hosted gateway accepts `http` while that setting stays `false`.

> [!WARNING]
>
> An `http` endpoint sends `CUSTOM_API_KEY` in cleartext, because the gateway puts it in an
> `Authorization: Bearer` header on every request. Anyone on the path between the gateway and the
> endpoint can read the key. Use `http` only for an endpoint with no `CUSTOM_API_KEY`. On a
> self-hosted gateway that dispatches to a row with a key, set `REQUIRE_HTTPS_CUSTOM_ENDPOINTS=true`
> as well.

## Model names

Write `custom/<model>`, for example `custom/qwen3-14b`. Give the row a **Routing handle** when you have several custom endpoints: `acme-gpu/qwen3-14b` pins that row.

## Send a request

```bash theme={null}
curl https://gateway.langwatch.ai/v1/chat/completions \
  -H "Authorization: Bearer $LANGWATCH_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "custom/qwen3-14b",
    "messages": [{"role": "user", "content": "Say hi in one word"}]
  }'
```

## Endpoints

| Endpoint | Served |
| - | - |
| `POST /v1/chat/completions` | Yes |
| `POST /v1/messages` | Yes, translated |
| `POST /v1/embeddings` | Yes, when the endpoint serves it |
| `POST /v1/images/generations`, `POST /v1/images/edits` | No, rejected with HTTP 400. Image traffic needs a direct OpenAI, Azure OpenAI, Gemini, Vertex AI or Bedrock credential. See [Images](https://langwatch.ai/docs/ai-gateway/api/images.md) |
| `GET /v1/models` | Yes, the names under Custom Models |

Streaming, tools and other features depend on what the endpoint implements. A feature the endpoint does not support comes back as the endpoint's own error.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.