> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# CI smoke test

> Create a short-lived virtual key, send a real completion through the LangWatch AI Gateway, check the response, then revoke the key.

The job below needs only the `langwatch` CLI, `curl` and `jq`, and it exercises the management API, the data plane and one provider in a single run.

Run it on a schedule so a broken credential or an expired secret shows up before a caller finds it.

> [!NOTE]
>
> Runtime: typically 3 to 8 seconds wall-clock. One gateway request + two REST round-trips.

## What it checks

* The management API at `/api/gateway/v1/*` answers with your CI token.
* A virtual key is created at the scope you name and its secret is returned once.
* The gateway resolves that secret and dispatches to a provider.
* The response carries `X-LangWatch-Gateway-Request-Id`, so the request is traceable.
* The key is revoked when the job ends, whether it passed or failed.

## Prerequisites

* An API key for the calling project, in `CI_LANGWATCH_TOKEN`. It needs the `virtualKeys:create` and `virtualKeys:delete` permissions.
* The scope the CI key lives at, in `CI_SCOPE`, written as `type:id`. The types are `org`, `team` and `project`, for example `org:acme`.
* At least one model provider credential visible from that scope, configured under **Settings → Model Providers**.
* `jq` on the runner.

## The script

```bash theme={null}
#!/usr/bin/env bash
# ci-smoke-test.sh
set -euo pipefail

export LANGWATCH_API_KEY="$CI_LANGWATCH_TOKEN"
GATEWAY_URL="${LANGWATCH_GATEWAY_URL:-https://gateway.langwatch.ai}"
MODEL="${SMOKE_MODEL:-openai/gpt-6-luna}"
STAMP=$(date -u +%Y%m%dT%H%M%S)

cleanup() {
  if [ -n "${VK_ID:-}" ]; then
    echo "Revoking $VK_ID"
    langwatch virtual-keys revoke "$VK_ID" --format json >/dev/null || true
  fi
}
trap cleanup EXIT

# 1. Create the key.
echo "Creating a virtual key at $CI_SCOPE"
CREATE=$(langwatch virtual-keys create \
  --name "ci-smoke-$STAMP" \
  --scope "$CI_SCOPE" \
  --format json)

VK_ID=$(echo "$CREATE" | jq -r '.virtual_key.id')
VK_SECRET=$(echo "$CREATE" | jq -r '.secret')

[ -n "$VK_ID" ] && [ "$VK_ID" != "null" ] || { echo "FAIL: no virtual key id"; exit 1; }
[ -n "$VK_SECRET" ] && [ "$VK_SECRET" != "null" ] || { echo "FAIL: no secret"; exit 1; }

# 2. Report what the key enumerates. Discovery can be empty or partial while
# dispatch still works, so this is a diagnostic, not a gate. Step 3 is the check.
MODELS=$(curl -sS --fail-with-body --max-time 15 \
  -H "Authorization: Bearer $VK_SECRET" \
  "$GATEWAY_URL/v1/models" | jq -r '.data | length')

echo "Models the key enumerates: $MODELS"

# 3. Send a completion.
BODY=$(mktemp)
HEADERS=$(mktemp)
curl -sS --fail-with-body --max-time 60 \
  -H "Authorization: Bearer $VK_SECRET" \
  -H "Content-Type: application/json" \
  -D "$HEADERS" -o "$BODY" \
  -X POST "$GATEWAY_URL/v1/chat/completions" \
  -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}],\"max_completion_tokens\":16,\"reasoning_effort\":\"none\"}"

# 4. Check the response.
REQUEST_ID=$(grep -i '^x-langwatch-gateway-request-id:' "$HEADERS" \
  | awk '{print $2}' | tr -d '\r')
[ -n "$REQUEST_ID" ] || { echo "FAIL: no X-LangWatch-Gateway-Request-Id"; exit 1; }

CONTENT=$(jq -r '.choices[0].message.content' "$BODY")
[ -n "$CONTENT" ] && [ "$CONTENT" != "null" ] || { echo "FAIL: empty content"; exit 1; }

echo "OK: request_id=$REQUEST_ID"
```

Set `SMOKE_MODEL` to a model your own providers serve. Write it as `provider/model`, for example `anthropic/claude-sonnet-4-5` or `azure/<your deployment name>`. See [Model naming](https://langwatch.ai/docs/ai-gateway/model-naming.md).

## GitHub Actions

```yaml theme={null}
# .github/workflows/gateway-smoke.yml
name: Gateway smoke test
on:
  schedule:
    - cron: "*/15 * * * *"
  workflow_dispatch:

jobs:
  smoke:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: npm i -g langwatch
      - run: bash ci-smoke-test.sh
        env:
          CI_LANGWATCH_TOKEN: ${{ secrets.CI_LANGWATCH_TOKEN }}
          CI_SCOPE: ${{ vars.CI_SCOPE }}
          LANGWATCH_GATEWAY_URL: https://gateway.langwatch.ai
```

Wall-clock time is one provider completion plus three short API calls, so it is dominated by the model you pick. Give the job a timeout rather than an expected duration.

## Reading a failure

| Failure | Cause | Where to look |
| - | - | - |
| `FAIL: no virtual key id` | The token lacks `virtualKeys:create`, or the scope id is wrong | Run `langwatch virtual-keys list` with the same token |
| `curl` exits 7 | The gateway hostname does not resolve or does not answer | DNS and the load balancer. See [DNS and TLS](https://langwatch.ai/docs/ai-gateway/self-hosting/dns-and-tls.md) |
| `401 invalid_api_key` | The gateway could not resolve the secret | [Troubleshooting](https://langwatch.ai/docs/ai-gateway/troubleshooting.md#401-on-every-request) |
| `FAIL: the key can route to no models` | No provider credential is visible from `CI_SCOPE` | Add a credential at that scope under **Settings → Model Providers** |
| `400 model_not_recognized` | `SMOKE_MODEL` is not a model any visible credential serves | [Troubleshooting](https://langwatch.ai/docs/ai-gateway/troubleshooting.md#400-model_not_recognized) |
| `502 provider_error` | The gateway is healthy and the provider is not | The provider's status page, and the body `curl` printed |
| `FAIL: empty content` | The provider answered with no message content | Read `$BODY` in the job log |

## Against a self-hosted install

```bash theme={null}
export LANGWATCH_ENDPOINT="https://langwatch.acme.com"
export LANGWATCH_GATEWAY_URL="https://gateway.acme.com"
bash ci-smoke-test.sh
```

The two hostnames can be the same if your load balancer routes `/api/*` to the control plane and `/v1/*` to the gateway.

## Alerting

Fire your own alert from the job's failure step:

```yaml theme={null}
- name: Alert on failure
  if: failure()
  run: |
    curl -X POST https://events.pagerduty.com/v2/enqueue \
      -d '{"routing_key":"${{ secrets.PD_KEY }}","event_action":"trigger","payload":{"summary":"Gateway smoke test failed","severity":"error","source":"github-actions"}}'
```

Run this more often than you read `/readyz`. The probe reports process state only, while this exercises authentication, dispatch, the provider and the spend record.

## See also

* [Health checks](https://langwatch.ai/docs/ai-gateway/self-hosting/health-checks.md): what each probe reports, and the synthetic request pattern.
* [Observability](https://langwatch.ai/docs/ai-gateway/observability.md): what each response header carries.
* [Troubleshooting](https://langwatch.ai/docs/ai-gateway/troubleshooting.md): the symptom sections this table points at.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.