DNS and TLS

Point a public hostname at the LangWatch AI Gateway and terminate TLS in front of it.

Give the gateway its own hostname, separate from the app. LLM clients and coding agents send requests to the gateway hostname; browsers use the app hostname. This guide covers the chart Ingress on ingress-nginx, a cloud load balancer in front of the Service, and the checks that prove the endpoint works. For Envoy Gateway and other Gateway API implementations, see Routing and ingress controllers.

Prerequisites

• A DNS zone you control, for example acme.com.
• An ingress controller, or the AWS Load Balancer Controller for the NLB option below.
• cert-manager with a ClusterIssuer, or a certificate you provide yourself.

1. Set the ingress values

   The gateway is published from the app Ingress settings. Set ingress.gateway.host next to the app hosts:

   ingress:
     enabled: true
     className: nginx
     annotations:
       cert-manager.io/cluster-issuer: letsencrypt-prod
     hosts:
       - host: langwatch.acme.com
         http:
           paths:
             - path: /
               pathType: Prefix
     tls:
       - secretName: langwatch-tls
         hosts:
           - langwatch.acme.com
     gateway:
       host: gateway.acme.com
       tls:
         secretName: gateway-tls
   

   Replace both hosts. The chart renders a second Ingress, <release>-gateway-ingress, with the same className and annotations. It routes /v1 and /health on the gateway host to the gateway Service. The control plane derives the public URL CLI users get from ingress.gateway.host.

   With className: nginx the chart adds annotations to the gateway Ingress only. They keep streamed responses unbuffered, let a stream run for an hour, and raise the body limit to 32 MB. Keys in ingress.gateway.annotations override them.

2. Issue the certificate

   With the cert-manager.io/cluster-issuer annotation, cert-manager requests a certificate for ingress.host and stores it in ingress.tls.secretName. A DNS-01 issuer works before the DNS record exists:

   apiVersion: cert-manager.io/v1
   kind: ClusterIssuer
   metadata:
     name: letsencrypt-prod
   spec:
     acme:
       server: https://acme-v02.api.letsencrypt.org/directory
       privateKeySecretRef:
         name: letsencrypt-prod
       solvers:
         - dns01:
             cloudflare:
               apiTokenSecretRef:
                 name: cloudflare-api-token
                 key: api-token
   

   To bring your own certificate, create the TLS Secret yourself and drop the cert-manager.io/cluster-issuer annotation:

   kubectl -n langwatch create secret tls gateway-tls --cert=fullchain.pem --key=privkey.pem
   
3. Create the DNS record

   Point a CNAME at the ingress controller’s load balancer hostname:

   kubectl -n ingress-nginx get svc ingress-nginx-controller \
     -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
   

   Type:   CNAME
   Name:   gateway
   Target: <load balancer hostname>
   

   Use a CNAME, never an A record: cloud load balancer addresses change.

4. Verify

   dig +short gateway.acme.com
   openssl s_client -connect gateway.acme.com:443 -servername gateway.acme.com </dev/null 2>&1 \
     | grep -E 'subject=|issuer='
   curl -sS https://gateway.acme.com/health
   curl -sS https://gateway.acme.com/v1/models -H "Authorization: Bearer $LANGWATCH_VIRTUAL_KEY"
   

   The /health call returns {"status":"ok","checks":{"gateway":"ok","control_plane":"ok"}}. The /v1/models call returns 200 with the models your key can route to. Continue with the post-install checklist.

What the ingress publishes

Path	Match	Routes
/v1	Prefix	/v1/chat/completions, /v1/messages, /v1/responses, /v1/embeddings, /v1/audio/*, /v1/images/*, /v1/models, the realtime and ElevenLabs routes
/health	Exact	The public status endpoint for uptime monitors

/healthz, /readyz, /startupz, /debug/control-plane and /internal/* stay in-cluster. Metrics are on their own port (9464). The Gemini-native routes under /v1beta are outside the /v1 prefix and are not published by the chart Ingress. To publish them, route /v1beta to the gateway Service with your own route, as described in Routing and ingress controllers.

Cloud load balancer in front of the Service

To terminate TLS on a cloud load balancer instead of the ingress, leave ingress.gateway.host empty and turn the Service into a LoadBalancer. On AWS with the Load Balancer Controller and an ACM certificate:

gateway:
  publicUrl: https://gateway.acme.com   # no ingress host to derive it from
  service:
    type: LoadBalancer
    port: 443
    targetPort: 5563
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:<region>:<account>:certificate/<id>
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp

Both examples terminate TLS in front of the gateway and reach port 5563 over plain HTTP, so the Authorization header that carries the virtual key crosses that last hop without encryption.

The ingress terminates at the ingress controller and the network load balancer terminates at the load balancer, which puts the plaintext hop inside the cluster network in the first case and between the load balancer and the pods in the second. Both assume that network is trusted. Where it is not, run a service mesh with mutual TLS between the terminator and the gateway pods.

Request the certificate first and publish the validation CNAME records ACM prints:

aws acm request-certificate --domain-name gateway.acme.com --validation-method DNS --region <region>
aws acm describe-certificate --certificate-arn <arn> --region <region> \
  --query 'Certificate.DomainValidationOptions[].ResourceRecord'

Then read the load balancer hostname and point the CNAME at it:

kubectl -n langwatch get svc langwatch-gateway \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

A Service of type LoadBalancer publishes every route, including the probes and /debug/control-plane, which returns the resolved LW_GATEWAY_BASE_URL and takes no key. Put a load balancer listener rule or a security group in front of it when the address is public. Set service.internal.enabled: true to also render a ClusterIP Service named langwatch-gateway-internal for in-cluster callers, so they do not go out through the load balancer.

Other ingress controllers

Set ingress.className for your controller. The ingress-nginx annotations are added only when className is nginx. Put the controller’s equivalents in ingress.gateway.annotations so they apply to the gateway host only.

Three settings matter for every controller: response buffering off, an idle or read timeout above the longest stream you allow, and a body limit of at least 32 MB. The gateway lets a provider call run up to 14 minutes. Routing and ingress controllers has the full list and an Envoy Gateway HTTPRoute.

AWS ALB example. The certificate comes from ACM rather than ingress.gateway.tls.secretName, so gateway.publicUrl states the https URL the CLI should use:

ingress:
  enabled: true
  className: alb
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:<region>:<account>:certificate/<id>
  gateway:
    host: gateway.acme.com
    annotations:
      alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600
gateway:
  publicUrl: https://gateway.acme.com

Outbound endpoint policy

TLS in front of the gateway is one boundary. The other is the provider endpoints that customers configure on their own credentials. Three variables, set through gateway.security.* in the chart, control them:

Chart value	Env var	Effect
security.requireHTTPSCustomEndpoints	REQUIRE_HTTPS_CUSTOM_ENDPOINTS	Refuse a provider base URL with an http:// scheme
security.blockLocalHTTPCalls	BLOCK_LOCAL_HTTP_CALLS	Refuse a provider base URL that resolves to a private, loopback or other non-public address
security.allowedProxyHosts	ALLOWED_PROXY_HOSTS	Hostnames that may resolve to a private address while the block is on

Both switches default to false, so a self-hosted install can route to a model server on the cluster network. Cloud metadata addresses, 0.0.0.0, :: and link-local ranges are refused with every setting. Setting ENVIRONMENT to anything other than local requires both switches to be true.

Common failures

Symptom	Cause	Fix
curl reports an unknown certificate	cert-manager has not issued gateway-tls yet	kubectl -n langwatch describe certificate gateway-tls shows the pending challenge
Streamed responses arrive all at once	Buffering is on at the proxy	With ingress.className: nginx the chart adds proxy-buffering: "off" to the gateway Ingress; on another controller, set the equivalent in ingress.gateway.annotations
A long stream stops at 60 seconds	The load balancer idle timeout is shorter than the stream	Raise the controller’s idle or read timeout
CLI users get http://localhost:5563 as the gateway URL	The control plane has no LW_GATEWAY_PUBLIC_URL	Set ingress.gateway.host, or gateway.publicUrl
A provider credential with a private base URL fails to dispatch	BLOCK_LOCAL_HTTP_CALLS is on	Add the hostname to ALLOWED_PROXY_HOSTS

More symptoms are on Troubleshooting.