Kubernetes (Helm)

Deploy LangWatch on any Kubernetes cluster using the official Helm chart. The chart supports everything from single-node development to highly-available production with replicated ClickHouse.

Prerequisites

Kubernetes 1.28+
Helm 3.12+
kubectl configured for your cluster
A StorageClass that supports dynamic provisioning (for persistent volumes)
A domain name (for Ingress with TLS)
Default resource requirements: ~5 CPU and ~17 Gi RAM (requests). See Size Overlays for smaller or larger configurations.

Quick Start

Deploy LangWatch with all dependencies managed by the chart:

# Add the Helm repository
helm repo add langwatch https://langwatch.github.io/langwatch
helm repo update

# Install with auto-generated secrets (development only)
helm install langwatch langwatch/langwatch \
  --namespace langwatch --create-namespace \
  --set autogen.enabled=true \
  --wait --timeout 10m

Verify the installation:

kubectl -n langwatch get pods

Port-forward to access the UI:

kubectl -n langwatch port-forward svc/langwatch-app 5560:5560
# Open http://localhost:5560

autogen.enabled=true generates random secrets on the first install. This is fine for testing but not for production. helm upgrade reads the existing Secrets back and keeps every value.helm uninstall deletes the app Secret, so a reinstall generates new values: sessions end, stored model provider keys can no longer be decrypted, and API keys and virtual keys stop working (see Credentials secret). The chart-managed PostgreSQL also gets a new password while its retained volume keeps the old one. Under Argo CD it is worse: Argo renders without cluster access, so the secrets rotate on every sync. See Production Deployment below.

Operator access and feature flags

The Ops pages (/ops) are open only to the operators listed in ADMIN_EMAILS. No account has them on a new install. Add your email and upgrade:

app:
  extraEnvs:
    - { name: ADMIN_EMAILS, value: "you@yourcompany.com" }
workers:
  extraEnvs:
    - { name: ADMIN_EMAILS, value: "you@yourcompany.com" }

Experimental features stay off until an operator turns them on in Ops > Feature flags (/ops/feature-flags), for all projects or for chosen ones. Dashboard widgets need both release_dashboards and release_lwql_workbench. The Query API (POST /api/v1/query) works without either.

License

An Enterprise license goes in app.license.key, inline or from a Secret. It takes either an activation code (LW-XXXX-XXXX-XXXX-XXXX), which needs outbound HTTPS to connect.langwatch.ai, or a signed license key, which works offline:

app:
  license:
    key:
      secretKeyRef:
        name: langwatch-license
        key: license

See Editions and licensing for what happens at startup with each form.

Low-Resources Deployment

The default install requests ~5 CPU and ~17 Gi RAM. For smaller clusters or evaluation purposes, use the dev overlay which requests approximately ~2.8 CPU and ~6.9 Gi RAM:

helm pull langwatch/langwatch --untar

helm install langwatch ./langwatch \
  --namespace langwatch --create-namespace \
  --set autogen.enabled=true \
  -f langwatch/examples/overlays/size-dev.yaml \
  --wait --timeout 10m

The overlay comes from the chart you pulled, so it always matches the chart version you install. This configures smaller resource limits, single replicas, and disables evaluator preloading to reduce memory usage. Suitable for development, demos, and small teams.

Production Deployment

For production, you should:

Use external managed databases (PostgreSQL, Redis)
Provide Kubernetes Secrets from your secret manager
Expose via Ingress with TLS
Disable auto-generation

1. Provide the Secrets

With autogen.enabled: false the chart creates no Secrets; it reads them from Secrets you provide. Source them from your secret manager (External Secrets Operator, Sealed Secrets, Vault, Azure Key Vault, …) so values stay stable across syncs and can be rotated. Every value is an opaque random string, e.g. openssl rand -hex 32. Which keys you need depends on what you run: App Secret: referenced by secrets.existingSecret (default name langwatch-app-secrets). The app, workers, gateway and Langy pods all read it.

Key	Needed when
`credentialsEncryptionKey`	Always
`nextAuthSecret`	Always
`virtualKeyPepper`	Always
`LW_GATEWAY_INTERNAL_SECRET`, `LW_GATEWAY_JWT_SECRET`	`gateway.chartManaged: true` (default)
`LANGY_INTERNAL_SECRET`	`langyagent.chartManaged: true` (default)
`LWQL_CLICKHOUSE_PASSWORD`, `LWQL_POSTGRES_READER_PASSWORD`	Optional, see LangWatchQL below

Every key name can be changed to match your secret store, for example the dashed names Azure Key Vault produces through dataFrom: secrets.secretKeys.* for the app and LangWatchQL keys, gateway.secrets.internalSecretKey and gateway.secrets.jwtSecretKey for the gateway keys (the app and the gateway pod both read them), and langyagent.secrets.internalSecretKey for Langy. ClickHouse Secret: chart-managed ClickHouse only (skip it with clickhouse.chartManaged: false). Referenced by clickhouse.auth.existingSecret, and the name must differ from the default <release>-clickhouse, since the deployment composes CLICKHOUSE_URL from the password key at runtime.

Key	Needed when
`password`	Always
`clusterSecret`	`clickhouse.replicas > 1` (Keeper inter-node auth). Setting it upfront keeps a later move to replicated ClickHouse a one-line change

Redis and PostgreSQL Secrets: chart-managed Redis or PostgreSQL only. Set redis.auth.existingSecret and postgresql.auth.existingSecret to Secrets holding their passwords, with the key names in redis.auth.secretKeys.passwordKey and postgresql.auth.secretKeys.*. With autogen.enabled=false the render fails without them (or an inline redis.auth.password / postgresql.auth.password), because a generated password would change on every Argo CD sync. A missing required key leaves the pods that read it in CreateContainerConfigError, naming the key. When the Secret already exists without LANGY_INTERNAL_SECRET, helm install and helm upgrade stop at render time and name the key instead. Argo CD renders without cluster access, so under Argo CD only the pod error shows it.

autogen.enabled controls the app Secret (app, gateway and Langy keys), the LangWatchQL passwords and the chart-managed ClickHouse, PostgreSQL and Redis credentials. Use it with plain helm only: it reads existing values back from the cluster, which Argo CD cannot do, so under Argo CD the generated values change on every sync. Each database’s chartManaged setting decides whether the chart runs that database: to bring your own PostgreSQL, Redis or ClickHouse, set its chartManaged: false.

LangWatchQL is optional. The two LWQL_* values are passwords for dedicated read-only identities the app creates for user-written queries: a PostgreSQL role lwql_ro (read-only transactions, a statement timeout, SELECT on approved views only) and a matching ClickHouse identity. The app keeps writing through its own connection. Without these keys every pod still starts and LangWatchQL stays unavailable; set lwql.enabled: false to turn it off entirely.To add the keys later on chart-managed ClickHouse, delete the <release>-lwql-access-render-<hash> Job so the next sync or upgrade runs it again. Once it is Complete, restart the ClickHouse StatefulSet and the app and workers Deployments, which read the passwords at startup.With external PostgreSQL, the DATABASE_URL user needs CREATE ROLE, or at least ALTER ROLE on a pre-created lwql_ro, because the app re-converges the role on every boot. Without it LangWatchQL stays unavailable and the rest of LangWatch runs normally. See the chart README for the exact role statements.

Clusters that forbid root pods (Kyverno/Gatekeeper runAsNonRoot policies): the Langy assistant pod runs as UID 0 with a small set of added capabilities, which it needs to isolate each session under its own UID. If your policy blocks that, turn Langy off. Every other LangWatch pod already runs as non-root:

langyagent:
  chartManaged: false

This removes the pod and every reference to it, and LANGY_INTERNAL_SECRET is no longer needed. A sandboxed runtime (see Langy) hardens Langy where root pods are allowed; it does not satisfy a runAsNonRoot policy. For Pod Security Admission restricted, which also rejects the bundled Prometheus, apply the strict-admission overlay instead.

For external databases, create additional secrets:

# PostgreSQL (RDS, Cloud SQL, etc.)
kubectl create secret generic langwatch-db \
  --namespace langwatch \
  --from-literal=connectionString="postgresql://user:password@host:5432/langwatch"

# Redis (ElastiCache, Memorystore, etc.)
kubectl create secret generic langwatch-redis \
  --namespace langwatch \
  --from-literal=connectionString="redis://:password@host:6379"

2. Create a Values File

Start from the production example and customize. This configuration requests approximately ~8.4 CPU and ~32 Gi RAM across all pods:

# values-production.yaml

autogen:
  enabled: false

secrets:
  existingSecret: langwatch-app-secrets

app:
  replicaCount: 2
  http:
    baseHost: "https://langwatch.example.com"
    publicUrl: "https://langwatch.example.com"
  resources:
    requests: { cpu: 500m, memory: 4Gi }
    limits: { cpu: 1000m, memory: 4Gi }
  podDisruptionBudget:
    minAvailable: 1
  # More than one app or worker replica cannot share the default
  # local-filesystem volume, so the chart refuses to render until object
  # storage is on. Credentials come from app.dataplane.providers.awsS3.*,
  # or from IRSA on the ServiceAccount set in global.serviceAccount.
  dataplane:
    enabled: true
    provider: awsS3
    bucket: langwatch-objects

workers:
  enabled: true
  replicaCount: 2
  resources:
    requests: { cpu: 500m, memory: 4Gi }
    limits: { cpu: 1000m, memory: 4Gi }
  podDisruptionBudget:
    minAvailable: 1

# External PostgreSQL
postgresql:
  chartManaged: false
  external:
    connectionString:
      secretKeyRef:
        name: langwatch-db
        key: connectionString

# External Redis
redis:
  chartManaged: false
  external:
    connectionString:
      secretKeyRef:
        name: langwatch-redis
        key: connectionString

# Chart-managed ClickHouse (production sizing)
clickhouse:
  cpu: 4
  memory: "8Gi"
  storage:
    size: 100Gi
  auth:
    # Operator-owned Secret (created above). With autogen.enabled=false the
    # chart does not materialise a ClickHouse Secret; point at the one you
    # created and the deployment composes CLICKHOUSE_URL at runtime from
    # its password key.
    existingSecret: langwatch-clickhouse-creds

# Ingress with TLS, on ingress-nginx. For Envoy Gateway, Gateway API or
# another controller, see /self-hosting/deployment/routing.
#
# `/api/internal` is hard-blocked at the edge by default (ingress.blockedPaths)
# so the private control plane is never internet-reachable. Blocked requests
# answer 503 on ingress-nginx. See the Security guide for the opt-out, the
# AWS/GKE caveat, and what the block does not cover.
ingress:
  enabled: true
  className: nginx
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
  hosts:
    - host: langwatch.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
  tls:
    - secretName: langwatch-tls
      hosts:
        - langwatch.example.com
  # AI Gateway host. Renders a second Ingress with the settings above, plus
  # the nginx streaming settings the gateway needs.
  gateway:
    host: gateway.langwatch.example.com
    tls:
      secretName: langwatch-gateway-tls

# Prometheus monitoring
prometheus:
  chartManaged: true
  server:
    retention: 30d
    persistentVolume:
      size: 20Gi

3. Install

helm install langwatch langwatch/langwatch \
  --namespace langwatch \
  -f values-production.yaml \
  --wait --timeout 10m

4. Verify

# Check all pods are running
kubectl -n langwatch get pods

# Check ingress
kubectl -n langwatch get ingress

# Check logs
kubectl -n langwatch logs deploy/langwatch-app --tail=50

Then follow First run to create the first account, add the license and connect your identity provider.

High-Availability Deployment

For HA with replicated ClickHouse, multiple app/worker replicas, and PodDisruptionBudgets. This configuration requests approximately ~31 CPU and ~88 Gi RAM across all pods:

# values-ha.yaml (extends production values above)

app:
  replicaCount: 3
  podDisruptionBudget:
    minAvailable: 2

workers:
  replicaCount: 3
  podDisruptionBudget:
    minAvailable: 2

langwatch_nlp:
  replicaCount: 2
  podDisruptionBudget:
    minAvailable: 1

langevals:
  replicaCount: 2
  podDisruptionBudget:
    minAvailable: 1

# 3-node replicated ClickHouse with Keeper
clickhouse:
  replicas: 3
  cpu: 8
  memory: "16Gi"
  storage:
    size: 300Gi
    storageClass: gp3

  # Cold storage and backups
  objectStorage:
    bucket: "langwatch-data"
    region: "us-east-1"
    useEnvironmentCredentials: true
  cold:
    enabled: true
    defaultTtlDays: 49 # Recommend multiples of 7 to align with weekly partition boundaries
  backup:
    enabled: true

postgresql:
  chartManaged: false
  external:
    connectionString:
      secretKeyRef:
        name: langwatch-db
        key: connectionString

redis:
  chartManaged: false
  external:
    connectionString:
      secretKeyRef:
        name: langwatch-redis
        key: connectionString

helm install langwatch langwatch/langwatch \
  --namespace langwatch \
  -f values-ha.yaml \
  --wait --timeout 15m

Replicated ClickHouse requires an odd number of replicas (3, 5, 7) for Keeper consensus. Use 3 replicas unless you have a specific reason for more.

Overlay System

The chart ships with composable overlay files in examples/overlays/. Combine them to build your deployment configuration:

Size Overlays

Overlay	Use Case	Approx Resources (requests)
(default, no overlay)	Quick start, small production	~5 CPU, ~17 Gi
`size-minimal.yaml`	CI smoke tests, tiny demos	~2.6 CPU, ~5.2 Gi
`size-dev.yaml`	Local dev, small teams	~2.8 CPU, ~6.9 Gi
`size-prod.yaml`	Production, single-node CH	~8.9 CPU, ~23 Gi
`size-ha.yaml`	HA production, replicated CH	~23 CPU, ~85 Gi

size-prod.yaml and size-ha.yaml run more than one replica, so they need app.dataplane (S3 or Azure Blob) for stored objects. cold-storage-s3.yaml covers ClickHouse only.

Access Overlays

Overlay	Description
`access-nodeport.yaml`	NodePort on 30560 (Kind, bare-metal)
`access-ingress.yaml`	Nginx Ingress with TLS template

Infrastructure Overlays

Overlay	Description
`postgres-external.yaml`	External PostgreSQL (RDS, Cloud SQL)
`redis-external.yaml`	External Redis (ElastiCache, Memorystore)
`clickhouse-external.yaml`	External ClickHouse instance
`clickhouse-replicated.yaml`	3-node replicated ClickHouse
`cold-storage-s3.yaml`	S3 cold storage + backups
`local-images.yaml`	Local images with `pullPolicy: Never`
`langy-disabled.yaml`	Runs without the Langy assistant pod
`langy-gvisor.yaml`	Runs Langy under the gVisor sandbox runtime
`strict-admission.yaml`	Pod Security Admission `restricted` clusters

Composing Overlays

Overlays are composable, later files override earlier ones:

# Production with external DBs and S3 cold storage
helm install langwatch langwatch/langwatch \
  -f examples/overlays/size-prod.yaml \
  -f examples/overlays/access-ingress.yaml \
  -f examples/overlays/postgres-external.yaml \
  -f examples/overlays/redis-external.yaml \
  -f examples/overlays/cold-storage-s3.yaml \
  --set clickhouse.objectStorage.bucket=my-langwatch-data \
  --set clickhouse.objectStorage.region=us-east-1 \
  --set autogen.enabled=true

ClickHouse Configuration

Standalone vs Replicated

Mode	Replicas	Engine	When to Use
Standalone	1	MergeTree	Development, small production
Replicated	3+ (odd)	ReplicatedMergeTree + Keeper	HA production

Switch to replicated mode:

clickhouse:
  replicas: 3 # Automatically uses ReplicatedMergeTree + Keeper

External ClickHouse

To use an existing ClickHouse instance:

clickhouse:
  chartManaged: false
  external:
    url:
      value: "http://user:password@clickhouse-host:8123/langwatch"
    # For replicated instances:
    cluster: "my_cluster"

The chart-managed ClickHouse is pinned to a version the release is tested against. An external instance is yours, so the version is your choice. LangWatch supports ClickHouse 25.8 and later, including 26.x and ClickHouse Cloud. The upgrade applies the release’s ClickHouse migrations before the app starts, so the version has to accept every migration in the release you deploy.

Auto-Tuning

The clickhouse-serverless subchart automatically tunes ClickHouse parameters based on the CPU and memory you allocate:

clickhouse:
  cpu: 4 # Tunes thread pools, merge concurrency
  memory: "8Gi" # Tunes memory limits, cache sizes, per-query limits

You only need to set these two values, the subchart computes optimal settings for query limits, merge threads, insert batching, and S3 download parallelism.

AI Gateway sub-chart

The umbrella chart bundles the AI Gateway as a sub-chart that runs alongside the core LangWatch app, and installs it by default. It gives you virtual keys, hierarchical budgets, multi-provider routing via Bifrost, guardrails, and prompt caching, all governed by the same control plane. To run without it:

gateway:
  chartManaged: false

The defaults are a HorizontalPodAutoscaler running 2 to 10 replicas, a ClusterIP service and no Ingress. To publish the gateway, set ingress.gateway.host on the app Ingress, or route to the <release>-gateway Service yourself and set gateway.publicUrl. Routing and ingress controllers lists the paths, timeouts and body sizes for ingress-nginx, Envoy Gateway and other controllers. For per-environment tuning (replicas, autoscaling, ingress hostname + TLS, image registry mirror, secrets injection) see AI Gateway → Self-hosting → Helm. Three things to know when running it:

One Secret holds everything (or use autogen.enabled=true and let the chart materialise it). Both langwatch-app and the AI Gateway pod mount LW_GATEWAY_INTERNAL_SECRET and LW_GATEWAY_JWT_SECRET from the same secrets.existingSecret Secret (default langwatch-app-secrets), which also holds the app keys credentialsEncryptionKey, nextAuthSecret and virtualKeyPepper. The Production Deployment section above lists the keys the Secret needs. With autogen.enabled=false and a required key missing, the pods that read it stay in CreateContainerConfigError naming the key (under plain helm, a missing LANGY_INTERNAL_SECRET in an existing Secret stops the render instead); under autogen.enabled=true the chart materialises the Secret on first install via lookup-or-rand.
A public route needs DNS + TLS. The gateway is what your LLM clients hit, so it usually wants its own hostname (e.g. gateway.your-corp.com), separate cert, separate route. See AI Gateway → Self-hosting → DNS & TLS.
Worker pods must be running. Budget enforcement reads from a ClickHouse rollup that trace-processing folds into. The workers also run the upgrade, so the chart refuses workers.enabled=false.

Upgrade

helm repo update
helm upgrade langwatch langwatch/langwatch \
  --namespace langwatch \
  -f values-production.yaml \
  --wait --timeout 45m

The new worker pods run pnpm task upgrade on start, and the new api pods are not ready until it completes, so the previous version keeps serving meanwhile and if it fails. When upgrades are serialised (serializeUpgrades on and no app.dataplane), a pre-upgrade Job runs it on the new image before any pod rolls instead, and a failure fails the helm upgrade. On a first install the worker runs it once on its first boot. Follow it on Ops > Upgrades or with pnpm task upgrade status. With serialised upgrades, to run the upgrade from your own pipeline instead, set app.migrations.preRoll: false and run it before helm upgrade; until then the new api pods are not ready. See Upgrade Guide for version-specific instructions.

Uninstall

helm uninstall langwatch --namespace langwatch

This does not delete the PostgreSQL, ClickHouse and Redis PersistentVolumeClaims, so that data is preserved. The stored-objects claim (<release>-stored-objects, rendered by default and whenever app.dataplane is off) is part of the release and is deleted. With the Delete reclaim policy, the default for most dynamically provisioned storage classes, the uploaded files and datasets on it are deleted too. With Retain, the PersistentVolume and its data stay behind and need manual cleanup. Back it up first.With autogen.enabled=true, the generated app Secret and the PostgreSQL and Redis password Secrets are deleted too; see the warning under Quick Start. Delete the remaining claims manually if you want a clean removal:

kubectl -n langwatch delete pvc --all

FAQ

Istio, Service Mesh

If you’re using Istio or another service mesh with automatic sidecar injection, Job pods may never finish because the sidecar keeps the pod alive after the main container exits. The chart ships no CronJobs of its own (cronjobs.jobs is empty by default). If you add jobs there, disable sidecar injection for them:

cronjobs:
  pod:
    annotations:
      sidecar.istio.io/inject: "false"

The chart-managed ClickHouse backup CronJobs (when clickhouse.backup.enabled is on) and the chart’s own setup Jobs (the LangWatchQL render and the stored-objects upgrade hooks) take no pod annotations, so the setting above does not reach them. For those, enable native sidecars (Kubernetes 1.29 or later, Istio ENABLE_NATIVE_SIDECARS=true), which stop when the Job’s container exits, or exclude the pods in your mesh’s injection configuration.

Two installs in one cluster

The bundled Prometheus creates a cluster-scoped ClusterRole named <release>-prometheus-server. A second install with the same release name in another namespace, for example a staging langwatch beside a production langwatch, fails with ClusterRole "langwatch-prometheus-server" ... exists and cannot be imported into the current release. Give the second install its own ClusterRole name, or use a different release name:

prometheus:
  server:
    clusterRoleNameOverride: langwatch-staging-prometheus-server

If both installs enable an Ingress, give each its own ingress.hosts and ingress.gateway.host. Overlapping host and path rules conflict in the ingress controller.

Custom StorageClass

Set a StorageClass for all persistent volumes:

clickhouse:
  storage:
    storageClass: "gp3"
postgresql:
  primary:
    persistence:
      storageClass: "gp3"
redis:
  master:
    persistence:
      storageClass: "gp3"

Air-Gapped Environments

For clusters without internet access:

Push LangWatch images to your private registry
Update images.app.repository, images.langwatch_nlp.repository, images.langevals.repository, plus the bundled services listed in Docker Images
If your registry requires authentication, attach the pull secret to the ServiceAccount the pods use. The chart has no chart-wide imagePullSecrets value; see Private Registries
License with a signed license key in app.license.key, not an activation code: redeeming a code needs outbound access to connect.langwatch.ai

Overview

Configuration

Deployment

Infrastructure

Operations

Ops Console

Langy

Prerequisites

Quick Start

Operator access and feature flags

License

Low-Resources Deployment

Production Deployment

1. Provide the Secrets

2. Create a Values File

3. Install

4. Verify

High-Availability Deployment

Overlay System

Size Overlays

Access Overlays

Infrastructure Overlays

Composing Overlays

ClickHouse Configuration

Standalone vs Replicated

External ClickHouse

Auto-Tuning

AI Gateway sub-chart

Upgrade

Uninstall

FAQ

Istio, Service Mesh

Two installs in one cluster

Custom StorageClass

Air-Gapped Environments

​Prerequisites

​Quick Start

​Operator access and feature flags

​License

​Low-Resources Deployment

​Production Deployment

​1. Provide the Secrets

​2. Create a Values File

​3. Install

​4. Verify

​High-Availability Deployment

​Overlay System

​Size Overlays

​Access Overlays

​Infrastructure Overlays

​Composing Overlays

​ClickHouse Configuration

​Standalone vs Replicated

​External ClickHouse

​Auto-Tuning

​AI Gateway sub-chart

​Upgrade

​Uninstall

​FAQ

​Istio, Service Mesh

​Two installs in one cluster

​Custom StorageClass

​Air-Gapped Environments

Prerequisites

Quick Start

Operator access and feature flags

License

Low-Resources Deployment

Production Deployment

1. Provide the Secrets

2. Create a Values File

3. Install

4. Verify

High-Availability Deployment

Overlay System

Size Overlays

Access Overlays

Infrastructure Overlays

Composing Overlays

ClickHouse Configuration

Standalone vs Replicated

External ClickHouse

Auto-Tuning

AI Gateway sub-chart

Upgrade

Uninstall

FAQ

Istio, Service Mesh

Two installs in one cluster

Custom StorageClass

Air-Gapped Environments