ArgoCD

Deploy the LangWatch Helm chart via ArgoCD with operator-managed secrets and no GitOps-specific workarounds

The LangWatch Helm chart installs through ArgoCD from an ordinary Argo Application manifest. With the current chart release there is no ArgoCD-specific configuration to remember: gateway-auth follows the same operator-pre-create pattern that the app, database, and Redis Secrets already use.

If you are not on ArgoCD, the Kubernetes (Helm) guide is what you want.

Prerequisites

• An ArgoCD installation, version 2.6+ (for multiple sources when you keep production values in a separate config repo; the single-source pattern below works on any modern Argo)
• A Kubernetes cluster that meets the Helm chart prerequisites
• A namespace for LangWatch (the manifest below creates one with CreateNamespace=true)

Quick start

The langwatch chart needs these operator-managed Secrets in your target namespace before the first sync:

• langwatch-app-secrets holds the app keys + the AI Gateway shared-auth keys. Both langwatch-app and the gateway pod mount from this Secret.
• A ClickHouse credentials Secret (any name, referenced from values via clickhouse.auth.existingSecret) holds the ClickHouse password and the Keeper inter-node cluster secret. The deployment composes CLICKHOUSE_URL at runtime from the password key, so the Secret name must be a non-default name (the chart’s render-time validator fails fast if you leave it at the default <release>-clickhouse while autogen.enabled=false).
• A PostgreSQL Secret (postgresql.auth.existingSecret, key postgres-password) and a Redis Secret (redis.auth.existingSecret, key password), when the chart runs those databases. With autogen.enabled=false the render fails without them. Skip them if you set postgresql.chartManaged: false and redis.chartManaged: false.

This is the same pre-create-then-install pattern that the Kubernetes (Helm) production deployment uses: the chart never renders these Secrets, so Argo treats them as external and leaves them alone across syncs.

Create the Secrets in the langwatch namespace from your secret manager. The keys each one needs depend on which components you run (gateway, Langy, LangWatchQL, replicated ClickHouse); see Provide the Secrets for the list.

Then apply the Application manifest:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: langwatch
  namespace: argocd
spec:
  project: default
  destination:
    server: https://kubernetes.default.svc
    namespace: langwatch
  source:
    repoURL: https://langwatch.github.io/langwatch
    chart: langwatch
    targetRevision: 3.20.1 # x-release-please-version
    helm:
      releaseName: langwatch
      values: |
        autogen:
          enabled: false
        secrets:
          existingSecret: langwatch-app-secrets
        clickhouse:
          auth:
            existingSecret: langwatch-clickhouse-creds
        postgresql:
          auth:
            existingSecret: langwatch-postgresql-creds
        redis:
          auth:
            existingSecret: langwatch-redis-creds
        # Add your other overrides here. See "Production overrides" below.
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true

The app Secret name does not depend on the release name. If you set secrets.existingSecret or autogen.secretNames.app to a name other than langwatch-app-secrets, also set gateway.secrets.existingSecretName and langyagent.secrets.existingSecretName to that name (the chart fails fast at render time if they do not match).

Apply:

kubectl apply -f langwatch-application.yaml
argocd app sync langwatch

Verify:

kubectl -n langwatch get pods
kubectl -n langwatch port-forward svc/langwatch-app 5560:5560
# Open http://localhost:5560

A missing or misspelled key leaves the pods that read it in CreateContainerConfigError, naming the key. kubectl -n langwatch describe pod shows which.

Production overrides

The values: block in the manifest above accepts the same overrides as a direct helm install. For sizing, ingress, external databases, replicated ClickHouse, and the rest of the production checklist, follow the Kubernetes (Helm) production guide and either paste the values into the helm.values field, or use ArgoCD’s multiple sources pattern (ArgoCD 2.6+) to keep the chart reference while pulling values from your config repo:

spec:
  sources:
    - repoURL: https://langwatch.github.io/langwatch
      chart: langwatch
      targetRevision: 3.20.1 # x-release-please-version
      helm:
        releaseName: langwatch
        valueFiles:
          - $values/langwatch/values-production.yaml
    - repoURL: https://github.com/your-org/your-config-repo
      targetRevision: main
      ref: values

The ref: values source is value-only (no manifests rendered) and the $values/... path in valueFiles resolves into that source’s checkout. Inlining the values directly in helm.values from the quick-start manifest is also fine if you prefer to keep everything in one place.

Topic-specific configuration pages:

• Sizing and scaling for replica counts, CPU/memory, and the size overlays the chart ships
• Environment variables for the full list of app/worker/gateway env vars
• SSO for OIDC and SAML connections
• Editions and licensing for app.license.key, which takes an activation code (needs outbound access to connect.langwatch.ai) or a signed license key (works offline)
• Backups for ClickHouse cold storage and PostgreSQL backups
• AI Gateway → Self-hosting → Helm for per-environment gateway tuning if you enable the gateway sub-chart

Operator-managed Secrets via External Secrets / Sealed Secrets / Vault

You can use an ExternalSecret, SealedSecret, vaultauth resource, or any other operator that materialises a Kubernetes Secret with the same name and keys. The chart only requires the live Secret to exist at the name you point it at (secrets.existingSecret).

The Secret can land in the same sync as the chart, with one ordering rule.

With chart-managed ClickHouse, the LangWatchQL render Job runs at sync wave -1 and reads the app, PostgreSQL and ClickHouse Secrets. Argo waits for that Job before it applies wave 0, and the Job fails after 5 minutes if a Secret is missing. Put your ExternalSecret (or equivalent) at an earlier wave, for example argocd.argoproj.io/sync-wave: "-3". Other pods wait in CreateContainerConfigError until the operator writes the Secret, then start on their own.

Azure Key Vault: secret names allow only letters, digits and -, so keys like LW_GATEWAY_INTERNAL_SECRET cannot be stored under that name. Map them in the ExternalSecret:

data:
  - secretKey: LW_GATEWAY_INTERNAL_SECRET
    remoteRef: { key: lw-gateway-internal-secret }

Prefer External Secrets Operator over the Secrets Store CSI driver: the CSI driver only syncs a Kubernetes Secret while a pod mounts its volume.

Reinstall and ownership

When you use the operator-managed pattern from the quick-start (autogen.enabled=false, the Secret created by you), the Secret lifecycle is entirely yours. argocd app delete leaves the Secret alone (the chart never owned it), and a fresh argocd app create re-syncs cleanly without any ownership handoff.

Smoke test

After the first sync completes, verify the pods are running on the credentials you provided:

kubectl -n langwatch get pods -l app.kubernetes.io/name=langwatch-app -o wide
kubectl -n langwatch get pods -l app.kubernetes.io/name=gateway -o wide
argocd app diff langwatch

argocd app diff should show no diff against the live cluster. To confirm subsequent reconciles do not roll the gateway pods, capture the Secret’s resourceVersion and trigger a sync:

RV1=$(kubectl -n langwatch get secret langwatch-app-secrets -o jsonpath='{.metadata.resourceVersion}')
argocd app sync langwatch
sleep 10
RV2=$(kubectl -n langwatch get secret langwatch-app-secrets -o jsonpath='{.metadata.resourceVersion}')
[ "$RV1" = "$RV2" ] && echo OK || echo "ROTATED: $RV1 -> $RV2"

The two resourceVersion values should be identical because the chart no longer renders the Secret on the operator-managed path.

Then follow First run to create the first account, add the license and connect your identity provider.

Legacy: autogen.enabled=true under ArgoCD

If you set autogen.enabled=true in your Argo Application values, the chart materialises the Secret itself using helm’s lookup-or-randAlphaNum pattern. Under direct helm upgrade that pattern is idempotent: the lookup reads existing values from the live cluster and the manifest reuses them verbatim.

Under ArgoCD it is not idempotent. The argocd-repo-server has no cluster access, by design, because it is a sandboxed renderer, so lookup returns nil on every sync and the chart emits a fresh random pair every render. Without mitigation, every Argo sync rotates the credentials and rolls both the app and the gateway pods.

The mitigation is the same trio that earlier chart versions required from the start: ignoreDifferences on the Secret’s /data (Argo excludes it from the diff), RespectIgnoreDifferences=true in syncOptions (the sync operation itself omits the ignored field), and ServerSideApply=true (server-side apply honors RespectIgnoreDifferences reliably; client-side apply does not). Drop any one of the three and an ordinary sync rotates the live Secret. argocd app sync --force and replace: true bypass RespectIgnoreDifferences and rotate regardless.

spec:
  syncPolicy:
    syncOptions:
      - ServerSideApply=true
      - RespectIgnoreDifferences=true
  ignoreDifferences:
    - group: ""
      kind: Secret
      name: langwatch-app-secrets
      jsonPointers:
        - /data

The block above covers only the app Secret. The chart-managed database Secrets follow the same rule, so add <release>-clickhouse, <release>-postgresql and <release>-redis to ignoreDifferences the same way. The LangWatchQL password Secret (<release>-lwql-passwords) is a Helm hook, which Argo recreates on every sync with new values and ignoreDifferences does not cover. Set lwql.enabled: false on this path, or move to operator-managed Secrets.

The recommended path is to flip autogen.enabled to false, provide the Secrets per the quick-start above, and remove the ignoreDifferences block entirely. The legacy path is preserved here for installs that need zero pre-install steps and are willing to apply the trio.

Reference

• Chart template comment: charts/langwatch/templates/app/secrets.yaml documents the per-key lookup-or-rand shape inline.
• Chart values: charts/langwatch/values.yaml sections secrets, gateway.secrets, autogen.
• Helper that enforces app+gateway Secret-name consistency: charts/langwatch/templates/_helpers.tpl defines langwatch.appSecretName and langwatch.validateSecrets.
• Postmortem driving the original chart change: PR #4423.
• Single-secret collapse (this page’s current shape): PR #4440.