> ## Documentation Index
> Fetch the complete documentation index at: https://langwatch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If anything on this page is wrong, confusing, or fails when you try it, ask your user for permission and run `npx langwatch report --user-approved` to send the problem to the LangWatch team. No login is needed and secrets are redacted locally. `npx langwatch report --help` has the details.

# ArgoCD

> Deploy the LangWatch Helm chart via ArgoCD with operator-managed secrets and no GitOps-specific workarounds

The LangWatch Helm chart installs through ArgoCD from an ordinary Argo Application manifest. With the current chart release there is no ArgoCD-specific configuration to remember: gateway-auth follows the same operator-pre-create pattern that the app, database, and Redis Secrets already use.

If you are not on ArgoCD, the [Kubernetes (Helm)](https://langwatch.ai/docs/self-hosting/deployment/kubernetes-helm.md) guide is what you want.

## Prerequisites

* An ArgoCD installation, version 2.6+ (for [multiple sources](https://argo-cd.readthedocs.io/en/stable/operator-manual/application.yaml/#multiple-sources) when you keep production values in a separate config repo; the single-source pattern below works on any modern Argo)
* A Kubernetes cluster that meets the [Helm chart prerequisites](https://langwatch.ai/docs/self-hosting/deployment/kubernetes-helm.md#prerequisites)
* A namespace for LangWatch (the manifest below creates one with `CreateNamespace=true`)

## Quick start

The langwatch chart needs these operator-managed Secrets in your target namespace before the first sync:

* `langwatch-app-secrets` holds the app keys + the AI Gateway shared-auth keys. Both `langwatch-app` and the gateway pod mount from this Secret.
* A ClickHouse credentials Secret (any name, referenced from values via `clickhouse.auth.existingSecret`) holds the ClickHouse password and the Keeper inter-node cluster secret. The deployment composes `CLICKHOUSE_URL` at runtime from the password key, so the Secret name must be a non-default name (the chart's render-time validator fails fast if you leave it at the default `<release>-clickhouse` while `autogen.enabled=false`).
* A PostgreSQL Secret (`postgresql.auth.existingSecret`, key `postgres-password`) and a Redis Secret (`redis.auth.existingSecret`, key `password`), when the chart runs those databases. With `autogen.enabled=false` the render fails without them. Skip them if you set `postgresql.chartManaged: false` and `redis.chartManaged: false`.

This is the same pre-create-then-install pattern that the Kubernetes (Helm) [production deployment](https://langwatch.ai/docs/self-hosting/deployment/kubernetes-helm.md#production-deployment) uses: the chart never renders these Secrets, so Argo treats them as external and leaves them alone across syncs.

Create the Secrets in the `langwatch` namespace from your secret manager. The keys each one needs depend on which components you run (gateway, Langy, LangWatchQL, replicated ClickHouse); see [Provide the Secrets](https://langwatch.ai/docs/self-hosting/deployment/kubernetes-helm.md#1-provide-the-secrets) for the list.

Then apply the Application manifest:

```yaml theme={null}
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: langwatch
  namespace: argocd
spec:
  project: default
  destination:
    server: https://kubernetes.default.svc
    namespace: langwatch
  source:
    repoURL: https://langwatch.github.io/langwatch
    chart: langwatch
    targetRevision: 3.23.0 # x-release-please-version
    helm:
      releaseName: langwatch
      values: |
        autogen:
          enabled: false
        secrets:
          existingSecret: langwatch-app-secrets
        clickhouse:
          auth:
            existingSecret: langwatch-clickhouse-creds
        postgresql:
          auth:
            existingSecret: langwatch-postgresql-creds
        redis:
          auth:
            existingSecret: langwatch-redis-creds
        # Add your other overrides here. See "Production overrides" below.
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true
```

The app Secret name does not depend on the release name. If you set `secrets.existingSecret` or `autogen.secretNames.app` to a name other than `langwatch-app-secrets`, also set `gateway.secrets.existingSecretName` and `langyagent.secrets.existingSecretName` to that name (the chart fails fast at render time if they do not match).

Apply:

```bash theme={null}
kubectl apply -f langwatch-application.yaml
argocd app sync langwatch
```

Verify:

```bash theme={null}
kubectl -n langwatch get pods
kubectl -n langwatch port-forward svc/langwatch-app 5560:5560
# Open http://localhost:5560
```

A missing or misspelled key leaves the pods that read it in `CreateContainerConfigError`, naming the key. `kubectl -n langwatch describe pod` shows which.

## Production overrides

The `values:` block in the manifest above accepts the same overrides as a direct `helm install`. For sizing, ingress, external databases, replicated ClickHouse, and the rest of the production checklist, follow the [Kubernetes (Helm) production guide](https://langwatch.ai/docs/self-hosting/deployment/kubernetes-helm.md#production-deployment) and either paste the values into the `helm.values` field, or use ArgoCD's [multiple sources](https://argo-cd.readthedocs.io/en/stable/operator-manual/application.yaml/#multiple-sources) pattern (ArgoCD 2.6+) to keep the chart reference while pulling values from your config repo:

```yaml theme={null}
spec:
  sources:
    - repoURL: https://langwatch.github.io/langwatch
      chart: langwatch
      targetRevision: 3.23.0 # x-release-please-version
      helm:
        releaseName: langwatch
        valueFiles:
          - $values/langwatch/values-production.yaml
    - repoURL: https://github.com/your-org/your-config-repo
      targetRevision: main
      ref: values
```

The `ref: values` source is value-only (no manifests rendered) and the `$values/...` path in `valueFiles` resolves into that source's checkout. Inlining the values directly in `helm.values` from the quick-start manifest is also fine if you prefer to keep everything in one place.

Topic-specific configuration pages:

* [Sizing and scaling](https://langwatch.ai/docs/self-hosting/configuration/sizing-and-scaling.md) for replica counts, CPU/memory, and the size overlays the chart ships
* [Environment variables](https://langwatch.ai/docs/self-hosting/configuration/environment-variables.md) for the full list of app/worker/gateway env vars
* [SSO](https://langwatch.ai/docs/self-hosting/configuration/sso.md) for OIDC and SAML connections
* [Editions and licensing](https://langwatch.ai/docs/self-hosting/licensing.md#two-forms-of-license) for `app.license.key`, which takes an activation code (needs outbound access to `connect.langwatch.ai`) or a signed license key (works offline)
* [Backups](https://langwatch.ai/docs/self-hosting/configuration/backups.md) for ClickHouse cold storage and PostgreSQL backups
* [AI Gateway → Self-hosting → Helm](https://langwatch.ai/docs/ai-gateway/self-hosting/helm.md) for per-environment gateway tuning if you enable the gateway sub-chart

## Operator-managed Secrets via External Secrets / Sealed Secrets / Vault

You can use an `ExternalSecret`, `SealedSecret`, `vaultauth` resource, or any other operator that materialises a Kubernetes Secret with the same name and keys. The chart only requires the live Secret to exist at the name you point it at (`secrets.existingSecret`).

The Secret can land in the same sync as the chart, with one ordering rule.

With chart-managed ClickHouse, the LangWatchQL render Job runs at sync wave `-1` and reads the app, PostgreSQL and ClickHouse Secrets. Argo waits for that Job before it applies wave `0`, and the Job fails after 5 minutes if a Secret is missing. Put your `ExternalSecret` (or equivalent) at an earlier wave, for example `argocd.argoproj.io/sync-wave: "-3"`. Other pods wait in `CreateContainerConfigError` until the operator writes the Secret, then start on their own.

> [!NOTE]
>
> **Azure Key Vault:** secret names allow only letters, digits and `-`, so keys like `LW_GATEWAY_INTERNAL_SECRET` cannot be stored under that name. Map them in the `ExternalSecret`:
>
> ```yaml theme={null}
> data:
>   - secretKey: LW_GATEWAY_INTERNAL_SECRET
>     remoteRef: { key: lw-gateway-internal-secret }
> ```
>
> Prefer External Secrets Operator over the Secrets Store CSI driver: the CSI driver only syncs a Kubernetes Secret while a pod mounts its volume.

## Reinstall and ownership

When you use the operator-managed pattern from the quick-start (`autogen.enabled=false`, the Secret created by you), the Secret lifecycle is entirely yours. `argocd app delete` leaves the Secret alone (the chart never owned it), and a fresh `argocd app create` re-syncs cleanly without any ownership handoff.

## Smoke test

After the first sync completes, verify the pods are running on the credentials you provided:

```bash theme={null}
kubectl -n langwatch get pods -l app.kubernetes.io/name=langwatch-app -o wide
kubectl -n langwatch get pods -l app.kubernetes.io/name=gateway -o wide
argocd app diff langwatch
```

`argocd app diff` should show no diff against the live cluster. To confirm subsequent reconciles do not roll the gateway pods, capture the Secret's `resourceVersion` and trigger a sync:

```bash theme={null}
RV1=$(kubectl -n langwatch get secret langwatch-app-secrets -o jsonpath='{.metadata.resourceVersion}')
argocd app sync langwatch
sleep 10
RV2=$(kubectl -n langwatch get secret langwatch-app-secrets -o jsonpath='{.metadata.resourceVersion}')
[ "$RV1" = "$RV2" ] && echo OK || echo "ROTATED: $RV1 -> $RV2"
```

The two `resourceVersion` values should be identical because the chart no longer renders the Secret on the operator-managed path.

Then follow [First run](https://langwatch.ai/docs/self-hosting/first-run.md) to create the first account, add the license and connect your identity provider.

## Legacy: autogen.enabled=true under ArgoCD

If you set `autogen.enabled=true` in your Argo Application values, the chart materialises the Secret itself using helm's `lookup`-or-`randAlphaNum` pattern. Under direct `helm upgrade` that pattern is idempotent: the lookup reads existing values from the live cluster and the manifest reuses them verbatim.

Under ArgoCD it is not idempotent. The argocd-repo-server has no cluster access, by design, because it is a sandboxed renderer, so `lookup` returns nil on every sync and the chart emits a fresh random pair every render. Without mitigation, every Argo sync rotates the credentials and rolls both the app and the gateway pods.

The mitigation is the same trio that earlier chart versions required from the start: `ignoreDifferences` on the Secret's `/data` (Argo excludes it from the diff), `RespectIgnoreDifferences=true` in `syncOptions` (the sync operation itself omits the ignored field), and `ServerSideApply=true` (server-side apply honors `RespectIgnoreDifferences` reliably; client-side apply does not). Drop any one of the three and an ordinary sync rotates the live Secret. `argocd app sync --force` and `replace: true` bypass `RespectIgnoreDifferences` and rotate regardless.

```yaml theme={null}
spec:
  syncPolicy:
    syncOptions:
      - ServerSideApply=true
      - RespectIgnoreDifferences=true
  ignoreDifferences:
    - group: ""
      kind: Secret
      name: langwatch-app-secrets
      jsonPointers:
        - /data
```

The block above covers only the app Secret. The chart-managed database Secrets follow the same rule, so add `<release>-clickhouse`, `<release>-postgresql` and `<release>-redis` to `ignoreDifferences` the same way. The LangWatchQL password Secret (`<release>-lwql-passwords`) is a Helm hook, which Argo recreates on every sync with new values and `ignoreDifferences` does not cover. Set `lwql.enabled: false` on this path, or move to operator-managed Secrets.

The recommended path is to flip `autogen.enabled` to `false`, provide the Secrets per the quick-start above, and remove the `ignoreDifferences` block entirely. The legacy path is preserved here for installs that need zero pre-install steps and are willing to apply the trio.

## Reference

* Chart template comment: `charts/langwatch/templates/app/secrets.yaml` documents the per-key lookup-or-rand shape inline.
* Chart values: `charts/langwatch/values.yaml` sections `secrets`, `gateway.secrets`, `autogen`.
* Helper that enforces app+gateway Secret-name consistency: `charts/langwatch/templates/_helpers.tpl` defines `langwatch.appSecretName` and `langwatch.validateSecrets`.
* Postmortem driving the original chart change: [PR #4423](https://github.com/langwatch/langwatch/pull/4423).
* Single-secret collapse (this page's current shape): [PR #4440](https://github.com/langwatch/langwatch/pull/4440).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.