Multi-tenant SaaS reseller pattern

Provision per-customer virtual keys and budgets from your application backend, audit everything, and bill accurately.

This pattern lets a SaaS product built on LLMs give each customer their own budget without handing out direct provider credentials. You become the upstream provider; the LangWatch AI Gateway becomes your provisioning + enforcement + audit layer.

The shape

End user's app
      │
      ▼
YOUR SaaS backend ──provisions VKs via──▶ LangWatch Management API (/api/gateway/v1/*)
      │
      │  issues VK to end-user's runtime
      ▼
End user's runtime ──/v1/chat/completions──▶ LangWatch Gateway ──▶ Upstream provider (your account)

Key properties:

• Each end-user has their own VK. One per customer, or per seat, your call.
• You set per-customer budgets. Enforced at the gateway. When a customer hits their cap, the gateway returns 402 budget_exceeded. Your code doesn’t need to track spend.
• All audit + trace data lands in your LangWatch project. End-users never see LangWatch.
• End-users can’t escape your policy. Policy rules, model allowlists, cache rules all attach to the VK.

Step 1: Model the tenancy

Decide your scope granularity:

Model	VK per	Budget scope	Use case
Per customer	one VK per tenant	virtual_key	Most SaaS apps, one account = one VK
Per seat	one VK per LangWatch user	principal	Internal platform teams where every seat is a real LangWatch user
Per team	one VK per sub-org	team-scope budgets	Customers have their own teams/projects

For this walkthrough, assume per-customer model. Note that principal budgets target real LangWatch user accounts, which your external customers are not; for reseller tenancy the per-customer unit is the virtual key itself.

Step 2: Configure the upstream ModelProvider once

Your LangWatch org owns a single ModelProvider row per upstream (OpenAI, Anthropic) at ORGANIZATION scope. End-users never see these. Configure them under Settings → Model Providers → Add Model Provider (Scope = Organization), then open each row’s Advanced (Gateway) tab and set the per-credential gateway caps:

Field	Suggested value
RPM	100000 (your bulk-purchase cap from the upstream)
Fallback priority	10 for primary, 20 for backup
Provider config (JSON)	region/deployment overrides as needed

No separate “binding” entity to mint. The ModelProvider id is what your code references:

# Look up the id once and store it in your backend config
langwatch model-providers list --format json | \
  jq -r '.[] | select(.provider == "openai" and .scope.scope_type == "ORGANIZATION") | .id'
# → mp_01HZX...

Step 3: Provision a VK with its cap when a customer signs up

One call mints the key AND its budget, atomically: the key can never exist for a moment without the cap you asked for. Server-side code (Node.js example, plain REST):

const LW = "https://app.langwatch.ai/api/gateway/v1";
const headers = {
  Authorization: `Bearer ${process.env.LANGWATCH_BACKEND_TOKEN}`,
  "Content-Type": "application/json",
};

export async function provisionCustomer(customerId: string, planLimitUsd: string) {
  const res = await fetch(`${LW}/virtual-keys`, {
    method: "POST",
    headers,
    body: JSON.stringify({
      name: `customer-${customerId}`,
      description: `Tenant key for ${customerId}`,
      // scopes omitted: the key defaults to your backend token's project,
      // which is exactly where you want the customer's traces to land.
      budget: {
        limit_usd: planLimitUsd, // from your plans table
        window: "month",
        on_breach: "block",      // or warn for grace periods
      },
      config: {
        modelsAllowed: ["gpt-5-mini", "gpt-5", "claude-haiku-4-5-20251001"],
        cache: { mode: "respect" },
      },
    }),
  });
  const { virtual_key, secret } = await res.json();

  // Persist virtual_key.id; hand the secret to the customer exactly once.
  return { vkId: virtual_key.id, secret };
}

To also cap each end user inside the tenant (their seats, their agents’ users), add one attributed-user template on the key; it covers every current and future end user with lazy per-user buckets, nothing to provision per user:

await fetch(`${LW}/budgets`, {
  method: "POST",
  headers,
  body: JSON.stringify({
    scope: { kind: "attributed_user", anchor_virtual_key_id: vkId },
    name: `customer-${customerId} per-user cap`,
    window: "month",
    limit_usd: "25",
    on_breach: "block",
  }),
});

Requests then carry the end-user id in the OpenAI user field (or the X-LangWatch-End-User-Id header); see Budgets → per-end-user budgets.

The eligible-provider set is the union of every ModelProvider visible from the key’s scopes (the ones you configured in Step 2). Pin routing_policy_id plus routing_mode: "policy" if you run a custom ordering; otherwise leave routing at its default.

In a signup webhook:

const { vkId, secret } = await provisionCustomer(newCustomer.id, "25");
await db.customers.update({
  where: { id: newCustomer.id },
  data: { langwatchVkId: vkId },
});
// Send `secret` to the customer via email / dashboard UI / API response
// exactly once. Your system never stores it in plaintext.

Step 4: Change the cap when the plan changes

The budget field on update upserts the key’s own cap, so plan changes are one call keyed by the VK id you stored (no budget id bookkeeping):

export async function setCustomerPlan(vkId: string, planLimitUsd: string) {
  await fetch(`${LW}/virtual-keys/${vkId}`, {
    method: "PATCH",
    headers,
    body: JSON.stringify({ budget: { limit_usd: planLimitUsd, window: "month" } }),
  });
}

Pass budget: null to remove the cap entirely (the spend history stays). Changes propagate to the gateway within 30 s via the /changes feed, no restart, no customer impact.

Step 5: End-user makes a call

Your customer’s app calls the gateway directly with the VK you gave them:

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.langwatch.ai/v1",
    api_key=os.environ["MY_SAAS_API_KEY"],  # the VK secret you sent them
)

The gateway:

• Authenticates the VK.
• Checks the customer’s budget.
• Applies your modelsAllowed allowlist.
• Dispatches to OpenAI using your provider credential.
• Meters the cost against the customer’s budget.
• Emits a LangWatch trace into your project, attributed to the key.

Step 6: Show the customer their spend and bill them

For display, the read-back is one REST call per key:

const res = await fetch(`${LW}/virtual-keys/${vkId}/spend`, { headers });
const spend = await res.json(); // current UTC month by default
// { virtual_key_id, spent_usd: "12.4531", requests: 1841, window: {...} }

or from a shell:

langwatch virtual-keys spend vk_01HZX... -f json

The number comes from the same cost path the LangWatch dashboard reads, so what you invoice matches what you see in the UI. For the enforcement view (spend against the cap), list VK-scoped budgets:

langwatch gateway-budgets list --scope-type virtual-key -f json | \
  jq '[.budgets[] | {vk: .scope_id, spent: .spent_usd, limit: .limit_usd}]'

spend_available: false in that output means spend could not be totalled server-side; hold billing until it recovers rather than invoicing a zero.

For invoicing, do not poll aggregates: ingest the per-request spend event stream through a signed webhook endpoint into your own billing ledger, and reconcile against spend-summaries checksums at period close. Every event carries exact integer quantities per token class, integer nano-USD cost, the tenant key, the end-user id, and your echoed metadata, which is what a rebilling invoice needs and what an aggregate cannot give you. The full billing loop (receive, verify, dedup, settled supersession, reconcile, period close) is its own cookbook: Metering and rebilling your customers, with a runnable reference implementation in TypeScript and Python.

Rating stays in your billing system: you define the markup over the gateway-reported cost (or bill passthrough plus a management fee), and events carry rate_version so cost is re-derivable.

Handling over-limit customers

When on_breach: block, the gateway returns 402 budget_exceeded. Your customer sees:

{
  "error": {
    "type": "budget_exceeded",
    "message": "Budget exceeded for scope=virtual_key window=month",
    "code": "budget_exceeded"
  }
}

Your customer’s app should catch this and either:

• Show an upgrade CTA in the end-user’s UI.
• Fall back to a free-tier response (“you’ve hit your monthly cap; upgrade for unlimited”).

Because the response is standard OpenAI-compatible, your error handler only needs to switch on error.type.

Suspension, rotation, revocation

Customer stops paying, or you need an emergency stop → POST /virtual-keys/:id/disable. Requests are rejected with the distinct 403 virtual_key_disabled until you call /enable; budgets, config, and any running rotation grace stay intact, and propagation is seconds. This is the reversible kill switch, use it for anything the customer might come back from. See Virtual Keys → Disable and enable.

Customer resets their API key in your UI → POST /virtual-keys/:id/rotate, get a new secret, send it to the customer. The old secret keeps authenticating for a 24 h grace window, so a client you have not reached yet does not start failing mid-rollout. The gateway’s cache TTL (~60 s) is only how long the new secret takes to be accepted everywhere, not when the old one expires.

Customer is gone for good → POST /virtual-keys/:id/revoke. The next request returns 403 virtual_key_revoked. One-way; for a cancellation grace period, disable now and revoke after the grace, rather than scheduling a bare revoke.

Audit: who did what

Every write through your backend credential is audited with action, target, and before/after metadata. Scoped API keys attribute the write to the key’s owning user; legacy project keys carry no user, so their writes record the machine principal svc_<your_project_id>. Filter the audit log on target virtual_key to get a per-customer provisioning history, under /settings/audit-log in the LangWatch UI.

If you need SIEM export: query ONLY the gateway-shape AuditLog rows with a read-only role scoped to that table, and ship the filtered result set, never a whole-database dump (which would carry unrelated, sensitive application data). For example, a daily job running COPY (SELECT id, "organizationId", "userId", action, "targetKind", "targetId", "createdAt" FROM "AuditLog" WHERE "organizationId" = '<your-org-id>' AND "targetKind" IN ('virtual_key', 'budget', 'model_provider', 'routing_policy', 'cache_rule') AND "createdAt" >= now() - interval '1 day') TO STDOUT WITH CSV into your SIEM’s ingestion path: an explicit column list pinned to your organization, leaving the before/after payloads out unless your SIEM genuinely needs them. There is no public REST audit-export endpoint in v1, see Audit log → Querying programmatically for the supported paths (UI CSV download, direct SQL).

Gotchas

• Never let the customer see your backend API token. It has virtualKeys:create; they’d provision more VKs charged to you.
• Rotate VKs when an end-user leaves your customer’s org. Otherwise the ex-user keeps spend access until the budget resets.
• Name keys after your tenant id (customer-<id>) at creation time. The key’s name and id are your join keys for spend read-back and audit history; principal_user_id is only for real LangWatch user accounts, not your external customers.
• Test the 402 path in your app before go-live. Many apps have unhandled exceptions on budget breach and crash the user’s flow.
• Budgets scoped to virtual_key are the right level for per-customer enforcement, and the inline budget field on create/update manages exactly that budget for you.

Rate limits at your gateway level

If you want to throttle a specific customer without moving them to a different plan:

await fetch(`${LW}/virtual-keys/${vkId}`, {
  method: "PATCH",
  headers,
  body: JSON.stringify({ config: { rateLimits: { rpm: 60 } } }),
});

Gateway enforces at the VK level; customer sees 429 rate_limit_exceeded. Combine with a short-window budget (hour, minute) for finer control.

See also

• Metering and rebilling your customers: the event-driven billing loop this provisioning pattern plugs into.
• Management REST API: every endpoint used above.
• Virtual Keys: VK lifecycle semantics, including disable vs revoke.
• Budgets: hierarchical scope logic and per-end-user templates.
• Security: how backend tokens are isolated from VK secrets.
• CI smoke-test cookbook: validate the full flow in CI.