Production deployment¶

Install KDBL Context Lake (K-Lake) on your own Kubernetes cluster from the Helm chart. This is the production path — right-sized, hardened, and version-pinned. To try K-Lake first on a single VM, see the proof of concept; this guide is the real thing.

Production installs from the chart against a real cluster — the same product and schema you evaluated. A fresh install stands up its own database and re-crawls your live sources, so your documents need no copying (crawls are idempotent and skip unchanged content).

What a re-crawl does not rebuild

Re-crawling recovers indexed content, not the work you did around it. Entity merges you corrected by hand, per-source access grants, users and API tokens already issued to your AI assistants, source definitions and their credentials, schedules, and — if you enabled version history — the record of how documents changed over time, which a fresh crawl cannot reconstruct at all because it only ever sees the current version.

If you have invested in any of that during an evaluation, talk to us before standing production up separately: moving the database across preserves it, and re-crawling does not.

Prerequisites¶

A Kubernetes cluster (1.27+) with a default StorageClass (the database and workers bind persistent volumes).
metrics-server installed (present by default on most managed clusters, including AKS) — the built-in autoscaling reads CPU from it.
helm 3.8+ (OCI support).
A production licence from your KDBL representative, sized to your purchased capacity + term. The platform is fail-closed — it won't start without a valid licence — and you can renew it live later.

Install¶

The chart is published as an OCI artifact on the public registry, so there's no chart repository to add — install it directly:

export KDBL_LICENCE='KDBLLIC1.<your production licence>'

helm install kdbl oci://ghcr.io/kdbl-consulting/kdbl \
  --version <X.Y.Z> \
  --namespace kdbl --create-namespace \
  --set tier=medium \
  --set licence.blob="$KDBL_LICENCE"

Secrets (the master key and database passwords) are generated on first install and preserved across upgrades, so they never rotate underneath running pods. Run helm show values oci://ghcr.io/kdbl-consulting/kdbl --version <X.Y.Z> to see every configurable option.

Storage¶

The bundled database and cache each claim a persistent volume from your cluster's default StorageClass, which is what the command above uses. That is the right answer on managed Kubernetes and on k3s alike.

Set it explicitly if you want a specific class — a faster disk, or a cluster with no default:

  --set global.storageClass=managed-csi-premium   # e.g. Azure SSD
  --set global.storageClass=local-path            # e.g. k3s

If the volumes cannot be provisioned, the symptom is easy to misread: the database and cache pods sit in Pending while every other pod reports CrashLoopBackOff, because they have no database to reach. Check the pending pod's events before anything else —

kubectl -n kdbl describe pod kdbl-postgres-0 | tail -20

pod has unbound immediate PersistentVolumeClaims means the StorageClass, not the product. kubectl get sc shows what your cluster actually offers.

Choose a tier¶

The tier right-sizes every component for a scale. Pick the one matching the capacity you licensed:

Tier	Licensed capacity	Database (CPU / RAM / disk)	Worker pool	Extractor pool
`small`	up to 1 TB	2–4 / 4–8 Gi / 50 Gi	1 → 3	0 → 4
`medium`	up to 10 TB	4–8 / 16–24 Gi / 256 Gi	2 → 10	1 → 10
`large`	up to 50 TB	8–16 / 32–48 Gi / 1 Ti	4 → 50	2 → 25
`enterprise`	100 TB+	8–24 / 32–64 Gi / 2 Ti	8 → 200	2 → 50

baseline → ceiling shows what always runs and what each pool scales up to. The sizing guide maps a tier to node count / vCPU / RAM with worked examples.

The licence sets your capacity, the tier sets the footprint

The tier governs compute (how many workers, how big the database). Your licensed capacity is enforced separately, so choosing a leaner tier never changes what you're licensed to hold.

Front it with Ingress + TLS¶

Expose the console and API on a real hostname with TLS:

  --set console.access=ingress \
  --set ingress.host=kdbl.example.com \
  --set ingress.tls.enabled=true

The console (which proxies the API and MCP endpoint) is then reachable at your hostname. Bring your own certificate or your platform's cert-manager. Without ingress, the console is an internal ClusterIP you reach over a port-forward.

Autoscaling¶

The worker, metadata, extraction, and Smart Actions pools autoscale on CPU between each tier's baseline and ceiling — no extra components to install beyond the standard metrics-server.

Two pools can additionally scale on queue depth, which reacts sooner than CPU for work that spends its time waiting rather than computing: the worker's crawl backlog and the Smart Actions queue. Both are opt-in, because they need a Prometheus adapter (or KEDA) publishing the metric — see the tuning section below. Without it, both pools scale on CPU alone. You set the bounds once (via the tier) and K-Lake tracks demand. See the sizing guide for when to raise a ceiling.

Queue-depth autoscaling for large crawls (optional)¶

CPU is a coarse signal for ingest: crawling a large, deep file share is bound by how fast the source answers, not by CPU, so a big backlog of pending work can build up while each pod's CPU still looks moderate. For estates with very large or deeply-nested shares, K-Lake can scale the ingest workers on backlog depth instead — adding pods the moment the pending work grows and removing them as it drains. This lets one large crawl spread across many pods and finish far sooner.

It's off by default because it needs monitoring infrastructure the CPU autoscaler doesn't: a Prometheus that scrapes K-Lake's metrics and a metrics adapter that exposes them to Kubernetes' autoscaler (prometheus-adapter, or KEDA). Enable it in three steps:

Turn on the worker backlog metric on the chart:
```
helm upgrade kdbl oci://ghcr.io/kdbl-consulting/kdbl \
  --reset-then-reuse-values \
  --set worker.subtreeAutoscale.enabled=true \
  --set worker.subtreeAutoscale.averageValue=500
```
averageValue is how much pending backlog each worker should hold before another joins — lower scales out more eagerly. Start around 500 and tune it to your estate.
Map the metric in your adapter. The autoscaler looks up an external metric named kdbl_subtree_backlog_pending; your prometheus-adapter needs one rule telling it how to serve that from the scraped data (add to the adapter's rules.external):
```
- seriesQuery: 'kdbl_subtree_backlog{state="pending"}'
  metricsQuery: 'max(kdbl_subtree_backlog{state="pending"})'
  name: { as: kdbl_subtree_backlog_pending, matches: "" }
  resources: { namespaced: false }
```
(Using KEDA instead? Its Redis-Streams scaler reads the backlog directly — no adapter rule needed.) Restart the adapter to load the rule.

Smart Actions has its own metric. If you also enable actionHost.autoscaling.queueDepthAutoscale.enabled=true, add its rule too — the pool looks up kdbl_action_queue_depth_pending:

- seriesQuery: 'kdbl_action_queue_depth{state="pending"}'
  metricsQuery: 'max(kdbl_action_queue_depth{state="pending"})'
  name: { as: kdbl_action_queue_depth_pending, matches: "" }
  resources: { namespaced: false }

Both pools scale on CPU without any of this. The queue-depth metric is strictly an addition — the autoscaler takes whichever signal asks for more replicas — so if the adapter is missing or the rule is wrong, they keep scaling on CPU rather than stopping.

Prometheus has to be scraping these pools first. Every pod carries prometheus.io/scrape, which is enough for an annotation-discovery Prometheus. If you run the Prometheus Operator, set monitoring.serviceMonitor.enabled=true on the chart and it will create ServiceMonitors for the worker, metadata, extraction and Smart Actions pools (add monitoring.serviceMonitor.labels if your Prometheus selects them by a release label).

Verify it before relying on it:

kubectl get --raw /apis/external.metrics.k8s.io/v1beta1 | grep kdbl_subtree_backlog
kubectl -n <namespace> get hpa kdbl-worker      # target shows a live value, not <unknown>

Run a large crawl and watch the worker pool scale out, then settle back as the backlog clears.

The workers are safe to scale freely — work is leased exactly once, so extra pods share the load without duplicating it, and each new pod picks up its share of the pending backlog immediately.

Database durability & disaster recovery¶

The bundled database is a single instance — a turnkey start, but not highly available. For production DR, choose one of:

External highly-available managed database (recommended). Point K-Lake at your own managed PostgreSQL so its provider owns backup, replication, and point-in-time recovery:
```
  --set postgres.bundled=false \
  --set postgres.external.host=<your-db-host> \
  --set postgres.external.database=kdbl
```
The database must provide the pg_search, vector, and pg_trgm extensions (pg_search powers core keyword search). Fully-managed offerings that don't allow pg_search on their extension allowlist aren't compatible — use a self-managed PostgreSQL or a ParadeDB distribution.
Bundled database with nightly backup. For the bundled database, enable a nightly logical backup to a retained volume:
```
  --set postgres.backup.enabled=true
```
This is a same-cluster safety net (mainly against accidental deletes), not off-cluster disaster recovery — the backups share the cluster's failure domain. Prefer the external managed database for true DR.

High availability¶

Pod disruption budgets ship on by default. During node drains and cluster upgrades, the stateless pools keep all-but-one replica serving, and the bundled database/cache block eviction of their sole instance so a routine drain can't silently take them down. For real HA of the data tier, use an external highly-available database (above).

Connect your identity provider¶

For SSO and AI-client (MCP) access, configure each tenant against your IdP (Entra ID, Google, Okta, Keycloak). See Connect your AI and the Enablement runbook.

Onboard your first tenant¶

With the cluster-admin token printed on install, create a tenant, its first admin user, and a personal access token in one step:

kdbl-control onboard my-org --name "My Org" --admin-email you@example.com

Then hand the printed token to a user and follow the quick start to add a source and get your first grounded answer. Turn on advanced features (MCP, per-file security trimming, directory correlation) with the Enablement runbook.

Upgrading¶

Upgrade in place with helm upgrade to a newer chart version — see Upgrading.