Production deployment¶

Install KDBL Context Lake (K-Lake) on your own Kubernetes cluster from the Helm chart. This is the production path — right-sized, hardened, and version-pinned. To try K-Lake first on a single VM, see the proof of concept; this guide is the real thing.

Production installs from the chart against a real cluster — the same product and schema you evaluated. A fresh install stands up its own database and re-crawls your live sources, so your documents need no copying (crawls are idempotent and skip unchanged content).

What a re-crawl does not rebuild

Re-crawling recovers indexed content, not the work you did around it. Entity merges you corrected by hand, per-source access grants, users and API tokens already issued to your AI assistants, source definitions and their credentials, schedules, and — if you enabled version history — the record of how documents changed over time, which a fresh crawl cannot reconstruct at all because it only ever sees the current version.

If you have invested in any of that during an evaluation, talk to us before standing production up separately: moving the database across preserves it, and re-crawling does not.

Prerequisites¶

A Kubernetes cluster (1.27+) with a default StorageClass (the database and workers bind persistent volumes).
metrics-server installed (present by default on most managed clusters, including AKS) — the built-in autoscaling reads CPU from it.
helm 3.8+ (OCI support).
A production licence from your KDBL representative, sized to your purchased capacity + term. The platform is fail-closed — it won't start without a valid licence — and you can renew it live later.

Install¶

The chart is published as an OCI artifact on the public registry, so there's no chart repository to add — install it directly:

export KDBL_LICENCE='KDBLLIC1.<your production licence>'

helm install kdbl oci://ghcr.io/kdbl-consulting/kdbl \
  --version <X.Y.Z> \
  --namespace kdbl --create-namespace \
  --set tier=medium \
  --set licence.blob="$KDBL_LICENCE"

Secrets (the master key and database passwords) are generated on first install and preserved across upgrades, so they never rotate underneath running pods. Run helm show values oci://ghcr.io/kdbl-consulting/kdbl --version <X.Y.Z> to see every configurable option.

Storage¶

The bundled database and cache each claim a persistent volume from your cluster's default StorageClass, which is what the command above uses. That is the right answer on managed Kubernetes and on k3s alike.

Set it explicitly if you want a specific class — a faster disk, or a cluster with no default:

  --set global.storageClass=managed-csi-premium   # e.g. Azure SSD
  --set global.storageClass=local-path            # e.g. k3s

If the volumes cannot be provisioned, the symptom is easy to misread: the database and cache pods sit in Pending while every other pod reports CrashLoopBackOff, because they have no database to reach. Check the pending pod's events before anything else —

kubectl -n kdbl describe pod kdbl-postgres-0 | tail -20

pod has unbound immediate PersistentVolumeClaims means the StorageClass, not the product. kubectl get sc shows what your cluster actually offers.

Choose a tier¶

The tier right-sizes every component for a scale. Pick the one matching the capacity you licensed:

Tier	Licensed capacity	Database (CPU / RAM / disk)	Worker pool	Extractor pool
`small`	up to 1 TB	2–4 / 4–8 Gi / 50 Gi	1 → 3	1 → 4
`medium`	up to 10 TB	4–8 / 16–24 Gi / 256 Gi	2 → 10	1 → 10
`large`	up to 50 TB	8–16 / 32–48 Gi / 1 Ti	4 → 50	2 → 25
`enterprise`	100 TB+	8–24 / 32–64 Gi / 2 Ti	8 → 200	2 → 50

baseline → ceiling shows what always runs and what each pool scales up to. The sizing guide maps a tier to node count / vCPU / RAM with worked examples.

The licence sets your capacity, the tier sets the footprint

The tier governs compute (how many workers, how big the database). Your licensed capacity is enforced separately, so choosing a leaner tier never changes what you're licensed to hold.

Front it with Ingress + TLS¶

Expose the console and API on a real hostname with TLS:

  --set console.access=ingress \
  --set ingress.host=kdbl.example.com \
  --set ingress.tls.enabled=true

The console (which proxies the API and MCP endpoint) is then reachable at your hostname. Bring your own certificate or your platform's cert-manager. Without ingress, the console is an internal ClusterIP you reach over a port-forward.

Or let administrators manage the certificate from the console. With --set console.tls.managed=true, the console's Settings › Security › HTTPS certificate page (open Manage on that card, or press ⌘K / Ctrl+K and type HTTPS) generates a key and certificate request, imports what your CA returns (or a certificate and key, or a .pfx), and keeps it applied across upgrades. The chart creates an empty Secret, kdbl-console-tls, and gives the API permission to read and update that one Secret only. How the Secret reaches browsers is set by console.tls.serving:

traefik (k3s, k3d): automatic. It serves the certificate for every host name, so no ingress.host is needed, and the console can switch an HTTP-to-HTTPS redirect on and off. It does this by making the console's certificate traefik's default for the whole cluster, so on a traefik that also serves other applications, every https host without a certificate of its own presents the console's. On a shared ingress controller, prefer ingress-tls.
ingress-tls: with ingress.host set, the console Ingress's TLS entry uses the Secret (unless ingress.tls.secretName names another). Only the console's host name presents it; browsing by IP address keeps the controller's own certificate.
external, for example ingress-nginx or AKS application routing: point the controller's default certificate at the Secret (for ingress-nginx, --default-ssl-certificate=<namespace>/kdbl-console-tls) and configure any HTTP-to-HTTPS redirect on the controller.

Leave it off where a platform team or cert-manager owns certificates. The page then stores a certificate but says it isn't applied. See Console HTTPS certificate.

Autoscaling¶

The worker, metadata, extraction, and Smart Actions pools autoscale on CPU between each tier's baseline and ceiling — no extra components to install beyond the standard metrics-server.

Two pools can additionally scale on queue depth, which reacts sooner than CPU for work that spends its time waiting rather than computing: the worker's crawl backlog and the Smart Actions queue. Both are opt-in, because they need a Prometheus adapter (or KEDA) publishing the metric — see the tuning section below. Without it, both pools scale on CPU alone. You set the bounds once (via the tier) and K-Lake tracks demand. See the sizing guide for when to raise a ceiling.

Queue-depth autoscaling for large crawls (optional)¶

CPU is a coarse signal for ingest: crawling a large, deep file share is bound by how fast the source answers, not by CPU, so a big backlog of pending work can build up while each pod's CPU still looks moderate. For estates with very large or deeply-nested shares, K-Lake can scale the ingest workers on backlog depth instead — adding pods the moment the pending work grows and removing them as it drains. This lets one large crawl spread across many pods and finish far sooner.

It's off by default because it needs monitoring infrastructure the CPU autoscaler doesn't: a Prometheus that scrapes K-Lake's metrics and a metrics adapter that exposes them to Kubernetes' autoscaler (prometheus-adapter, or KEDA). Enable it in three steps:

Turn on the worker backlog metric on the chart:
```
helm upgrade kdbl oci://ghcr.io/kdbl-consulting/kdbl \
  --reset-then-reuse-values \
  --set worker.subtreeAutoscale.enabled=true \
  --set worker.subtreeAutoscale.averageValue=500
```
averageValue is how much pending backlog each worker should hold before another joins — lower scales out more eagerly. Start around 500 and tune it to your estate.
Map the metric in your adapter. The autoscaler looks up an external metric named kdbl_subtree_backlog_pending; your prometheus-adapter needs one rule telling it how to serve that from the scraped data (add to the adapter's rules.external):
```
- seriesQuery: 'kdbl_subtree_backlog{state="pending"}'
  metricsQuery: 'max(kdbl_subtree_backlog{state="pending"})'
  name: { as: kdbl_subtree_backlog_pending, matches: "" }
  resources: { namespaced: false }
```
(Using KEDA instead? Its queue scaler reads the backlog directly — no adapter rule needed.) Restart the adapter to load the rule.

Smart Actions has its own metric. If you also enable actionHost.autoscaling.queueDepthAutoscale.enabled=true, add its rule too — the pool looks up kdbl_action_queue_depth_pending:

- seriesQuery: 'kdbl_action_queue_depth{state="pending"}'
  metricsQuery: 'max(kdbl_action_queue_depth{state="pending"})'
  name: { as: kdbl_action_queue_depth_pending, matches: "" }
  resources: { namespaced: false }

Both pools scale on CPU without any of this. The queue-depth metric is strictly an addition — the autoscaler takes whichever signal asks for more replicas — so if the adapter is missing or the rule is wrong, they keep scaling on CPU rather than stopping.

Prometheus has to be scraping these pools first. Every pod carries prometheus.io/scrape, which is enough for an annotation-discovery Prometheus. If you run the Prometheus Operator, set monitoring.serviceMonitor.enabled=true on the chart and it will create ServiceMonitors for the worker, metadata, extraction and Smart Actions pools (add monitoring.serviceMonitor.labels if your Prometheus selects them by a release label).

Verify it before relying on it:

kubectl get --raw /apis/external.metrics.k8s.io/v1beta1 | grep kdbl_subtree_backlog
kubectl -n <namespace> get hpa kdbl-worker      # target shows a live value, not <unknown>

Run a large crawl and watch the worker pool scale out, then settle back as the backlog clears.

The workers are safe to scale freely — work is leased exactly once, so extra pods share the load without duplicating it, and each new pod picks up its share of the pending backlog immediately.

Database durability & disaster recovery¶

The bundled database is a single instance — a turnkey start, but not highly available. For production DR, choose one of:

External highly-available managed database (recommended). Point K-Lake at your own managed PostgreSQL so its provider owns backup, replication, and point-in-time recovery:
```
  --set postgres.bundled=false \
  --set postgres.external.host=<your-db-host> \
  --set postgres.external.database=kdbl
```
The database must provide the pg_search, vector, and pg_trgm extensions (pg_search powers core keyword search). Fully-managed offerings that don't allow pg_search on their extension allowlist aren't compatible — use a self-managed PostgreSQL or a ParadeDB distribution.
Bundled database with nightly backup. For the bundled database, enable a nightly logical backup to a retained volume:
```
  --set postgres.backup.enabled=true
```
This is a same-cluster safety net (mainly against accidental deletes), not off-cluster disaster recovery — the backups share the cluster's failure domain. Prefer the external managed database for true DR.

High availability¶

Pod disruption budgets ship on by default. During node drains and cluster upgrades, the stateless pools keep all-but-one replica serving, and the bundled database/cache block eviction of their sole instance so a routine drain can't silently take them down. For real HA of the data tier, use an external highly-available database (above).

Connect your identity provider¶

For SSO and AI-client (MCP) access, configure each tenant against your IdP (Entra ID, Google, Okta, Keycloak). See Connect your AI and the Enablement runbook.

Onboard your first tenant¶

With the cluster-admin token printed on install, create a tenant, its first admin user, and a personal access token in one step:

export KDBL_API_URL=https://kdbl.example.com
export KDBL_API_TOKEN=<the cluster-admin token printed on install>

kdbl-control onboard my-org --name "My Org" --admin-email you@example.com

onboard goes through the API rather than the database, so it needs the URL and a cluster-admin token — set them in the environment as above, or pass --api-url / --api-token on the command itself.

Then hand the printed token to a user and follow the quick start to add a source and get your first grounded answer. Turn on advanced features (MCP, per-file security trimming, directory correlation) with the Enablement runbook.

Upgrading¶

Upgrade in place with helm upgrade to a newer chart version — see Upgrading.

Identifier-parts search (0.1.45)¶

Keyword search can also match part of an underscore-joined identifier: kozijn or 90x114 finds IFC_kozijn_90x114. A passage that holds the word on its own still ranks above one that only holds it inside an identifier.

This needs a rebuilt keyword index. A new install has it from the start. An existing deployment keeps its current index, and searches exactly as before (exact identifiers included), until you enable the one-off rebuild — from 0.1.45 a chart switch:

Pick a quiet window. The rebuild runs online — search and ingest carry on, and search keeps using the current index until the new one is complete — but heavy ingest, re-extraction or redaction while it runs makes the new index grow much larger than it should.
Check free disk on the database volume. Both indexes exist until the new one takes over, so plan for free space of about 1.5× your current keyword index (on a test estate the new index came out about 1.2× the old one; the rest is headroom). The current size is lexical_bytes in GET /api/capacity?include_cluster=true (cluster admin).

Turn it on after the upgrade:

helm upgrade kdbl oci://ghcr.io/kdbl-consulting/kdbl --version <X.Y.Z> \
  --namespace kdbl --reset-then-reuse-values \
  --set search.identifierPartsRebuild=true

On a proof-of-concept stack: ./scripts/poc-up.sh --set search.identifierPartsRebuild=true.

Watch it. One worker builds the new index one partition at a time. The worker metrics kdbl_bm25_parts_partitions{state="built"} climbs to kdbl_bm25_parts_partitions{state="total"}, then kdbl_bm25_parts_ready becomes 1: the new index is live and part-of-identifier matching is on. On a test estate one 45 GB partition took about 30 minutes; a large estate should plan for a few hours. If a worker restarts, the next one carries on from the partitions already built.
Turn it off again once it is live (--set search.identifierPartsRebuild=false). It is a one-off. Leaving it on is harmless (once live it costs one small read every few minutes), so on a proof-of-concept stack, where changing it means re-running poc-up.sh, you can simply leave it.

If it grows too large, it stops itself. While each partition builds, the worker checks the new index's size. If it grows far beyond what is expected (by default 3× that partition's current keyword index), the worker cancels the build, deletes the partial index so the disk comes back at once, and holds the rebuild: kdbl_bm25_parts_held is 1 and the worker logs HELD. Search keeps using the current index, so users see no change. Wait for a quieter period, then release the hold:

kubectl exec kdbl-postgres-0 -- psql -U kdbl -d kdbl -c \
  "UPDATE kdbl_bm25_parts_migration SET held_at = NULL, held_partition = NULL;"

The next check picks up again from the partition that was stopped.

If the worker logs build WEDGED, a partition build has stopped making progress and cannot be cancelled from inside the database (kdbl_bm25_parts_guard_wedged_total increases — alert on it). Restart PostgreSQL promptly: with the bundled database, delete the kdbl-postgres-0 pod. Then release the hold as above; the worker cleans up the partial build and continues.

Pausing and rolling back. Maintenance mode stops the rebuild between partitions, and it resumes when maintenance mode is off; a later upgrade waits for at most the partition in flight. Until the new index takes over there is nothing to roll back. Afterwards, rolling back to an earlier release is safe: older versions search the new index unchanged, without part-of-identifier matching.

External database. The rebuild needs a session on the database server itself, not through a connection pooler. With the bundled database the chart provides one. If your external database URL points at a pooler, the worker refuses to build and says so (needs a session on Postgres itself); set KDBL_BM25_PARTS_BUILD_POSTGRES_URL on the worker to the database server's own address.