Diagnostic packs¶

K-Lake has no call-home. It does not report crashes, does not send usage data, and has no channel to the vendor at all — that property is what makes air-gapped and regulated deployments possible, and nothing about support changes it.

The consequence is that when you open a support ticket, we cannot see your deployment. A diagnostic pack closes that gap on your terms: one command collects everything we would otherwise ask for over a week of email into a single archive that you generate, inspect, and send.

kdbl-control support pack \
  --api-url https://kdbl.internal --api-token "$KDBL_API_TOKEN" \
  --namespace kdbl

✓ diagnostic pack written
  path     kdbl-diag-20260811T183203Z.tar.gz
  size     4.2 MiB
  sha256   7626128ad87794a29caf181c5d8c8c4519486dff3c9c6c16c40a3e4d890a447c

Send that file to your KDBL support contact through whatever channel your policy allows. Nothing leaves your network until you do.

You can also generate a pack from the web console (Support → Diagnostics) or the REST API — the same collector backs all three.

What it contains¶

Directory	Contents
`summary.txt`	Triage view — versions, licence, and every anomaly the collector noticed. Start here.
`MANIFEST.json`	What was collected, what failed, what was truncated, and a SHA-256 per file.
`in-band/`	What K-Lake knows about itself: configuration, sources and their settings, queue depth, crawls, grouped errors, per-pod self-reports, database health.
`cluster/`	What your orchestrator knows: pod state, events, node capacity, logs, Helm values.

The two halves are collected differently, and deliberately so. K-Lake holds no Kubernetes credentials — granting the API permission to read pod logs would make an internet-facing component able to export your logs, just to save you one command. So the cluster/ half is collected by the CLI using your kubectl credentials, from your workstation.

The cluster/ half is deployment-wide. K-Lake's own data in in-band/ is scoped to the token you generate with, but pod logs interleave every tenant a pod serves — so on a multi-tenant deployment cluster/ can name sources and paths belonging to other tenants. Nobody gains access they did not already have (reading those logs needs cluster credentials), but the archive carries them, which matters when deciding who may receive it. Use --no-kubectl for a pack whose contents match your token's scope exactly.

Either half is optional. No kubectl, or --no-kubectl, gives an in-band-only pack. An unreachable API gives a cluster-only pack — which matters, because "the API won't start" is a common reason to need one. Whichever half is missing is stated explicitly in MANIFEST.json, so nobody mistakes a partial pack for a clean bill of health.

Runtime tests¶

Everything above describes how your deployment is configured. Add --runtime-tests and the pack also records what actually works:

kdbl-control support pack --runtime-tests

• running runtime tests…
  8 pass · 1 warn · 0 fail · 1 skipped

Check	What it catches
`database.round_trip`	A database that reads but cannot write — disk full, write-ahead log wedged, a replica promoted by mistake — answers `SELECT 1` perfectly and fails everything the product does. Also reports median and p95 latency.
`queue.dispatch`	A reachable queue that cannot hold an entry, and an eviction policy that silently discards queued work under memory pressure.
`search.embeddings`	An embedder that answers but returns a degenerate or wrong-sized vector, which makes hybrid search silently wrong rather than failing. Also flags CPU-speed embedding.
`search.rerank`	A reranker running on CPU, which leaves search correct and unusably slow. Projects the cost over your configured pool.
`fleet.clock_skew`	Clock drift between pods, which breaks token validity and licence evaluation in ways that never point at the clock.
`licence.enforcement`	Enforcement that is actively blocking operations, and capacity nearing its limit.
`storage.headroom`	Network mounts that have stopped responding, and volumes running out of space.
`extraction.plugin`	An extractor that heartbeats healthily but fails every document — an OCR model missing from an offline image, weights that never loaded. Pushes a test document through and checks the text comes back.
`enrichment.plugin`	An enricher that has stopped answering, so entity extraction silently produces nothing.
`sources.connectivity`	Expired credentials, a rotated key, a firewall change — all of which present as "the crawl finds nothing" and are otherwise only reported after the next crawl fails.
`mcp.configuration`	MCP enabled without a resource URI, which answers happily and then fails every client authorization.

Every check is bounded by its own timeout, so a wedged dependency is reported as wedged rather than hanging the pack. Nothing mutates your data: the two checks that write at all use a rolled-back transaction and a throwaway key.

A check that could not run reports skipped, never pass — "we didn't look" and "we looked and it's fine" are different answers.

Redaction levels¶

kdbl-control support pack --level metadata     # most private
kdbl-control support pack --level operational  # default
kdbl-control support pack --level full --acknowledge-full

Level	Includes	Use when
`metadata`	Counts, configuration, versions, error signatures. Every file path, source name, user and tenant replaced by a pseudonym generated for this pack alone.	The pack will be seen by more people than your named support engineer.
`operational` (default)	Adds file paths, source identifiers and verbatim error messages. No document content, no chunk text, no search queries.	Almost every ticket.
`full`	Adds audit-trail file-key samples and search query strings — what your users searched for and which files came back.	Retrieval, extraction or redaction faults, where the query is the bug.

full requires a cluster-admin token and an explicit acknowledgement. A tenant-admin requesting it gets an error rather than a quietly downgraded pack, because a pack that silently came back at a lower level would be read as evidence the data was not there.

The metadata pseudonyms are salted per pack and the salt is never stored. The same path appears identically throughout one pack — so we can still tell that two errors concern the same file — and differently in every other pack.

Pseudonymisation covers in-band/ only. The cluster/ half (pod logs, events, describe output) goes through the credential scrubber and the leak gate, so it carries no secrets — but file paths, share names and pod names appear verbatim, because they cannot be reliably identified inside free-form log text. If you need a pack with no verbatim identifiers anywhere, use --no-kubectl, which omits the cluster/ half entirely.

What is never included¶

Document content, extracted text, or chunk text
Source credentials in any form. The encrypted secret blobs are never read.
The master key, licence signing material, or download signing secret
API tokens, personal access tokens, or session cookies

Every text file in a pack passes through a credential scrubber, and the result is then checked by a separately written detector. If that detector matches anything, the pack is not written at all — there is no "generated with warnings" outcome.

Before you send it¶

Open it. Everything is plain text or JSON, and it is meant to be read.

tar xzf kdbl-diag-*.tar.gz
less kdbl-diag-*/summary.txt

The sha256 printed on generation is over the archive itself, so you and we can confirm we are looking at the same file.

Sending it somewhere else¶

If attaching the file isn't convenient, --upload-url PUTs it to a presigned URL you supply — your storage, your egress rule, your audit trail:

kdbl-control support pack --upload-url "https://your-bucket.example/…?X-Amz-Signature=…"

This is never a KDBL endpoint. There is no address in this product that uploads to us.

--encrypt-to-vendor writes an additional copy sealed to KDBL, so the file can travel by email or as a ticket attachment without its contents being readable in transit or at rest by anyone but us:

kdbl-control support pack --encrypt-to-vendor

The plaintext pack is always written too — read that one, send the sealed one. Encrypting in place would take away the "inspect it before you send it" property this whole feature is built on.

Each sealed pack names the key it was sealed to, so KDBL can tell a pack that needs an older key from one that arrived damaged. To check which key a pack needs without opening it:

kdbl-control support key-id kdbl-diag-20260811T183203Z.tar.gz.sealed

Comparing two packs¶

"It worked last week" is a common way for a problem to be described. If you have a pack from before and one from after, the difference is usually the answer:

kdbl-control support diff last-week.tar.gz today.tar.gz

kdbl pack diff
  before  last-week.tar.gz
  after   today.tar.gz

config
  ~ config.sources[source_id=smbfs://nas/finance].bulk_ingest: false → true
state
  ~ state.queue.failed: 3 → 4102 (+4099)

2 difference(s).

Timestamps, counters, latencies, memory readings and pod names are excluded — they differ between any two packs of the same untouched deployment, and including them would bury the two lines above in thousands. Named counters like failed are reported as deltas instead of being dropped, because the magnitude is the finding.

Works offline on packs someone sent you: it needs no cluster and no API.

Auditing¶

Generating a pack is a bulk export of operational data, and your own security team should be able to see that it happened:

kdbl-control support history

{"packs": [{"at": "2026-08-11T18:32:03Z", "principal": "alice@example.com",
            "level": "operational", "scope": "tenant",
            "digest": "7626128ad877…", "bytes": 184320}]}

K-Lake records that a pack was generated, by whom, at what level, and its digest. It does not keep a copy — a cache of bulk exports sitting in your cluster would be a standing liability with no diagnostic value.

Seeing what a pack would contain¶

To review the scope before generating anything:

kdbl-control support preview --level operational

This lists every section, what it carries, and what is excluded, without collecting a single row. It is the same description the web console shows on its consent screen.

Useful flags¶

Flag	Default	Notes
`--namespace`	`kdbl`	Namespace holding the deployment.
`--log-lines`	`2000`	Per-container tail. Truncation is always recorded in the manifest.
`--since`	`24h`	Only collect logs newer than this.
`--no-kubectl`	off	Skip cluster collection entirely.
`--kube-context`	current context	Which cluster to collect from. Worth setting explicitly if your kubeconfig holds more than one.
`--runtime-tests`	off	Also exercise the dependencies (see above). Adds a few seconds.
`--encrypt-to-vendor`	off	Also write a copy sealed to KDBL, alongside the readable one.
`--upload-url`	—	PUT the pack to a presigned URL you supply.
`--out`	`kdbl-diag-<utc>.tar.gz`	Output path.

If your deployment is in licence lockdown¶

The support endpoints stay reachable when an expired licence has blocked everything else. A deployment in lockdown is exactly the one you are most likely to need a pack from.