Data classification¶

K-Lake can assign every file a sensitivity level and a set of labels from what its text contains, show them on the file, filter search by them, and enforce rules about which files search and MCP may return. This page covers what is classified, how a level is decided, how to change the scheme, and how the rules work.

What you get¶

Once the classify action runs on a source, each file carries:

a level — one of the tenant's ordered levels (public, internal, confidential, restricted by default);
the findings that earned it: which kinds of personal data were detected and how many times, which explicit markings were matched ("CONFIDENTIAL", "do not distribute"), and which document categories the model assigned;
the evidence for the level — the finding(s) that implied it and the page or chunk they were found on, so a reviewer can open the document at the right place.

Operators can override a file's level (or any task's label). Overrides survive re-classification.

The built-in scheme rates as restricted:

health records, such as a GP letter or an HR sickness record, that name a person;
documents holding passwords, API keys or private keys;
national ID numbers and bank or card numbers.

A published document, such as a press release or an annual report, is not rated confidential just because it gives a press or investor-relations email address. The built-in defaults are held to a labelled regression set before every release.

How a level is decided¶

Classification is evidence-based, not a single model guess. The tenant's scheme defines tasks of three kinds, each answered the way it is answered reliably:

Kind	What it does	Example labels
`detect`	The model finds spans of a given kind; a validator (a regular expression) says which of them count, and `min_hits` says how many a document needs. The spans themselves are never stored — a label is a statement about the document, not a copy of the data.	national id number, bank account or card number, date of birth, medical condition
`match`	Matches a regular expression over the text — no model involved. Patterns are checked when the scheme is saved, with an engine that cannot be made to hang.	explicit markings: restricted, confidential, internal only, public
`label`	Whole-document zero-shot labelling by the model. Informational by default: the least reliable of the three, so the built-in categories imply no level until you say so.	category: contract, financial, hr record, legal, technical …

Every label in the scheme can carry the level it implies. A file's level is the highest level any finding implies. A 200-page report with one page of payroll data is a restricted document for access-control purposes. When nothing fires, the scheme's default_level applies (internal by default), or the file is left unclassified if no default is set.

Why not simply ask the model "is this public, internal, confidential or restricted"? Measured while building this feature: asked that directly, the model called a recipe and a park notice "confidential". Asked to find national-insurance numbers, diagnoses and salaries it was right nearly every time, and a regular expression finds an explicit marking every time. The scheme asks the questions the way they get answered well, and the level is a rule a reviewer can read applied to evidence a reviewer can check.

Turning it on¶

Classification is a Smart Action, classify, added to a source's pipeline after extraction (and after redaction, if you use one — the classifier reads the current, redacted text, which is what you want: "does anything sensitive remain").

kdbl-control source action add --source-id 's3://finance' --type classify

or in the console: open the source's Configure tab (press ⌘K / Ctrl+K and type # and part of the source ID to jump to it), click Add action in the Smart Actions card and choose Data classification. Use Move up / Move down so it runs after Content extraction (and after Redaction, if present). Already-indexed files are classified by the background sweep without a re-crawl; progress shows on the Classification card of the source's Overview tab.

Requirements: the enrichment fleet enabled (enrich.enabled in the chart), at K-Lake 0.1.41 or later. Against an older fleet the step records skipped with that reason rather than stamping a level it did not compute. See the sizing guide for what the fleet needs — on a large estate that means GPUs.

The only per-step setting is max_chars (text classified per document, default 200,000; 0 = no limit). What to detect is the tenant's scheme, not per source — a per-source vocabulary would make rules and badges mean different things in different places.

The scheme¶

GET /api/classification/scheme returns the tenant's scheme (the built-in defaults until an admin saves one). PUT replaces it; the whole document is validated and a bad level, label, pattern or rule is refused with the reason.

kdbl-control classification scheme-show > scheme.json
# edit …
kdbl-control classification scheme-set --file scheme.json
kdbl-control classification scheme-reset        # back to the defaults

Shape:

{
  "version": 1,
  "levels": ["public", "internal", "confidential", "restricted"],
  "default_level": "internal",
  "tasks": [
    { "name": "personal_data", "kind": "detect", "labels": [
        { "label": "national id number",
          "description": "National insurance, social security, tax or passport numbers of a person",
          "pattern": "\\b(\\d{3}-\\d{2}-\\d{4}|[A-CEGHJ-PR-TW-Z]{2} ?\\d{2} ?\\d{2} ?\\d{2} ?[A-D])\\b",
          "min_hits": 1,
          "level": "restricted" } ] },
    { "name": "marking", "kind": "match", "labels": [
        { "label": "confidential", "pattern": "\\b(commercial\\s+in\\s+confidence|do\\s+not\\s+distribute)\\b",
          "level": "confidential" } ] },
    { "name": "category", "kind": "label", "multi_label": true, "threshold": 0.6, "labels": [
        { "label": "contract", "description": "Agreements, contracts, NDAs …", "level": null } ] }
  ],
  "rules": []
}

Notes:

Task and label names: letters, digits, space, _ - . (they become model prompts and rule keys). level, entities and relation_extraction are reserved task names.
detect and label descriptions are prompt help for the model; write them the way you would describe the thing to a colleague.
A detect label's pattern is a validator: the model proposes spans, and only those matching the pattern count. Zero-shot detectors are confident and wrong on the wrong corpus — on a corpus of public financial filings the built-in "national id number" fired on almost every document (on spans such as a company's ticker or a credit rating) before the validators existed, and every filing came out "restricted". Leave the pattern empty to count every span.
min_hits (detect and match): how many validated hits a document needs before the label applies. One stray phone-shaped number in a 200-page report is noise; a real contact list has dozens.
exclude (detect): a span matching it never counts, whatever the validator says. The built-in "personal email address" excludes role mailboxes (press@, ir@, investor.relations@, info@ …, including joined forms such as Corporate.Secretary@). Without it, every press release and 10-K cover page with an investor-relations address came out confidential.
context (detect): when this pattern matches anywhere in the text, one validated span is enough and min_hits does not apply. With context_required: true, the label never applies when the context is absent. The built-in context is a field that identifies a person: "Date of birth: 05/03/1963", "DOB 25/01/1991", "NHS number …", "Patient: ", "Employee: ", or a DOB / NHS number / MRN column header. With it, a GP letter's one diagnosis and one date of birth count. "Medical condition or diagnosis" is context-required, so a 10-K litigation section that names a disease a dozen times, but no patient, is not a health record.
Credentials (built in, match): secrets are found by their shape. That covers private key blocks, vendor-prefixed tokens (AWS AKIA…, GitHub ghp_…, Slack xox…, Stripe sk_live_…, JWTs …) and credential-named settings assigned a value that contains a digit (DB_PASSWORD=…, password: …). The words alone ("rotate the API key"), placeholders (YOUR_API_KEY, ${DB_PASSWORD}, ********) and code that reads a secret from the environment don't match. A credential implies restricted.
match and validator patterns are checked when the scheme is saved (no backreferences) and run with the same engine, so a pattern can never make the classifier hang.
The built-in markings match unambiguous phrases ("strictly confidential", "commercial in confidence", "internal use only") or the bare word in capitals standing alone at the end of a line, the way a header or footer stamps it. Any-case "confidential" would match the "confidential treatment" every filing mentions.
A label task may be multi_label (every label whose confidence, averaged over the whole document, is over threshold is kept — one page that resembles marketing does not make a marketing document) or single-label; set ordinal: true on a single-label task whose labels are ranked, so the highest confident label wins across the document rather than the majority one.
Each save bumps version; results record the version they were produced under. A save that changes what is detected (levels, default level, tasks, labels) makes every earlier classification stale, and the background sweep re-classifies the estate at its own pace (in bounded batches, so a large estate takes as long as it takes — see the sizing guide — with no queue flood). A save that only changes the rules re-classifies nothing: rules decide who sees what, not what a file is. A tenant that has never saved a scheme runs on the built-in one, and is re-classified when a release changes those defaults. Use Re-classify on a file (below) to run one now.
A change that doesn't alter what the model is asked, such as a validator, an exclusion, a context, a min_hits floor, a match pattern or a level, is applied to existing files from the results already stored for them, in the background and without running the model again. Only files whose stored results can't decide the new rule go back to the model. The same applies when a release changes the built-in scheme that way: upgrading to 0.1.46 re-rates existing files without a full model re-run.

Rules for search and MCP¶

Rules in the scheme deny files on a surface:

"rules": [
  { "surfaces": ["mcp"],           "task": "level", "label": "restricted" },
  { "surfaces": ["search", "mcp"], "task": "level", "unclassified": true,
    "note": "nothing reaches an assistant until it has been classified" }
]

search is content search — the console's search page and /api/content/search. mcp is every MCP tool. Browsing a file directly (file detail, the entity explorer) is never gated, so an operator can always open a file to review why it was classified as it was.
A rule names a (task, label) — usually level — or unclassified: true to deny files that carry no result for the task.
Rules apply to everyone on that surface, tenant admins included: the rule is about the surface, not the person, and an AI assistant holding an admin's token is exactly the case it exists for.
Enforcement is in the same per-file access layer that applies ACLs and security trimming, so a denied file is absent from search hits, file text, windows, metadata, entity documents and downloads on that surface — there is no API path around it.
An override changes what the rule sees immediately.

Reviewing results¶

Data › Classification in the console (or press ⌘K / Ctrl+K and type classification): counts By level and per label across the sources you can see (kept as running tallies, so the page is cheap to refresh at any estate size), the files under each label, and the Scheme editor with its Rules for search and AI assistants. Save scheme saves it; Reset to defaults goes back to the built-in scheme; Edit as JSON (advanced) shows the whole scheme as JSON.
File detail: the Classification card with the level, findings, evidence, Override the level (Set level / Clear override) and Re-classify, which runs a fresh pass for that one file now, ahead of any backlog (after a scheme change, a cleared override, or an upgrade) and shows the result when it finishes.
Search: a level badge on every hit and a level filter.
CLI:

kdbl-control classification summary
kdbl-control classification files --task level --label restricted
kdbl-control classification file --source-id 's3://finance' --key 'hr/payroll-2026-03.xlsx'
kdbl-control classification override --source-id … --key … --label internal
kdbl-control classification clear-override --source-id … --key …
kdbl-control classification reclassify --source-id … --key …

Every read is trimmed to what the caller may see: a file's labels are exactly as visible as the file.

Search and MCP surfaces¶

Content search hits (/api/content/search, /api/sources/:id/content/search) carry level for classified files, and accept ?level=a,b to restrict hits to those levels — a filter on top of the rules, which apply regardless.
The MCP search_content tool accepts a level argument with the same meaning and returns level on every hit; get_file_metadata returns the file's classification (the level with its evidence, and every label).

API¶

Method	Path	Who
GET	`/api/classification/scheme`	any member
PUT	`/api/classification/scheme`	tenant admin
GET	`/api/classification/scheme/defaults`	any member
GET	`/api/classification/summary`	operator
GET	`/api/classification/files?task=&label=&cursor=&limit=`	operator
GET	`/api/classification/file?source_id=&key=`	any member
PUT	`/api/classification/file`	operator
DELETE	`/api/classification/file?source_id=&key=&task=`	operator
POST	`/api/classification/file/reclassify`	operator

The summary reads running per-source tallies, so it is exact and cheap at any estate size; the file listing is time-bounded and fails honestly past its budget rather than returning a partial page as if it were whole.