Supported file types¶

This page lists the file types K-Lake extracts content from, what it gets out of each one, and which extraction engine does the work. It also covers what happens to a file K-Lake can't read, and the limits that apply to every file.

Every file a source lists is recorded whatever its type: its name, path, size, dates and permissions are indexed and searchable, and security trimming applies to it. This page is about the next step, content extraction: reading the text inside the file so that search, citations and AI answers can use it.

Extraction is switched on per source

Extraction is off until you enable it on a source. Once enabled it reads every type below by default. You can narrow it with an extension allow-list, a "record but don't extract" list, path globs, a size cap and a modified-date window. See Content extraction.

How a file reaches an engine¶

The engine is chosen per source, not per file type. The default engine, klex, reads almost everything on this page. There are four exceptions:

Plain text is read directly by the worker. Files with extensions such as .txt, .md, .csv and .log never go to an extraction engine (the full list is in the matrix). This is the fastest path.
Audio and video go to the media engine, kmedia, when it is deployed. The choice is made by file extension, whatever engine the source uses. Without kmedia, audio and video are recorded but not transcribed.
Pages klex couldn't read are handed to kdoc2 automatically whenever kdoc2 is running: scanned pages, pages klex read with low confidence, and tables it couldn't structure. The kdoc2 result replaces klex's for just those pages; the rest of the document is untouched. Files klex failed on are retried on kdoc2 when the source has Fallback extraction with kdoc2 as its engine. A source opts out of the page re-reads with "on_unread_tables": false on that step.
Pages that need OCR can go to your own cloud document service instead, when Cloud extraction is set up and switched on for the source. Scanned pages and tables klex couldn't structure are read by Azure AI Document Intelligence, Amazon Textract or Google Document AI in your own cloud account; only those pages leave your environment, and each file sent is recorded in the source's egress audit. You can also route a whole source to the Cloud document service.

K-Lake recognises a file by its contents when its storage reports only a generic binary content type (application/octet-stream, common for objects uploaded to S3 or MinIO by scripts), so a scanned PDF stored that way still reaches OCR.

flowchart LR
  F[File listed by a source] --> T{Plain text?}
  T -- yes --> W[Read directly]
  T -- no --> AV{Audio / video<br/>and kmedia deployed?}
  AV -- yes --> M[kmedia: transcript + on-screen text]
  AV -- no --> E[Source's engine<br/>klex by default]
  E -- pages needing OCR, unread tables --> K[kdoc2, when it is running]
  E -- pages needing OCR --> C[Your cloud document service,<br/>if Cloud extraction is on]

The matrix¶

Supported formats are part of our regular extraction testing. Best effort formats have a reader in the default engine, but we don't test them routinely. They usually extract well, but confirm on a sample of your own files before you depend on one. All engines named below are described in Content extraction.

Documents¶

Format	Extensions	What is indexed	Engine	Status
PDF (born-digital)	`.pdf`	Text in reading order, tables as structured grids, real page numbers	klex	Supported
PDF (scanned or mixed)	`.pdf`	OCR text for each page with no text layer; the pages that already have a text layer are read directly	klex, then kdoc2 when it is running, or your cloud document service with Cloud extraction	Supported
Word	`.docx`, `.docm`, `.doc`	Text, tables, text inside embedded images (OCR)	klex	Supported
Excel	`.xlsx`, `.xlsm`, `.xls`	Cell text, sheets as tables	klex	Supported
PowerPoint	`.pptx`, `.pptm`, `.ppt`	Slide text and notes, tables, text inside embedded images (OCR)	klex	Supported
OpenDocument	`.odt`, `.ods`, `.odp` (and flat `.fodt`/`.fods`/`.fodp`)	Text and tables	klex	Supported
Rich Text	`.rtf`	Text	klex	Supported
EPUB	`.epub`	Text	klex	Supported
HTML	`.html`, `.htm`, `.xhtml`	Visible text, tables	klex	Supported
XML	`.xml`	Element text	klex	Supported
Apple iWork	`.pages`, `.numbers`, `.key`	Text	klex	Best effort
Other office formats	OneNote `.one`, Access `.mdb`, WordPerfect `.wpd`, Quattro Pro, Hangul `.hwp`, Excel 5/95 workbooks	Text	klex	Best effort
Other document formats	FictionBook `.fb2`, compiled help `.chm`, InDesign `.idml`, FrameMaker `.mif`, translation memories `.tmx`/`.xlf`, dBASE `.dbf`, RSS/Atom feeds	Text	klex	Best effort

Plain text and data¶

Format	Extensions	What is indexed	Engine	Status
Plain text and markup-light text	`.txt`, `.text`, `.md`, `.markdown`, `.mkd`, `.mdown`, `.rst`, `.org`, `.log`	The text as written	Built in	Supported
Delimited data	`.csv`, `.tsv`, `.tab`	The text as written	Built in	Supported
JSON, YAML, source code, config files	`.json`, `.yaml`, `.py`, `.js`, `.sql`, `.ini`, …	Read as plain text	klex	Best effort

Text is decoded as UTF-8, with invalid bytes tolerated. A file with a text extension that turns out to be binary is skipped rather than indexed as noise.

Email and archives¶

Format	Extensions	What is indexed	Engine	Status
Email message	`.eml`, `.msg`	Headers, body, and the text of attachments	klex	Supported
Mailbox	`.mbox`, `.pst`	Every message, with attachments	klex	Best effort
Outlook rich-text attachment	`winmail.dat`	Contents	klex	Best effort
Archives	`.zip`, `.7z`, `.rar`, `.tar`, `.jar`, `.gz`, `.bz2`, `.xz`	The text of every readable file inside, recursively	klex	Best effort

Attachments and archive members belong to their parent

The text of an email attachment or a file inside an archive is indexed as part of the parent file. A search hit on it opens the email or the archive, not the member. The members are not listed or permission-checked as separate files, so treat access to an archive as access to everything in it.

Images (OCR)¶

Format	Extensions	What is indexed	Engine	Status
PNG, JPEG, BMP, WebP	`.png`, `.jpg`, `.jpeg`, `.bmp`, `.webp`	Text in the image (OCR), including rotated scans	klex; kdoc2 re-reads low-confidence images when it is running; your cloud document service with Cloud extraction	Supported
TIFF	`.tif`, `.tiff`	Text in the image (OCR); klex reads every page of a multi-page TIFF	klex	Supported
GIF	`.gif`	Text in the image (OCR), first frame	klex	Supported
HEIC/HEIF, Photoshop, JPEG XL	`.heic`, `.heif`, `.psd`, `.jxl`	Not read: recorded, but no text is extracted	—	Not supported

An image with no readable text (a photo, a logo) is recorded and marked no text. Photo metadata such as EXIF is not indexed.

Engineering and CAD¶

Format	Extensions	What is indexed	Engine	Status
AutoCAD DXF (ASCII)	`.dxf`	Title block, text and multiline text, dimensions, leaders, attributes, tables	klex	Supported
AutoCAD DXF (binary)	`.dxf`	A summary only: version, units, code page	klex	Supported
STEP 3D models	`.stp`, `.step`, `.p21`	Part and assembly names, product description	klex	Supported
IFC building models	`.ifc`	Project, site, building, storeys, spaces, element names and properties	klex	Supported
AutoCAD DWG, MicroStation DGN, Creo `.prt`	`.dwg`, `.dgn`, `.prt`	Little or no text	klex	Not supported: export drawings to DXF or PDF to index them

CAD files are read for what a person searches them by (drawing numbers, titles, part names, room names), not their geometry. Coordinates are never indexed.

Audio and video¶

These need the media engine, kmedia, which is not deployed by default (see Sizing).

Kind	Extensions	What is indexed
Audio	`.mp3`, `.m4a`, `.aac`, `.wav`, `.flac`, `.ogg`, `.oga`, `.opus`, `.wma`, `.aif`, `.aiff`	Transcript of the spoken words, timecoded
Video	`.mp4`, `.m4v`, `.mov`, `.mkv`, `.webm`, `.avi`, `.wmv`, `.flv`, `.mpg`, `.mpeg`	Transcript of the spoken words, plus the text shown on screen (slides, captions, titles), each timecoded

Timecodes. Each segment of a transcript records when it was spoken or shown, so a search hit opens the player at that moment.
Language. Speech is transcribed in its own language, detected automatically: most major languages are recognised. You can pin one language for the fleet. Words shown on screen are read in English.
Embedded subtitles. When a video carries a subtitle track, K-Lake indexes that text instead of transcribing the audio, because it is exact. Subtitle text is not timecoded, so a hit on it can't jump to the moment it is spoken. Separate .srt/.vtt files beside a video are not used.
Not produced: speaker labels, translation, or a detected-language field.
Long videos. On-screen text is read from about the first 20 minutes of a video. The transcript covers the whole file.
Not routed to kmedia: MPEG transport streams (.ts, .mts, .m2ts) and .3gp. A file whose storage reports a generic binary content type rather than an audio or video one may also be skipped.

Without kmedia, audio and video files are recorded (name, path, size, dates and permissions), but their content is not indexed.

What each engine reads¶

The matrix above describes the default deployment: klex, with kdoc2 alongside it where scanned documents matter. If you point a source at another engine instead, it reads only its own formats, and everything else in that source is skipped.

	klex (default)	kdoc2	kvision	kmedia	Cloud document service²
PDF text	✔	✔	✔		✔
PDF / image OCR	✔	✔	✔		✔
Office (modern)	✔
Office (legacy `.doc`/`.xls`/`.ppt`)	✔
OpenDocument, RTF, EPUB	✔
HTML, Markdown, text	✔
Email, archives	✔
CAD (DXF, STEP, IFC)	✔
Audio, video				✔
Image types for OCR	PNG, JPEG, BMP, WebP, TIFF, GIF	PNG, JPEG, BMP, WebP, TIFF¹, GIF¹	PNG, JPEG, BMP, WebP, TIFF¹		PNG, JPEG, BMP, WebP, TIFF

kdoc, the previous-generation document engine, was retired in 0.1.47. Every format it read is read by klex (and, for scanned PDFs and images, kdoc2): see What's new.

¹ First frame or page only. ² Your own Azure, AWS or Google document service, through Cloud extraction. Pages it reads leave your environment.

kdoc2 and the Cloud document service are companions, not replacements. They read only PDFs and images. A source pointed at either one alone skips its Office documents, email and text, so run them alongside klex rather than instead of it.

kdoc2 keeps the layout of scanned pages: a two-column page is read one column at a time rather than line by line across both columns, and a table on a scanned page is indexed as a table, with each row kept on one line.

OCR languages¶

Engine	Out of the box
klex	English
kdoc2	Latin-script languages (English and Western European languages)
kmedia, on-screen text	English
Cloud document service	Determined by your provider's service

Other OCR languages need a different language pack built into the engine image. Ask your KDBL contact if your estate has a significant volume of scanned documents in another script.

Limits¶

Limit	Default	Notes
Largest file extracted	2 GiB	Set per deployment. Each source can set a lower ceiling. Larger files are recorded and marked oversize.
Largest audio or video file	512 MiB	Set separately for the media fleet.
Text kept per file	50 million characters	Covers everything in the file, including archive members and attachments. Text past the cap is not indexed.
Time to extract one file	30 minutes per attempt	A file that takes longer is retried, then marked failed. Very long recordings on CPU-only media fleets are the usual cause.
Scanned pages OCR'd by klex in one file	2,000	Further pages are handed to kdoc2 when it is running.

Files in an archive or cold storage tier (for example Azure Archive, S3 Glacier) are recorded but never extracted, because reading them would trigger a rehydration and its cost. Password-protected and encrypted files can't be read.

When a file isn't extracted¶

A file that isn't extracted still appears in search results by name and path, with its metadata, and its permissions are still enforced. The file's detail view gives the reason:

Reason	Meaning
Oversize	Larger than the extraction ceiling.
Unsupported type	The engine assigned to the source doesn't read this format.
Binary, not text	The file has a text extension but binary content.
Empty	The file has no bytes.
No text	The file was read, but contains no text: a photo, silent audio, a blank scan.
Failed	The file couldn't be read: it is corrupt, encrypted, or password-protected. Failed files are retried, then quarantined; the source page lists them, and Fallback extraction can retry them on kdoc2.

After you add an engine (for example kmedia, or kdoc2 for scans), re-extract only the affected types rather than the whole source:

kdbl-control crawl --source-id <ID> --force-reextract --only-ext pdf,tif,tiff

What's new in 0.1.47¶

Scanned pages, poorly read pages and unread tables are re-read automatically whenever kdoc2 is running, for every source, with no setting to switch on. "on_unread_tables": false on a source's Fallback extraction step turns this off for that source, scanned pages included.
Text in files attached to a PDF is kept when a page of that PDF is re-read for its tables or by OCR. Before, it could be lost. PDFs indexed before 0.1.47 need re-extracting once to get this, and to restore attachment text already lost:
```
kdbl-control crawl --source-id <ID> --force-reextract --only-ext pdf
```
kdoc is retired. Every format it read is read by klex, with kdoc2 for scans and tables. See Upgrading.

What's new since 0.1.43¶

kdoc is retired (0.1.47), and no format loses support. What the previous-generation document engine read is now read by:
- PDFs, text and OCR: klex, with kdoc2 for scanned pages and tables.
- Word, PowerPoint and Excel (.docx, .pptx, .xlsx): klex.
- HTML, Markdown and plain text: klex and the built-in text reader.
- PNG, JPEG and TIFF images (the image types kdoc read): klex, with kdoc2.
Semantic (hybrid) search comes from the embedding service. See Upgrading. * CAD: DXF, STEP and IFC are read (previously indexed by name only). * WebP images are OCR'd by klex. * Low-confidence OCR pages (skewed, photocopied, photographed) are re-read by kdoc2 (automatically whenever kdoc2 is running, from 0.1.47). * kdbl-control crawl --force-reextract --only-ext … re-extracts chosen types only. * The default per-file ceiling is 2 GiB (previously 50 MiB). * Cloud extraction: scanned pages, images and tables can be read by your own Azure AI Document Intelligence, Amazon Textract or Google Document AI account instead of GPUs. See Cloud extraction. * Scanned pages keep their layout (kdoc2): two-column pages are read one column at a time, tables on scanned pages are indexed as tables with each row on one line, and small print and photographed pages lose far fewer lines. * PDF page numbers no longer shift after a blank or scanned page. Previously every later page could be labelled one page early in search results and citations. Re-extract affected sources to correct documents already indexed. * PDFs with a damaged embedded image (which failed with "Unable to end a page") are now indexed from their text layer instead of failing. * klex reads born-digital PDFs up to 6× faster, extracts spreadsheets with very large tables, and reports a file that runs out of memory as a failure of that file (out_of_memory) rather than failing the extractor it ran on. * Cleaner CAD text: STEP and IFC words no longer break where the file wrapped a line, GD&T callouts read as symbols and values, drawing fractions read as "2 1/2", title blocks on sheet-size borders fill the drawing's title and author, and large IFC models always list their full spatial structure first. Re-extract CAD files indexed earlier. * Files with a generic content type (application/octet-stream, common for S3 or MinIO uploads by scripts) are recognised by their contents, so scanned PDFs among them reach OCR instead of being skipped.