Connect your AI to KDBL Context Lake (K-Lake)¶

Give a cloud AI assistant — Claude, Microsoft Copilot 365, Google Gemini Enterprise, or ChatGPT — secure, governed access to your on-prem K-Lake over the Model Context Protocol (MCP).

No inbound firewall change. K-Lake stays inside your network; a lightweight tunnel dials out to present a stable HTTPS endpoint the AI can reach. Users sign in with your own identity provider (Entra ID / Okta / Auth0 / Google), and every answer is trimmed to what that user is allowed to see.

Claude / Copilot / Gemini ──HTTPS──►  <you>.mcp.kdbl.co.uk        (public, kdbl-managed)
                                          │  outbound-only tunnel
                                          ▼
                                   k-lake MCP gateway  ─►  k-lake (on-prem)
                                          ▲
                              sign-in federated to YOUR IdP

You can connect in three contexts, in increasing order of production-readiness:

Local evaluation — a desktop/CLI Claude on the same machine, PAT-authenticated over localhost. No IdP, no public hostname, nothing to provision. See Evaluate locally first.
kdbl-managed (recommended for production) — a branded https://<you>.mcp.kdbl.co.uk, turnkey: paste one token and federate sign-in to your IdP. The flow in §1–5 below.
Your own connectivity — bring-your-own Cloudflare, an API gateway you already operate, or a direct firewall mapping. See Connectivity options.

Evaluate locally first (no IdP, no tunnel)¶

Before wiring an IdP and a public hostname, connect a local Claude — Claude Code (CLI) or the Claude Desktop app, running on the same machine as the stack (e.g. a POC) — straight to K-Lake with a PAT, over plain localhost. Fastest way to see grounded answers. It is not for production: a PAT carries that user's full visibility, and cloud/web clients can't reach localhost.

Cloud clients (claude.ai, ChatGPT web, Copilot, Gemini) live on the internet and cannot reach localhost — they need a public path from Connectivity options. Only a desktop/CLI client on the same host uses this local path.

1. Enable MCP on the API (it ships dark). Set the master switch and the resource URI to the address the client will actually call:

kubectl -n kdbl set env deploy/kdbl-api \
  KDBL_MCP_ENABLED=true \
  KDBL_MCP_RESOURCE_URI=http://localhost/mcp
kubectl -n kdbl rollout status deploy/kdbl-api

The embedded OAuth server stays dark when no signing keys are mounted — expected here, and /mcp still authenticates with a PAT. The boot log prints MCP endpoint enabled.

2. Expose /mcp to the host. The POC ingress proxies only / (UI) and /api/, not /mcp — add a route so traefik serves it on the host's port 80. This is stable and, unlike kubectl port-forward, survives pod restarts and laptop sleep:

kubectl -n kdbl apply -f - <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata: { name: kdbl-mcp, namespace: kdbl }
spec:
  ingressClassName: traefik
  rules:
  - http:
      paths:
      - path: /mcp
        pathType: Prefix
        backend: { service: { name: kdbl-api, port: { number: 80 } } }
EOF

Port-forward alternative: kubectl -n kdbl port-forward svc/kdbl-api 18080:80, then use http://localhost:18080/mcp everywhere below and set KDBL_MCP_RESOURCE_URI to match. Keep the forward running; on Docker Desktop the apiserver TLS handshake can need a retry to establish (net/http: TLS handshake timeout — retry, the cluster is fine). The ingress route above avoids this.

3. Verify the endpoint accepts your PAT (a tenant-user kdblpat_… — e.g. the one poc-up.sh / kdbl-control onboard printed):

curl -s -X POST http://localhost/mcp \
  -H "Authorization: Bearer $KDBL_PAT" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'
# → {"result":{...,"serverInfo":{"name":"kdbl-mcp",...}}}

4. Connect your local Claude:

Claude Code (CLI):

claude mcp add --transport http k-lake http://localhost/mcp \
  --header "Authorization: Bearer $KDBL_PAT"
claude mcp list      # shows k-lake; its tools appear in your NEXT session (/mcp)

Claude Desktop (needs Node/npx): add to ~/Library/Application Support/Claude/claude_desktop_config.json, then fully quit and reopen the app:

{
  "mcpServers": {
    "k-lake": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "http://localhost/mcp", "--header", "Authorization:${AUTH_HEADER}"],
      "env": { "AUTH_HEADER": "Bearer kdblpat_…" }
    }
  }
}

(The Authorization:${AUTH_HEADER} no-space form plus the env var avoids an mcp-remote argument-splitting bug.)

Ask: "Use k-lake to search my files for … and quote the source." The tools (search_content, get_file_text, list_sources, …) return cited results from your indexed content.

1. Enrollment (kdbl-managed) — we provision your hostname¶

Your KDBL contact runs one command, which provisions a dedicated outbound tunnel + DNS for your branded hostname and returns a tunnel token:

kdbl-control mcp enroll <your-slug>
# → KDBL_MCP_RESOURCE_URI = https://<your-slug>.mcp.kdbl.co.uk/mcp
# → <tunnel token>

(The *.mcp.kdbl.co.uk TLS certificate already covers your hostname — nothing to request. The tunnel is scoped so only the MCP + sign-in paths are exposed; the K-Lake admin API is never reachable from the internet.)

You receive: the tunnel token and your KDBL_MCP_RESOURCE_URI.

Off-boarding (operator): kdbl-control mcp disenroll <your-slug> deletes the tunnel + DNS and cuts access instantly. Idempotent.

2. Deploy on-prem¶

In your K-Lake cluster:

# a) the tunnel token (connects your cluster out to your hostname)
kubectl -n kdbl create secret generic kdbl-mcp-tunnel \
  --from-literal=token='<tunnel token from step 1>'

# b) the OAuth signing key k-lake mints its access tokens with (ES256, shared
#    across replicas). Generate once; keep it safe.
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out k1.pem
kubectl -n kdbl create secret generic kdbl-mcp-oauth-signing --from-file=k1.pem

# c) your IdP federation secret (the app secret from step 3)
kubectl -n kdbl create secret generic kdbl-mcp-oauth-federation \
  --from-literal=client_secret='<your IdP app secret>'

Set these in the K-Lake config (ConfigMap kdbl-api-config):

KDBL_MCP_ENABLED            = true
KDBL_MCP_RESOURCE_URI       = https://<your-slug>.mcp.kdbl.co.uk/mcp
KDBL_MCP_OAUTH_FED_ISSUER   = <your IdP issuer>      # e.g. https://login.microsoftonline.com/<tid>/v2.0
KDBL_MCP_OAUTH_FED_CLIENT_ID= <your IdP app client id>
KDBL_MCP_OAUTH_FED_SCOPES   = openid profile email   # + a groups claim (see §5)

Then apply the MCP tunnel + gateway manifests and roll the K-Lake API. Verify:

curl https://<your-slug>.mcp.kdbl.co.uk/.well-known/oauth-authorization-server   # 200
curl -X POST https://<your-slug>.mcp.kdbl.co.uk/mcp                              # 401 (needs sign-in)

3. Register K-Lake in your IdP (one app)¶

K-Lake acts as an OAuth client to one app registration in your IdP — the AI clients self-register with K-Lake (DCR), so there's nothing per-client to set up in your IdP. In every IdP the app needs:

Redirect / callback URI: https://<your-slug>.mcp.kdbl.co.uk/oauth/callback
A client secret → the kdbl-mcp-oauth-federation Secret.
The OIDC scopes below, and (for group-based access, §5) a groups claim in the id_token.

Then set the four KDBL_MCP_OAUTH_FED_* env vars. The _GROUPS_CLAIM is the name of the groups claim in the upstream id_token — K-Lake reads it and re-stamps it onto its own token, so your K-Lake tenant's groups_claim is always just groups regardless of IdP.

Microsoft Entra ID (validated)¶

App registration → Authentication → Web redirect = the callback; Certificates & secrets → new secret; Expose-an-API not needed.
Token configuration → add a groups claim (id token). Manifest: requestedAccessTokenVersion: 2 (else you get v1 tokens with the wrong iss).
FED_ISSUER=https://login.microsoftonline.com/<tenant-id>/v2.0 · FED_SCOPES="openid profile email" · FED_GROUPS_CLAIM=groups

Okta¶

Applications → create OIDC Web app; Sign-in redirect URI = the callback; copy client id + secret.
Add a groups claim to the id_token (Security → API → Authorization Servers → default → Claims → add groups, filter e.g. Matches regex .*), or use the built-in groups scope.
FED_ISSUER=https://<your-okta-domain>/oauth2/default (org-level: drop /oauth2/default) · FED_SCOPES="openid profile email groups" · FED_GROUPS_CLAIM=groups

Auth0¶

Applications → Regular Web Application; Allowed Callback URL = the callback.
Auth0 omits groups by default and requires namespaced claims — add a Login Action: api.idToken.setCustomClaim('https://kdbl/groups', event.authorization.roles).
FED_ISSUER=https://<your-tenant>.auth0.com/ (keep the trailing slash — it's in the token's iss) · FED_SCOPES="openid profile email" · FED_GROUPS_CLAIM=https://kdbl/groups

Google (Workspace / Gemini Enterprise side)¶

Google Cloud Console → Credentials → OAuth client ID (Web); Authorized redirect URI = the callback.
Google id_tokens carry no groups — sign-in works, but group-based access needs K-Lake's directory sync (Workspace groups via the Admin SDK) or email/domain-based grants. See Directory sync.
FED_ISSUER=https://accounts.google.com · FED_SCOPES="openid email profile" (groups via directory sync, not the token)

4. Connect each AI client¶

All clients use the same URL and the same flow: they discover K-Lake's OAuth, self-register, show you a K-Lake consent screen, send you to your IdP to sign in, then call the tools. No client IDs or secrets to paste.

Server URL for every client: https://<your-slug>.mcp.kdbl.co.uk/mcp

Client	Where to add it
Claude (Team/Enterprise)	Settings → Connectors → Add custom connector → paste the URL → Connect.
ChatGPT (Enterprise/Team)	Settings → Connectors → add an MCP server → paste the URL.
Microsoft Copilot 365	Copilot Studio / agent builder → add an MCP server (tool) → paste the URL.
Google Gemini Enterprise	Agentspace / Gemini Enterprise → add an MCP connector → paste the URL.

On first connect you'll see "Connect to your K-Lake" → Allow access → your IdP sign-in → the assistant lists the K-Lake tools (search / read). Done.

5. Grant access to content (important)¶

Signing in does not by itself grant access to data. A freshly signed-in user authenticates but sees nothing until they're granted access to a source — this is the point: K-Lake is least-privilege by default.

Grant access by IdP group, so membership drives access automatically:

Have your IdP emit a groups claim (Entra: Token configuration → add groups; Okta/Auth0/Google: add a groups/roles claim) — set oidc_config.groups_claim accordingly.
Grant a group a role on a source (viewer/editor/owner). Members then inherit access; results are still per-file trimmed by the document ACLs on top.

For on-prem file shares whose ACLs are Windows/AD groups, map your IdP group IDs to the on-prem SIDs (K-Lake's directory sync — kdbl-control directory sync-graph for Entra). See Per-file security trimming.

Connectivity options (production)¶

For real use, cloud AI clients must reach K-Lake over a public HTTPS endpoint. That endpoint always terminates at the in-cluster MCP gateway (kdbl-mcp-gateway.kdbl.svc:80), which serves only /mcp and the sign-in/OAuth paths — the admin API is never exposed. How you publish it is your choice; the options trade convenience against how much network infrastructure you own. Sections 1–5 above assume option A; for B–D only the exposure changes — IdP federation (§3), client setup (§4), and access grants (§5) are identical.

A. kdbl-managed tunnel (recommended; default)¶

An outbound-only Cloudflare tunnel to a branded https://<you>.mcp.kdbl.co.uk that KDBL provisions and operates (the kdbl-control mcp enroll flow in §1). No inbound firewall change, no public IP, no certificate to manage — the on-prem side dials out. Best for almost everyone; choose another option only if policy forbids a third-party-operated tunnel.

B. Bring-your-own Cloudflare (your org + hostname)¶

Same outbound-tunnel model, but in your Cloudflare Zero Trust org on a hostname you own. Create a named tunnel, route your hostname → http://kdbl-mcp-gateway.kdbl.svc:80, run the connector (cloudflared) as a Deployment in the cluster, and set that hostname as KDBL_MCP_RESOURCE_URI. You own the DNS, the tunnel credential, and the audit trail; KDBL operates nothing. The tunnel/gateway are provider-agnostic — ngrok, Tailscale Funnel, or a future kdbl-operated relay work the same way. Keeps the no-inbound-port property of A.

C. API gateway / reverse proxy you already run¶

If you terminate external traffic at an existing gateway — Azure Application Gateway / APIM, AWS ALB + API Gateway, Cloudflare/Akamai, NGINX/Envoy, Apigee — publish a route there:

Host/path: https://<your-host>/mcp → upstream kdbl-mcp-gateway.kdbl.svc:80 (or the kdbl-api Service directly if you skip the gateway pod).
Forward the Authorization header unchanged, and don't strip the WWW-Authenticate 401 challenge — clients need it for OAuth discovery.
Set KDBL_MCP_RESOURCE_URI to the public URL; add the host's browser origin to KDBL_MCP_ALLOWED_ORIGINS if a browser-based client connects.
Terminate TLS at the gateway. This typically does mean an inbound listener, so pair it with the WAF / allowlist / mTLS your gateway already enforces.

Good when corporate policy requires all ingress to pass a sanctioned gateway with its own WAF, logging, and rate limiting.

D. Direct firewall port-mapping (simplest; highest exposure — use with care)¶

Map an inbound public port straight to the cluster ingress (NAT :443 → the traefik/MCP gateway, or a published k8s LoadBalancer/NodePort). Fastest to stand up, but it opens an inbound hole and you own everything in front of it. If you choose it:

Terminate real TLS (a valid cert for your host) — never expose /mcp in cleartext to the internet; bearer tokens ride the Authorization header.
Expose only /mcp + the OAuth/sign-in paths. Never publish /api/ or any admin route — keep them on a separate, internal-only ingress. The bundled MCP gateway already restricts paths; if you point the rule at kdbl-api directly, enforce the path allowlist at the firewall/reverse proxy yourself.
Require OAuth/IdP auth (§3) on an internet-facing port — never PAT auth. A PAT is a long-lived bearer carrying that user's full visibility; a leaked PAT on a public endpoint is a full data breach. IdP tokens are short-lived and audience-bound to your host.
Restrict the source: allowlist the AI vendor's published egress ranges, put a WAF / rate-limit in front, and alert on the MCP audit log.
Set KDBL_MCP_RESOURCE_URI to the public https://…/mcp and populate KDBL_MCP_ALLOWED_ORIGINS for any browser client.

Security trade-off in one line: A and B keep zero inbound ports (outbound dial-out) and are the safe default; C reuses controls you already operate; D is the most exposed — acceptable only with TLS, IdP-only auth, a strict path allowlist, and source restrictions all in place.

Security at a glance¶

Outbound-only — no inbound ports; the on-prem side dials out.
Your IdP is the source of truth — K-Lake federates sign-in to it; access tokens are short-lived and audience-bound to your hostname.
Scoped exposure — only /mcp + the sign-in/OAuth paths are public; the admin API is not.
Least privilege — no data access without an explicit (group) grant, and per-file ACL trimming on every result.
Per-tenant isolation — your hostname, tunnel, and token are yours alone; revoking the tunnel cuts access instantly.

See also the MCP server reference for the full auth model, env vars, and audit, and Connecting AI clients for per-client setup.