Security model¶

What kubelatch guarantees, what it doesn't, and what each guarantee rests on. Read this before granting permissions or putting kubelatch into production.

What it protects and what it doesn't¶

kubelatch controls who reaches the API servers through it, with what identity, and what gets logged. Specifically:

• Every request carries its own credential, revocable instantly and with an expiry.
• The cluster sees the real person (user:<login>) and only the groups from their active permissions.
• Every request is logged before it reaches the cluster, with an identifier that correlates it with the cluster's native audit log.
• The proxy's ServiceAccount can't impersonate anyone outside what kubelatch has granted.

kubelatch does not protect:

• What happens inside a namespace that's already been granted: the real boundary is the namespace (see Tiers).
• Native access to clusters (cloud identities, admin kubeconfigs, ServiceAccount tokens pulled from a pod). That stays in the cluster's native audit log, not kubelatch's.
• The database against someone who holds its Postgres role, or the encryption key against someone who can read the Secrets in the management cluster.
• Workloads: kubelatch isn't in their path.

Credentials: klt_ tokens¶

A credential is an opaque token: klt_ followed by 32 random bytes in base64url.

• kubelatch stores only its SHA-256 and the first 12 visible characters (klt_ + 8), which are enough to recognize it in the inventory and in the audit log without being able to use it.
• The token and the kubeconfig are shown only once, in a response with Cache-Control: no-store. The interface doesn't store them in browser storage or in the URL.
• There's no authorization cache: every request to the proxy queries the credential, so revoking it takes effect on the next request. Open streams are cut within 10 s at most.
• Expiry is mandatory: 30 days maximum for people and 90 for bots by default (KUBELATCH_MAX_TTL_USER, KUBELATCH_MAX_TTL_BOT). CI credentials last 1 h by default (KUBELATCH_CI_TOKEN_TTL).

A fast hash is enough because the token has 256 bits of entropy: there's no dictionary to try. Invitation and linking links (kli_…) follow the same scheme, single-use and with an expiry.

The proxy and its ServiceAccount¶

• No cookies. The proxy only accepts Authorization: Bearer klt_… and ignores cookies, so a web page can't use your session against a cluster.
• No client Impersonate-*. Any incoming Impersonate-* header is rejected with 400. kubelatch already uses impersonation with the kubelatch-proxy ServiceAccount; letting a nested impersonation through would let anyone ask to be someone else. That's why kubectl --as isn't supported. To test "what can X do", use native admin access: kubectl auth can-i --as user:x --as-group kubelatch:ns:apps:viewer.
• Its own Audit-ID. The proxy generates its own and never honors the client's.
• Strict routes. It rejects ., .., empty segments and any escaping, so nothing gets decoded twice along the way.
• Scoped ServiceAccount. kubelatch-proxy can only impersonate the users and groups in the list the reconciler maintains (resourceNames): subjects with active permissions and the groups from those permissions. A failure in the proxy can't reach system:masters or anyone kubelatch hasn't granted something to.
• Prefixed names. user: and bot: avoid collisions with the cluster's native identities.

The kubelatch-reconciler ServiceAccount is equivalent to cluster-admin by design: it has to be able to create the cluster-admin binding. Its writes are limited to the fixed names kubelatch manages, which bounds mistakes, not privilege. It's never in the path of a client request.

The namespace boundary¶

Inside a namespace that's already granted, kubelatch can't separate anything: whoever creates pods reaches its Secrets and its ServiceAccounts. That's why developer, debugger and admin require Pod Security Admission (baseline or restricted), and kube-system, kube-public, kube-node-lease, kubelatch-system and those in KUBELATCH_PROTECTED_NAMESPACES accept no permissions at all. viewer with * scope reads the logs of every namespace, and logs can contain secrets. All of this is explained in Permission tiers.

Optional hardening: bindings to kubelatch-*¶

A namespace admin, whether granted by kubelatch or native, can create a RoleBinding to kubelatch-developer for whichever subject they want. That binding doesn't grant access through kubelatch, because the proxy only impersonates whoever has valid permissions. But it would grant it to a native credential of that subject, and it confuses the reconciler, which logs bindings it doesn't own under names it manages.

deploy/k8s/hardening/vap-kubelatch-bindings.yaml is a ValidatingAdmissionPolicy (Kubernetes 1.30+) that only lets the kubelatch-system/kubelatch-reconciler ServiceAccount create or change bindings whose roleRef starts with kubelatch-. It's applied on each managed cluster, with that cluster's own kubeconfig, not the management cluster's. make e2e-install checks that it denies a manual binding and that the reconciler keeps working. The policy also denies an administrator the bootstrap's own bindings: moving the cluster to another instance or re-registering it requires removing it for a moment (Hardening the cluster).

Cluster tokens, encrypted¶

The tokens for each cluster's two ServiceAccounts are the keys to the kingdom. kubelatch stores them encrypted with AES-256-GCM, with a key derived by HKDF from KUBELATCH_ENCRYPTION_KEY, and never displays them. Each token is bound to its cluster and to its role (proxy or reconciler), so an encrypted token can't be moved from one cluster to another.

The same key, with other derivation labels, signs session cookies and the state cookies for GitHub login. If it's lost or rotated, everyone has to sign in again and the tokens for each cluster have to be pasted back in. Passwords and klt_ credentials don't depend on it.

Trusted proxies and the source IP¶

The audit log's source IP and the three per-IP rate limiters (login, CI exchange and proxy authentication failures) use the TCP connection's address. Behind an Ingress or another L7 proxy, that address is the proxy's.

KUBELATCH_TRUSTED_PROXIES says which networks X-Forwarded-For is trusted from, read right to left up to the first untrusted hop. It only accepts CIDR: a bare IP prevents startup, and so does a prefix wider than /8 for IPv4 or /16 for IPv6.

• Don't put in ranges a client could reach: they could choose the IP that appears in the audit log and dodge the limits.
• Behind an Ingress, only trust the controller pods' IPs. A CIDR covering the whole cluster's pods turns every pod into a trusted proxy. That's why the ingress overlay ships a mandatory NetworkPolicy that only admits traffic from the controller's namespace, and needs a CNI that enforces NetworkPolicy.
• Behind an L4 LoadBalancer with externalTrafficPolicy: Local, leave it empty.

The rate limiters group IPv6 by /64, so rotating addresses within the prefix doesn't multiply the attempts. With two replicas, each one counts its per-IP limits in memory; the account lockout after 5 failures is global.

Passwords and sessions¶

• Passwords from 12 to 128 characters, different from the login, with no composition rules. argon2id hashing with OWASP's parameters, at most 4 computations at a time.
• 5 consecutive failures lock the account for 15 minutes, and each IP gets 10 attempts per minute. The response is the same for a nonexistent login, a wrong password or a disabled account, and it takes the same time.
• Every login attempt, successful or not, is logged in the control plane's log with the IP. The exception is a login longer than 64 characters: it gets the same 401 without reaching the database or being logged.
• Changing your own password in Mi cuenta (My account) has its own limit: 10 attempts per minute per account, and every failure is logged.
• 12 h sessions in a signed cookie, revocable all at once. API mutations are protected against cross-origin requests.

After 5 failures, the 429 confirms that the login exists, and anyone can lock an account for 15 minutes. This is accepted for an internal tool; the per-IP limit makes it costlier.

Identity with GitHub¶

With the GitHub App configured, people's identity is GitHub's. The flow's details are in Identity; here's what matters for security:

• Two-factor. kubelatch doesn't implement it: it relies on whatever the organization enforces. It can only check this if the App has Organization → Administration: read; without that permission, it warns and trusts. Turn on mandatory 2FA in GitHub before anything else.
• Passwords. With GitHub active, only break-glass accounts sign in with a password, and they have no two-factor. Keep them few, with long passwords in a manager, and use them only when GitHub isn't available. The hashes for other accounts stay in the database but can't be used to sign in.
• Offboarding. Whoever leaves the organization gets disabled automatically (HMAC-signed webhook or hourly sync). The sync never disables on a GitHub error and never re-enables. The credentials' maximum TTL limits what a prolonged failure of both paths could leave open.
• The OAuth flow. A random state in a signed cookie valid for 10 minutes, HttpOnly, SameSite=Lax and scoped to the callback route. The code is exchanged server-side with the client secret and the person's token is discarded after login. PKCE isn't used: GitHub doesn't document it for GitHub Apps, and kubelatch is a confidential client, for which state plus client secret is what the OAuth 2.0 BCP requires. The App's installation tokens live only in memory.
• Races. Linking is a compare-and-swap on the GitHub id: a login that races against an unlink or relink of the same account doesn't get a session. An internal callback error redirects with ?error=server, with no details.
• Webhook with no deduplication. A signed, repeated delivery can disable someone an admin re-enabled after they rejoined the organization. The account stays disabled until an admin re-enables it again, with new credentials because the old ones were revoked: the hourly sync never re-enables, so it doesn't fix this offboarding. Guard GITHUB_WEBHOOK_SECRET like any secret and rotate it if it leaks. If someone who rejoined shows up disabled, check Usuarios (Users) and the user.disable control-plane events with via: github-webhook.

Session and state cookies don't carry the __Host- prefix; they're scoped to their path and are HttpOnly.

Without GitHub¶

Without the GITHUB_* variables, local login has neither two-factor nor automatic offboarding. A stolen password gives access to clusters with that person's permissions. Active mitigations: 12 h sessions, lockout on attempts and per-IP limit, credentials with an expiry, instant revocation and audit of every login. This is acceptable for a test or a GitHub-free environment. In production, configure the GitHub App and, meanwhile, disable the accounts of people who leave on the same day.

Secret-free CI credentials¶

A GitHub Actions workflow doesn't store any kubelatch secret: it exchanges an OIDC id_token that GitHub signs for that run (CI: trust rules). Verification is strict:

• Signature. RS256 only, with the public keys from <issuer>/.well-known/jwks (KUBELATCH_GITHUB_ACTIONS_ISSUER). kubelatch caches them for 1 h; an unknown key forces a refresh, at most once per minute. If GitHub doesn't respond, it keeps using the cached ones for up to 24 h after the last successful refresh. Past that window, all exchanges fail with 401.
• Claims. Exact iss, aud equal to KUBELATCH_BASE_URL, and exp, nbf and iat with a 30 s margin for clock skew.
• Single use. The jti is stored on exchange; a reused token is rejected with 401.
• Ids, not names. Trust rules compare repository_owner_id, repository_id, ref and environment. kubelatch doesn't interpret sub: it accepts both the classic repo:owner/name:… format and the immutable-id one.

Any token failure responds with the same 401 token no válido (invalid token); the reason only appears in the log and in the ci.exchange.failure event.

Immutable audit log¶

kubelatch writes two records to Postgres:

• audit_events: one row per request to the proxy, inserted before forwarding it and completed when it finishes.
• control_events: every control-plane action (creations, permissions, credentials, revocations, account changes, trust rules) in the same transaction as the action, and every login attempt.

Postgres triggers prevent modifying them: an audit_events row can only be completed once, control_events is never modified, and TRUNCATE is forbidden on both. Deletion is only allowed for the periodic retention job: it deletes in batches of 5,000 rows and, for each batch, sets the SET LOCAL kubelatch.audit_retention = 'on' switch that the triggers require for a DELETE. Rows have no foreign keys: deleting a cluster or a subject doesn't delete its history.

Retention is set with KUBELATCH_AUDIT_RETENTION, 90 days by default (see Audit and retention).

The cluster's native audit log, a second source¶

Keep each cluster's native audit log active. It records what doesn't pass through kubelatch (native access, ServiceAccount tokens used from pods) and is an independent second source. The API server uses kubelatch's Audit-ID as its own auditID, so a kubelatch row and its native event can be found through the same identifier.

Personal data¶

audit_events stores, per request, the IP, the login, the User-Agent and the query string. In an exec, the query contains command=, that is, what the person ran. control_events stores the IP and login of every login attempt.

• Purpose: security and traceability of access to clusters.
• Retention: KUBELATCH_AUDIT_RETENTION (90 days by default).

What kubelatch doesn't cover¶

Session recording, ingesting a cluster's native audit log, a relay for private clusters, namespace and quota management, kubectl --as, identity providers other than GitHub, and per-credential rate limiting. It also doesn't replace native emergency access to each cluster, which must exist and be tested (see Accounts and recovery).