The journey of a request¶

Follow a kubectl request from when it leaves your machine until the response comes back. This shows you what kubelatch checks, what the cluster sees, and what ends up in the audit log.

At a glance¶

sequenceDiagram
    participant K as kubectl
    participant P as kubelatch proxy
    participant DB as Postgres
    participant A as API server
    K->>P: GET /clusters/prod/api/v1/namespaces/apps/pods<br/>Authorization: Bearer klt_…
    P->>P: Route and Impersonate-* headers
    P->>DB: Looks up the token hash
    P->>DB: Subject, cluster and active permissions
    P->>DB: Inserts the audit row (start)
    P->>A: Same request with Impersonate-User,<br/>Impersonate-Group, Impersonate-Uid, Audit-ID
    A-->>P: Response (cluster's native RBAC)
    P-->>K: Response + Audit-ID
    P->>DB: Completes the row (status, duration)

1. The request reaches the proxy¶

Your kubeconfig points to https://<kubelatch>/clusters/<id> and carries a klt_… token as Bearer. Everything under /clusters/ goes to the proxy before any other route.

Every proxy response carries an Audit-ID header, errors included. It's what you look up in the logs or in the audit log to locate a specific request.

2. First filter: route and headers¶

Before looking at the token, the proxy rejects with 400:

• Routes with empty segments, . or .., escaped characters or a %.
• A cluster identifier that can't exist.
• Any Impersonate-* header coming from the client. This is why kubectl --as doesn't work through kubelatch: the proxy already uses impersonation, and letting a nested impersonation through would let someone ask to be another person.

These rejections leave no audit row: the request never gets far enough to identify anyone.

3. Who are you? The token¶

The proxy only accepts Authorization: Bearer klt_… matching the exact format of a kubelatch token. It ignores cookies. Then:

1. It computes the token's SHA-256 and looks up the credential by that hash. There's no cache: every request queries Postgres, so a revocation takes effect on the very next request.
2. It checks the credential isn't revoked or expired. If either check fails, or the token doesn't exist, it responds 401.
3. It loads the subject that owns the credential. If the account is disabled, it responds 403.
4. If the credential is restricted to a cluster and this isn't it, it responds 403.

4. What can you do here? Cluster and permissions¶

The proxy loads the cluster. If it doesn't exist, it responds 404; if it doesn't have tokens configured yet, 503.

Then it looks up the subject's active permissions on that cluster: not revoked and not expired at this instant. If there are none, it responds 403 with a clear message. Each permission translates into a group:

An expiry doesn't need anyone to do anything: as soon as a permission expires, the proxy stops sending its group.

5. The audit row, before forwarding¶

With the identity and groups resolved, the proxy classifies the request the way the API server would: verb, API group, resource, subresource, namespace and name. It inserts the audit row before forwarding. If it can't write it, it responds 503 and forwards nothing: no request goes through without its record.

It also updates the credential's Último uso (Last used), at most once per minute.

6. What the API server receives¶

The proxy rewrites the request before forwarding it:

The API server authenticates the proxy's ServiceAccount, checks that it can impersonate that user and those groups, and applies its native RBAC to the impersonated identity. kubelatch doesn't add system:authenticated: the API server itself does that.

The API server uses the Audit-ID header as the auditID of its own record. This way a kubelatch row and the cluster's native audit event correlate through the same identifier. In the native audit log, user is the proxy's ServiceAccount and impersonatedUser is user:<login>.

7. The response comes back¶

The proxy returns the API server's response as-is, with its own Audit-ID. If it can't reach the API server, it responds 502 with a summary of the problem (timeout, certificate mismatch, unreachable API server), without leaking internal addresses.

When it finishes, it completes the audit row with the status code, the duration, the end time and the error if there was one. A row can only be completed once.

Long-lived streams: exec, port-forward, watch and logs¶

kubectl exec, attach, port-forward and cp open a connection that upgrades to WebSocket or SPDY. The proxy sends these over a dedicated HTTP/1.1 transport, because the API server doesn't perform that upgrade over HTTP/2. All other requests go over HTTP/2. watch and logs -f are forwarded unbuffered, byte by byte.

A stream's audit row is opened when it starts (101 for upgraded connections) and completed when it closes, with its real duration.

Revoking cuts what's already open¶

Revoking a credential rejects the next request with 401. But an open exec or watch doesn't make new requests. That's why each replica keeps a record of its in-flight requests and revalidates each one every 10 s:

• Is the credential still unrevoked and unexpired?
• Is the account still enabled?
• Are the groups still exactly the same? Adding or removing a permission on that cluster also cuts the stream, and the client reconnects with the new groups.

Whatever's no longer valid is cut within 10 s at most. If the cut happens before the API server responds, the client gets a 403; if the stream was already open, the connection is closed. The audit row ends up with the error stream cortado: la credencial ya no es válida (stream cut: the credential is no longer valid).

If Postgres fails during that revalidation, open streams stay open. New requests do fail closed.

What gets logged when something fails¶

Rejection rows are limited to one per IP every 10 s, so no one can fill the table by probing tokens. The response to the client doesn't change because of that limit. The full list of codes is in Errors.