If something fails¶

The most common errors when using kubectl, k9s, Lens, Helm or the kubelatch CLI through kubelatch, and what to do about each one. Errors in a GitHub Actions job are in Credentials for GitHub Actions.

Every response from the proxy, including errors, carries the Audit-ID header: give it to an administrator (or look at it with kubectl -v=8) if you need them to investigate a specific request.

401 — not authenticated¶

• a kubelatch token is required …: your kubeconfig has no token, or it isn't kubelatch's. Check that you're pointing at the right file (KUBECONFIG or --kubeconfig).
• unknown credential: the token doesn't exist in kubelatch (for example, you typed it by hand and made a mistake, or it's from another instance).
• credential revoked: someone (you or an administrator) revoked it. Issue a new one: Expiry and revocation.
• credential expired: it passed its expiry date. Credentials don't reactivate; issue another: Get a credential.

403 — no permission¶

• the account is disabled: an administrator disabled your account (or a bot's). Ask them to reactivate it if appropriate.
• this credential is restricted to cluster ...: you issued it for a single cluster and are using it against another. Issue one with no cluster restriction, or one for that specific cluster.
• ... has no active permissions on cluster ...: you have no permission currently valid on that cluster (expired or revoked). The message ends with where to ask for one: Ask for access: and a link to kubelatch in Pro, or whom to ask.

• Forbidden from the cluster itself (without that kubelatch text in front): you have permissions, but your role doesn't cover that verb, resource or namespace. When kubelatch knows a role that would allow it, the message ends with that role and where to ask for it:

  Error from server (Forbidden): pods "web" is forbidden: User "user:alice" cannot create resource "pods/exec" in API group "" in the namespace "pagos" kubelatch: the debugger role in namespace pagos allows this. Ask for it: https://kubelatch.acme.com/request?cluster=prod&namespace=pagos&role=debugger&from=create+pods%2Fexec
  

  The link opens the request form with the cluster, the namespace and the role filled in (Ask for access). In Free the sentence names whom to ask instead of the link. Without one, check what your role allows in System roles, or, for a custom role, ask an administrator what it includes.

404 — unknown cluster¶

unknown cluster means the cluster id in your kubeconfig (https://<kubelatch>/clusters/<id>) doesn't exist in kubelatch, for example because an administrator deleted it. Check the context name with kubectl config get-contexts.

400 — invalid request¶

• kubelatch does not accept Impersonate-* headers (kubectl --as is not supported): you used --as or --as-group. kubelatch doesn't let you act as another user; you already act as your own. Remove the flag.
• invalid path: empty segments, '.' or '..' or escaped characters are not allowed: the request carries something encoded (%XX) or a double slash. Plain kubectl never generates this; check whether you're using curl or another tool with a badly encoded resource name.

502 and 503 — the cluster or kubelatch isn't responding¶

• 502 could not talk to the cluster's API server: kubelatch can't reach the real cluster. It's not a problem with your credential; tell an administrator.
• 503 the cluster has no tokens configured yet: the cluster is registered in kubelatch but isn't enrolled yet (its state reads Not enrolled): an administrator still has to run its enrollment command.
• 503 kubelatch is restarting: kubelatch is in the middle of a deployment. Retry in a few seconds; a watch retries on its own.

The kubelatch CLI¶

When the CLI's session is missing or over, kubectl fails with one of these messages (the CLI prints them, kubectl repeats them):

• no kubelatch session: run "kubelatch login": you haven't signed in on this machine, or you signed out. Run kubelatch login.
• the kubelatch session expired: run "kubelatch login": the 12 h session is over. Run kubelatch login; the kubeconfig doesn't change.
• the kubelatch session is no longer valid (revoked or expired on the server): run "kubelatch login": the session's credential was revoked from the web, or your account was disabled. Sign in again; if it keeps happening, ask an administrator.
• cannot listen on 127.0.0.1 for the browser …: use kubelatch login --device: the CLI can't open a local port for the browser to come back to. Use kubelatch login --device and type the code in a browser.
• no approval came from the browser within 5m0s …: nobody clicked Approve in time. Run kubelatch login again.
• the first login needs kubelatch's URL: kubelatch login https://kubelatch.example.com: this machine doesn't know your kubelatch yet. Give it the URL once.

The rest of the CLI is in The CLI.

GitHub login¶

If you can't sign in with GitHub, the login screen shows a specific message (you're not an active member of the organization, the account isn't linked, the organization doesn't enforce two-factor, the link expired...). The full list, with what to do in each case, is in Errors.

Signing in with your company account¶

If you can't sign in with your company account (Sign in with {name}, where {name} is the name your administrators gave the provider), the login screen shows a specific message: the account isn't in a domain this kubelatch accepts (you may have picked a personal account: Try another account), no kubelatch account has its e-mail address, your account is linked to another identity, the link expired... The full list, with what to do in each case, is in Errors, and what happens the first time is in Your first sign-in with your company account.

Certificate (x509) when connecting to a cluster or to kubelatch¶

If kubectl answers something like x509: certificate signed by unknown authority or certificate is not trusted:

• Against kubelatch: your kubeconfig doesn't carry the right CA. With the CLI, sign in again with kubelatch login <url> --ca-file <ca.crt>. With a kubeconfig downloaded from the web, add the CA to that file: kubectl --kubeconfig <downloaded-file> config set-cluster <id> --certificate-authority=<ca.crt> --embed-certs=true. An administrator can also configure kubelatch so new kubeconfigs already include the CA.
• Your kubeconfig talks only to kubelatch, never to the real Kubernetes cluster, so a certificate error from the cluster itself isn't something you can fix on your end: report it to an administrator.

If none of this fits, give an administrator the Audit-ID from the response (kubectl -v=8 shows it): with it they can find your request in Audit.