Accounts and recovery¶

What to do when someone is locked out of kubelatch, and when everyone is. Covers break-glass accounts and native cluster access for when kubelatch is down.

	Without an external login	With the GitHub App (GitHub login)	With an OIDC provider (OIDC login (SSO))
People	Username and password.	Sign in with GitHub. Only active members of the organization, with whatever 2FA it enforces.	Sign in with {name}. Only e-mail addresses the provider verifies, in the allowed domains, with whatever 2FA the provider enforces.
Passwords	All of them work.	Only break-glass accounts. The rest stay in the database but can't be used to sign in.	Only break-glass accounts, as with GitHub.
Onboarding	Create and invite: the person sets their own password.	Create and invite: the person links their GitHub account.	Create and invite: the person's first sign-in links the account by its e-mail address, or they use the invitation link.
Offboarding	Manual: Disable.	Automatic when they leave the organization, or Disable.	Manual: Disable. The provider gives no member list.
My account (click your name at the bottom of the sidebar)	Change password.	Link with GitHub, in Identities. Change password, only for break-glass accounts.	Link with {name}, in Identities. Change password, only for break-glass accounts.

On this page, {name} is the provider's name, OIDC_DISPLAY_NAME (SSO by default). GitHub and OIDC can be on together: the login page offers both, and each person can have either identity or both (GitHub and OIDC together).

Switching modes doesn't touch accounts. With GitHub, each one is linked separately with its own linking link; with an OIDC provider, most link themselves on their first sign-in. Bots never sign in: they only have credentials (Users and bots).

Without an external login there's neither two-factor nor automatic offboarding, and of the two external logins only GitHub disables people on its own: see Security model.

Account rules¶

Passwords: 12 to 128 characters, different from the login, no composition rules. Each person changes their own in My account by giving the current one; that closes their other sessions. Changing it is limited to 10 attempts per minute per account.
Sessions: a signed cookie valid for 12 h. All sessions close instantly on signing out, on a password change, on using a reset or linking link, on unlinking GitHub or the OIDC identity, on disabling the account, or on pressing Sign out web sessions on the account's page. That button closes web sessions only: a CLI session is a credential, and it is revoked in Credentials.
Lockout: 5 consecutive failures lock the account for 15 minutes (the fifth already responds 429). In addition, each IP (or each /64 on IPv6) gets 10 attempts per minute. Every attempt is logged in the control-plane audit log with the IP.
Last admin: kubelatch won't let you remove the role, disable, unlink, or unmark as break-glass the last admin who can still sign in.

Break-glass accounts¶

A break-glass account keeps its password when GitHub or OIDC login is on. It's for signing in when GitHub or the identity provider is down, the App has been uninstalled, or a configuration mistake (a mistyped domain, a wrong claim) keeps everyone else out. It can only be marked from the CLI. The commands use $MGMT_KUBECONFIG, the path to the management cluster's kubeconfig (see Install).

# mark an existing account
kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch exec -it deploy/kubelatch -- /kubelatch-server user set-break-glass <login>
# remove the mark
kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch exec -it deploy/kubelatch -- /kubelatch-server user set-break-glass <login> --off
# create a new break-glass account
kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch exec -it deploy/kubelatch -- /kubelatch-server user create rescue --admin --break-glass

Keep at least one, with its password in the organization's secrets manager. Don't use it day to day: it has no two-factor. In Users its Sign-in reads Password · break-glass, in amber, and the break-glass chip counts it. It signs in from Sign in with a password, at the foot of the login page. If you link it to GitHub and the person leaves the organization, it loses access like any other account. Linked to the OIDC provider, it keeps its password.

When a person is locked out¶

Situation	What to do	Who
Forgotten password	Users → the account's page → Reset access (a reset link, 24 h, single use). Share it over a secure channel. With GitHub or OIDC, this only applies to break-glass accounts.	An admin
Account locked from failed attempts	Users reads Locked until …. Wait 15 minutes, or Reset access, whose link unlocks it.	The person / an admin
GitHub account changed, lost, or linked by mistake	Users → the account's page → Linking link (24 h). If `github_taken` comes back, that GitHub account is already linked to someone else: Unlink GitHub on the other account first.	An admin
Linked to the wrong OIDC identity, or the person's account at the provider was recreated	Users → the account's page → Unlink {name}. The next sign-in with {name} links an identity again by e-mail if the link policy allows it; otherwise, generate a Linking link. `oidc_taken` means that identity already opens another account: Unlink {name} on that one first.	An admin
An e-mail address that went to someone else after its owner left (the newcomer sees `identity_mismatch`)	Don't unlink: that account is still its previous owner's. Disable it and create an account for the newcomer, who links on their first sign-in.	An admin
Can't sign in with GitHub	Read the login message. "Your GitHub account is not an active member of the organization.": missing from the organization, or the invitation is pending. "That GitHub account isn't linked to any kubelatch account. Ask an administrator for a linking link.": needs a linking link. "The GitHub organization doesn't require two-factor; an organization administrator must enable it before kubelatch accepts GitHub logins.": turn on 2FA in the organization. "Your account is disabled.": an admin re-enables it.	The person / an admin
Can't sign in through the OIDC provider	Read the login message; every one, with what to do about it, is in When a sign-in fails. The usual ones: "No kubelatch account has the e-mail address of that {name} account." (create the account, fill in its e-mail or, if the link policy doesn't cover it, generate a Linking link) and "That {name} account isn't in a domain this kubelatch accepts. Did you pick a personal account?" (Try another account, or add the domain to `OIDC_ALLOWED_DOMAINS`). "Your account is disabled.": an admin re-enables it.	The person / an admin
Link expired or already used	The person sees Link expired or already used, a `410` from the API. Generate another (Generate another link, in the account's Sign-in card): it voids the pending ones.	An admin
Person leaving the company	With GitHub, removing them from the organization is enough. With OIDC alone, or to get ahead of it: Disable; disabling them in the provider only stops their next sign-in. Nothing re-enables it on its own even if they come back.	An admin / GitHub
Lost or leaked credential	Revoke in Home, by the person themself, or in Credentials, by an admin. If all you have is the token, paste it whole in Credentials (Find a leaked token): it opens that credential, with Revoke at the foot. The next request gets `401` and open streams are cut within 10 s. Issue another.	The person / an admin
Compromised bot	Revoke its credentials or disable the bot.	An admin

The kubelatch-server user CLI is the rescue path. It runs with the same configuration as the server, applies pending migrations, and logs what it does in the control-plane audit log with no actor. Always use it with the management cluster's explicit kubeconfig.

New password for an existing admin. This also unlocks them, closes their sessions, and voids their pending links. It doesn't re-enable a disabled account.
```
kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch exec -it deploy/kubelatch -- /kubelatch-server user set-password admin
```

With GitHub or OIDC enabled, that password only works if the account is a break-glass account. The CLI warns if it isn't. Mark it:

kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch exec -it deploy/kubelatch -- /kubelatch-server user set-break-glass admin

If the only admin is disabled, or there isn't one, create another and re-enable the first one from Users:

kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch exec -it deploy/kubelatch -- /kubelatch-server user create rescue --admin --break-glass

Without a terminal (a runner, a script), use --password-stdin with kubectl exec -i. Exactly one trailing newline is stripped: a password that ends in a newline should be passed with two.

printf '%s' '<long password>' | kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch exec -i deploy/kubelatch -- /kubelatch-server user set-password admin --password-stdin

With two replicas it doesn't matter which one runs it: the CLI talks directly to Postgres and there's no need to stop the server.

If the pod won't start (Postgres down, broken configuration), the problem isn't accounts. Check the logs and the pod's status:

kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch logs deploy/kubelatch
kubectl --kubeconfig "$MGMT_KUBECONFIG" -n kubelatch describe pod

Native emergency access¶

kubelatch sits on the path of every request people make, but it is never the only way into a cluster, and workloads don't depend on it. Before going to production, set up and test native administrator access:

Managed clusters: the cloud provider's identity (aws eks update-kubeconfig, gcloud container clusters get-credentials, az aks get-credentials) with the platform administrator role, in the organization's identity manager and with MFA.
Self-managed: the administrator kubeconfig (kubeadm: /etc/kubernetes/admin.conf; k3s: /etc/rancher/k3s/k3s.yaml; Talos: talosctl kubeconfig) in a vault with audited access, or SSH to a control-plane node.

That access is for repairing kubelatch (for example, re-applying the bootstrap), acting when kubelatch is down, and doing what kubelatch doesn't allow: protected namespaces, nodes, --as. Its use is logged in the cluster's native audit log, not kubelatch's: review it with the same discipline.

When kubelatch is working, prefer a cluster-admin permission for at most 8 h, which does get audited: Grant permissions.

If the encryption key is lost¶

Without KUBELATCH_ENCRYPTION_KEY, everyone has to sign in again and every cluster's tokens have to be rotated. Passwords, klt_ credentials, permissions, and the audit log aren't affected. The procedure is in Upgrades and backups.