How to Configure GKE Break Glass Access
You set up gke break glass access by binding emergency user roles in Kubernetes RBAC. Teams grant these temporary roles only when an urgent production outage occurs. Many teams start by giving on-call engineers permanent cluster admin rights for speed. Broad standing permissions create high risk because one bad command can break the whole cluster. A good emergency setup keeps daily rights small and raises them only during a crisis. How gke break glass access works in Kubernetes RBAC Kubernete
Your coding agents run with your access.
Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.
You set up gke break glass access by binding emergency user roles in Kubernetes RBAC. Teams grant these temporary roles only when an urgent production outage occurs. Many teams start by giving on-call engineers permanent cluster admin rights for speed. Broad standing permissions create high risk because one bad command can break the whole cluster. A good emergency setup keeps daily rights small and raises them only during a crisis.
How gke break glass access works in Kubernetes RBAC
Kubernetes RBAC uses Roles and ClusterRoles to hold sets of permission rules. Every rule in this system is additive, and there are no deny rules. A standard Role sets permissions inside one single namespace for your team. A ClusterRole is not namespaced and covers resources across the whole cluster. It can also control cluster-scoped objects like nodes and non-resource endpoints.
To grant gke break glass access, you connect a role to a user identity. A RoleBinding grants permissions inside one namespace by referencing a Role or ClusterRole. A ClusterRoleBinding grants permissions across the entire cluster at once. During an incident, an admin or automated workflow creates this binding for the responder. When the fix is complete, the system deletes the binding to remove the extra rights.
Emergency work often requires reading pod logs or running commands inside a container. In Kubernetes RBAC, these actions use subresources like pod logs or pod exec. You must list these subresources in your role rules to allow deep debugging. If you omit them, the responder can view pods but cannot run shell commands inside them.
Where native role bindings fall short during an incident
Native role bindings solve the problem of who can log in during an outage. They let you keep daily permissions tight while offering a path for urgent fixes. They also record the role change in cluster logs for later review. This meets the basic need for gated entry into production clusters.
Still, a role binding only checks identity when the API request starts. Once an engineer has pod exec rights, RBAC cannot filter commands inside the shell. The responder can read secret files or view raw customer data in memory. Native RBAC does not rewrite sensitive fields like emails, phone numbers, or card numbers.
Break-Glass Access Procedures + Customer Support Access to Production: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Also, removing a role binding can take manual work after a long outage. If a team forgets to delete the binding, the emergency rights stay active forever. Teams need a way to approve single actions and control data in flight.
Designing safer emergency workflows for clusters
Start by splitting emergency roles into smaller tiers based on the outage type. Most incidents only need read rights for logs and events in one namespace. Create a dedicated RoleBinding in that namespace instead of a wide ClusterRoleBinding. Reserve full cluster-wide write rights for severe control plane or node failures.
Next, require a peer review step before anyone activates an emergency role. Link every request to an active incident ticket so auditors know why it happened. Set a strict time limit on the binding so it expires on its own. Finally, review all commands run during the session before closing the incident report.
Check Kubernetes API requests and responses on the wire
Kubernetes RBAC decides who may call the API server. RBAC does not check response payloads or split listing Secret names from reading Secret values when a role grants get access. With hoop.dev, you run an open-source proxy, a sidecar for your Kubernetes API, inline on the API connection. On every kubectl or API call an AI agent or engineer runs against production, hoop.dev decides to pass, mask, block, or hold it for human review.
On each call, hoop.dev guardrails read the method, normalized path, and headers to refuse bad deletes or full Secret reads with a 403 error, or hold risky calls for approval. On the way back, hoop.dev data masking walks JSON responses by key and masks matched values in Secrets, ConfigMaps, and pod logs in memory. View the project on GitHub.
Frequently asked questions
What is the difference between a RoleBinding and a ClusterRoleBinding for emergencies?
A RoleBinding limits emergency permissions to one specific namespace in your cluster. A ClusterRoleBinding grants those permissions across all namespaces and cluster-scoped nodes at once. Use a RoleBinding whenever the outage affects only one service or team namespace.
Can Kubernetes RBAC block specific shell commands during a break glass session?
No, Kubernetes RBAC only controls whether a user can start an exec subresource call. It cannot read, block, or filter individual shell commands inside the running container. You must treat pod exec permissions as full control over that container.
How do you revoke emergency permissions in Kubernetes RBAC?
Because Kubernetes RBAC has no deny rules, you cannot add a blocking rule later. You must delete the temporary RoleBinding or ClusterRoleBinding that granted the emergency permissions. Once you delete the binding, the API server stops allowing those extra actions.
Related guides
In this post