Kubernetes Workflow Engine
A kubernetes workflow engine runs multi-step jobs across pods in your cluster. You set it up by deploying a controller and binding service account roles. Platform teams often start by giving the engine broad cluster permissions to avoid errors. That shortcut works at first, yet it lets any job touch sensitive resources. When a single step fails, operators also need direct pod access to fix it. Broad human and machine roles turn small mistakes into major cluster incidents. How a kubernetes work
Your coding agents run with your access.
Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.
A kubernetes workflow engine runs multi-step jobs across pods in your cluster. You set it up by deploying a controller and binding service account roles. Platform teams often start by giving the engine broad cluster permissions to avoid errors. That shortcut works at first, yet it lets any job touch sensitive resources. When a single step fails, operators also need direct pod access to fix it. Broad human and machine roles turn small mistakes into major cluster incidents.
How a kubernetes workflow engine uses native RBAC
Every workflow step runs inside a pod and talks to the API server. Kubernetes uses role-based access control to check every single API call. In this system, all permissions are additive, and there are no deny rules. You start with zero access and add rules for each needed action.
A Role sets permissions inside one single namespace for your workflow pods. A ClusterRole sets rules across the whole cluster without namespace limits. ClusterRoles can cover cluster-wide objects like nodes or namespaced objects across all namespaces. They can also cover endpoints that do not map to standard cluster objects.
To grant these rules to your engine, you create a binding object. A RoleBinding grants permissions inside one namespace to a user or service account. That RoleBinding can point to a local Role or a shared ClusterRole. A ClusterRoleBinding grants permissions across the entire cluster at once. If a team binds the workflow controller with a ClusterRoleBinding, every namespace opens up.
Why native roles leave gaps during workflow failures
Native roles work well when automated jobs run without any human help. The API server checks who is calling and which resource they want. Still, workflows fail in production when data changes or external systems go down. Engineers must then read pod logs or run commands inside running containers.
Kubernetes handles these actions through subresources on the main pod object. In RBAC rules, a subresource uses a slash after the main resource name. One common subresource controls who can read the logs of a pod. Another subresource controls who can open a shell and run commands inside pods.
Kubernetes RBAC + Agentic Workflow Security: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Native roles decide who can start that shell session on a pod. Once the session starts, RBAC does not check individual commands inside the shell. An engineer who can exec into a workflow pod can run anything inside. They can read local secrets, change files, or query databases from that pod. Output from those commands can also return sensitive data like emails or card numbers. Native cluster controls do not rewrite matched fields in results during a live session.
Separating workflow execution from human debugging
A safe setup splits machine roles from human debugging access from day one. Give the workflow service account a local Role inside its own namespace only. Bind that Role with a RoleBinding so jobs stay inside one boundary. Avoid ClusterRoleBindings unless the engine truly must manage nodes or all namespaces.
Treat human access to workflow pods as a separate risk to manage. Engineers need to inspect broken steps, check state, and run safe fix commands. Giving them permanent pod exec rules across production namespaces creates a huge blind spot. Teams need controls that inspect individual commands and mask sensitive output in flight. For example, masking rewrites matched fields in results like phone numbers or national IDs.
Check Kubernetes API requests and responses on the wire
Kubernetes RBAC decides who may call the API server. RBAC does not check response payloads or split listing Secret names from reading Secret values when a role grants get access. With hoop.dev, you run an open-source HTTP proxy, a sidecar for your Kubernetes API. The kubectl client changes only the server address it talks to while RBAC stays in place.
On each call, hoop.dev guardrails read the method, normalized path, and headers to refuse bad deletes or full Secret reads with a 403 error. On the way back, hoop.dev data masking walks JSON responses by key and masks matched values in Secrets, ConfigMaps, and pod logs. Every request and verdict is logged without storing bearer tokens. View the project on GitHub.
Frequently asked questions
Should a kubernetes workflow engine use a Role or a ClusterRole?
Use a Role and a RoleBinding when workflows run in one namespace. Use a ClusterRole with a local RoleBinding to reuse rules across several namespaces. Avoid ClusterRoleBindings unless the engine must touch every namespace or cluster node.
Can Kubernetes RBAC block specific commands inside a pod exec session?
No, native RBAC only checks if a user can call the pod exec subresource. Once the API server allows the session, it does not inspect shell commands. You need a separate control layer to check or approve individual commands.
Can you add a deny rule to block a workflow from reading secrets?
No, Kubernetes RBAC permissions are additive and do not support deny rules. To block secret access, you must remove the secret rule from every bound role. Always start with small roles and add only the exact resources each job needs.
Related guides
In this post