How to Configure Haystack Kubernetes Approval

Published
Length
3 min read

Luis Felipe Dias

All posts

You set up haystack kubernetes approval by binding native cluster roles to your pipeline pods. Then you add human review steps before those AI agents run sensitive cluster commands. Teams often start by giving a Haystack service account broad cluster permissions. This shortcut makes early testing fast for developers. Still, broad permissions let automated pipelines change or delete live workloads without a human check. A single bad prompt can trigger a wide outage across your whole cluster. Ho

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

You set up haystack kubernetes approval by binding native cluster roles to your pipeline pods. Then you add human review steps before those AI agents run sensitive cluster commands. Teams often start by giving a Haystack service account broad cluster permissions. This shortcut makes early testing fast for developers. Still, broad permissions let automated pipelines change or delete live workloads without a human check. A single bad prompt can trigger a wide outage across your whole cluster.

How native haystack kubernetes approval works

Kubernetes uses Role-Based Access Control to manage what users and apps can do. The API server checks these rules before it allows any request. In this system, permissions only add access. There are no deny rules in native Kubernetes RBAC. You start with zero access and add rules one by one.

A Role sets permissions inside one single namespace. A ClusterRole sets permissions across the whole cluster. It can cover cluster-wide objects like nodes or namespaced objects across all namespaces. To grant these permissions to a Haystack pipeline, you create a binding. A RoleBinding grants permissions inside one namespace. It can point to a Role in that namespace or a shared ClusterRole. A ClusterRoleBinding grants permissions across the entire cluster at once.

When a Haystack agent needs to read pod logs or run commands, it uses subresources. Kubernetes RBAC names resources the same way they appear in the API URL. Subresources use a slash format in your role rules, such as the pod log resource. If an agent needs to run a command inside a container, it needs the pod exec subresource. For haystack kubernetes approval, teams often split these roles by risk. They give the agent read roles by default, and keep write or exec roles separate.

What native cluster roles solve and what they miss

Native roles and bindings solve the first part of cluster security. They verify which service account or user logs in to the API server. They also limit which namespaces and resource types a Haystack app can touch. If a pipeline only has a RoleBinding in a test namespace, it stays there. It cannot touch workloads in your production namespace.

Approval Chains & Escalation + Kubernetes RBAC: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

Still, native RBAC leaves a major operational gap for live operations. The API server only checks who makes the request and which resource they target. It does not check what individual commands or statements do in flight. Once a role grants pod exec access, the user or agent can run any shell command inside that pod. Native RBAC does not pause a risky command to ask a human for approval. It also does not inspect data coming back from pods. If pod logs contain emails, phone numbers, card numbers, or national IDs, Kubernetes sends them in plain text.

Why static bindings struggle with AI pipelines

AI pipelines in Haystack often pick tools and build commands based on user input. Static role bindings are either on or off all the time. If you leave high-level roles bound to the agent, every tool call runs right away. If you remove the binding, the pipeline breaks whenever it needs to fix a pod. Teams try to fix this by building custom ticket bots to add and remove bindings. This adds manual work and still grants full access while the binding is active. Once the binding exists, the agent can run any command in that scope without further checks.

Check Kubernetes API requests and responses on the wire

Kubernetes RBAC decides who may call the API server. RBAC does not check response payloads or split listing Secret names from reading Secret values when a role grants get access. With hoop.dev, you run an open-source HTTP proxy, a sidecar for your Kubernetes API. The kubectl client changes only the server address it talks to while RBAC stays in place.

On each call, hoop.dev guardrails read the method, normalized path, and headers to refuse bad deletes or full Secret reads with a 403 error. On the way back, hoop.dev data masking walks JSON responses by key and masks matched values in Secrets, ConfigMaps, and pod logs. Every request and verdict is logged without storing bearer tokens. View the project on GitHub.

Frequently asked questions

Can Kubernetes RBAC block one specific command inside a pod?

No, Kubernetes RBAC only controls access to the pod exec subresource as a whole. Once an account has that permission, the API server does not inspect the shell commands inside the session.

How do RoleBindings differ from ClusterRoleBindings for Haystack agents?

A RoleBinding limits your Haystack agent to resources inside one single namespace. A ClusterRoleBinding gives the agent permissions across all namespaces and cluster-wide nodes at once.

Does native Kubernetes RBAC support deny rules for risky actions?

No, permissions in Kubernetes RBAC are purely additive and never include deny rules. You must design roles with the smallest scope needed and add permissions only when required.

The query runs. The emails stay hidden.