How to Configure Haystack Kubernetes Connect

Published
Length
3 min read

Luis Felipe Dias

All posts

You set up a haystack kubernetes connect workflow by pairing service accounts with RBAC roles. Your AI pipeline then talks to the cluster API server to run tasks or read state. Many teams start by giving the pipeline a broad cluster role so tests pass fast. When those rules stay wide in production, one bad prompt can change live pods or read secrets. How a haystack kubernetes connect workflow uses native roles In Kubernetes, RBAC rules control what your Haystack app can do inside the cluster.

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

You set up a haystack kubernetes connect workflow by pairing service accounts with RBAC roles. Your AI pipeline then talks to the cluster API server to run tasks or read state. Many teams start by giving the pipeline a broad cluster role so tests pass fast. When those rules stay wide in production, one bad prompt can change live pods or read secrets.

How a haystack kubernetes connect workflow uses native roles

In Kubernetes, RBAC rules control what your Haystack app can do inside the cluster. The API server checks these rules on every request from your pipeline. A Role sets permissions inside one single namespace. A ClusterRole sets permissions across the whole cluster, including nodes and all namespaces.

All permissions in Kubernetes RBAC are purely additive. There are no deny rules in this system. You start with zero access and add rules for each action your pipeline needs. To grant these rules to your Haystack service account, you create a binding.

A RoleBinding grants access inside one specific namespace. That RoleBinding can point to a local Role or a broad ClusterRole. If your pipeline must work across every namespace, you use a ClusterRoleBinding instead. Most teams scope Haystack to a single namespace so the app stays contained.

Some cluster tasks use subresources in the API URL. For example, reading pod logs or running commands inside a pod uses a subresource path. You write these in RBAC rules with a slash between the resource and the subresource. Your Haystack app needs explicit rules for those subresources before it can read logs or run commands.

What native RBAC solves and what it misses

Native RBAC is great at deciding which service account can reach a cluster resource. It blocks a Haystack pipeline in one namespace from touching pods in another namespace. It also stops a read-only pipeline from deleting deployments or changing cluster nodes. This setup gives you a strong first line of defense at the API door.

Still, RBAC only checks who makes the call and which API verb they use. It does not inspect what happens inside the request once the call goes through. If your Haystack agent has permission to run commands in a pod, RBAC steps aside. It cannot tell a safe diagnostic command from a destructive shell command inside that container.

Kubernetes RBAC + End-to-End Encryption: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

RBAC also does not inspect the data that comes back from the cluster. Pod logs and command outputs often hold user emails, phone numbers, card numbers, or national IDs. Native cluster roles let all of that raw text flow straight into your Haystack pipeline. From there, sensitive fields can end up in model prompts or external logs.

Why static cluster keys create risk for AI pipelines

To connect Haystack to an external cluster, teams often export a static token or client certificate. They store that key in the app host so the pipeline can call the cluster API. Anyone who gets access to that host can use the same key from anywhere. Static keys also rarely map back to the human who triggered a specific pipeline run.

When an AI agent runs a bad command, cluster logs only show the shared service account name. Your security team cannot see which user or prompt caused the change. You also have to rotate those stored keys by hand when people leave the team. A safer setup checks every command in flight and rewrites sensitive fields in results.

Check Kubernetes API requests and responses on the wire

Kubernetes RBAC decides who may call the API server. RBAC does not check response payloads or split listing Secret names from reading Secret values when a role grants get access. With hoop.dev, you run an open-source HTTP proxy, a sidecar for your Kubernetes API. The kubectl client changes only the server address it talks to while RBAC stays in place.

On each call, hoop.dev guardrails read the method, normalized path, and headers to refuse bad deletes or full Secret reads with a 403 error. On the way back, hoop.dev data masking walks JSON responses by key and masks matched values in Secrets, ConfigMaps, and pod logs. Every request and verdict is logged without storing bearer tokens. View the project on GitHub.

Frequently asked questions

Should I use a Role or a ClusterRole for a Haystack pipeline?

Use a Role and a RoleBinding whenever your pipeline works in a single namespace. Only use a ClusterRoleBinding if the app truly needs access across every namespace.

Can I write a deny rule in Kubernetes RBAC to block risky commands?

No, Kubernetes RBAC permissions are purely additive and do not support deny rules. You must omit the permission from the role or check commands at runtime.

How do I let Haystack read pod logs without giving full pod access?

Grant read access only to the pod log subresource in your Role rules. This lets the pipeline read logs without allowing it to delete pods or run commands.

The query runs. The emails stay hidden.