Kubernetes Testing in CI

Published
Length
3 min read

Luis Felipe Dias

All posts

You run kubernetes testing in ci by giving your pipeline a scoped role. The job then sends test manifests to a namespace and checks pod status. Many teams start by giving the build runner a broad cluster admin role. That quick shortcut makes setup fast at first. Still, broad permissions expose every namespace and secret if a build script breaks. A leaked token can let an attacker read or delete live workloads across clusters. How to set up kubernetes testing in ci with native roles A safer se

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

You run kubernetes testing in ci by giving your pipeline a scoped role. The job then sends test manifests to a namespace and checks pod status. Many teams start by giving the build runner a broad cluster admin role. That quick shortcut makes setup fast at first. Still, broad permissions expose every namespace and secret if a build script breaks. A leaked token can let an attacker read or delete live workloads across clusters.

How to set up kubernetes testing in ci with native roles

A safer setup starts with native Role-Based Access Control on the API server. In Kubernetes, a Role holds rules that define a set of permissions. All rules are purely additive, which means there are no deny rules. A standard Role always sets permissions within one single namespace. By contrast, a ClusterRole covers cluster-wide resources like nodes or all namespaces at once.

For a test runner, you create a dedicated namespace for the build job. Next, you define a Role inside that namespace with only the needed rules. You then create a RoleBinding to link that Role to the runner identity. A RoleBinding grants permissions inside one namespace, even if it references a general ClusterRole. Avoid using a ClusterRoleBinding for test jobs because it grants access across the cluster.

Your test scripts often need to read pod logs or run a check command. In RBAC, the API server treats logs and command execution as subresources. You must grant permissions for these subresources explicitly in your Role definition. When a test finishes, the pipeline tears down the test pods in that namespace. This workflow keeps test jobs isolated from other workloads on the same cluster.

Why native roles leave a gap during test runs

Native roles and bindings solve the first part of pipeline security. They control which runner can talk to the API server and which namespace it uses. This keeps a test job in one namespace from deleting pods in another namespace. It also stops the runner from changing cluster nodes or global network rules.

Still, native RBAC only checks the API request when the session starts. Once a runner has permission to run commands in a pod, RBAC stops watching. It does not check the specific shell commands running inside the container. It also cannot see the database queries or data returned during the test. If a test pod connects to a shared staging database, raw data flows back.

Just-in-Time Access + Kubernetes RBAC: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

Logs from a test run might print real emails, phone numbers, or card numbers. Static keys stored in build settings also stay active long after the job ends. Teams also struggle when a test fails and an engineer needs to check why. Giving that engineer direct cluster credentials often leads to permanent, broad permissions again. Native roles decide who gets in, not what commands or data move in flight.

Scoping test runner permissions and secrets

To keep test runs safe, teams should split build tasks from cluster permissions. Start by giving each pipeline its own identity instead of sharing one global key. Bind that identity to a single namespace using a Role and a RoleBinding. Never reuse production keys or broad cluster roles inside a test runner.

Keep subresource permissions tight so jobs only run commands when a test requires it. Rotate runner credentials often so old tokens cannot be used outside the pipeline. Finally, check what data flows back into build logs during integration tests. Rewriting matched fields like emails or national IDs in results keeps build logs clean. When you pair tight namespace roles with runtime checks, your test pipeline stays safe.

Check Kubernetes API requests and responses on the wire

Kubernetes RBAC decides who may call the API server. RBAC does not check response payloads or split listing Secret names from reading Secret values when a role grants get access. With hoop.dev, you run an open-source HTTP proxy, a sidecar for your Kubernetes API. The kubectl client changes only the server address it talks to while RBAC stays in place.

On each call, hoop.dev guardrails read the method, normalized path, and headers to refuse bad deletes or full Secret reads with a 403 error. On the way back, hoop.dev data masking walks JSON responses by key and masks matched values in Secrets, ConfigMaps, and pod logs. Every request and verdict is logged without storing bearer tokens. View the project on GitHub.

Frequently asked questions

Should a CI runner use a Role or a ClusterRole for tests?

Use a Role paired with a RoleBinding inside a single test namespace. A ClusterRoleBinding grants cluster-wide permissions, which is too broad for most test jobs.

How do test runners read pod logs or run check commands?

Kubernetes treats pod logs and command execution as subresources in the API. You must add explicit rules for these subresources inside the runner Role.

Can native Kubernetes RBAC block specific shell commands inside a pod?

No, native RBAC only checks if the runner can start the execution call. It does not inspect or filter the individual shell commands run inside the pod.

The query runs. The emails stay hidden.