EKS Backups

Published
Length
3 min read

Luis Felipe Dias

All posts

Teams create eks backups by saving both Kubernetes cluster state and persistent volume snapshots. You must copy API objects and storage disks on a set schedule. Many engineering teams start by giving backup tools and admin users full cluster rights. Broad permissions make setup fast, but they expose live apps and secrets to mistakes. A bad script or human error can wipe out production workloads in seconds. How eks backups work across state and storage A Kubernetes cluster stores its live stat

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

Teams create eks backups by saving both Kubernetes cluster state and persistent volume snapshots. You must copy API objects and storage disks on a set schedule. Many engineering teams start by giving backup tools and admin users full cluster rights. Broad permissions make setup fast, but they expose live apps and secrets to mistakes. A bad script or human error can wipe out production workloads in seconds.

How eks backups work across state and storage

A Kubernetes cluster stores its live state as objects inside the control plane. Your workloads also write business data to external block or file storage volumes. Good eks backups must capture both the API definitions and the attached disk data. If you only save disk snapshots, you lose your service routes and pod specs. If you only save API manifests, you lose the database records inside your volumes.

To read cluster state, a backup operator queries the Kubernetes API server. Kubernetes role-based access control governs what this operator or user can read and write. A Role sets permissions within one single namespace for local backup tasks. A ClusterRole covers cluster-scoped resources like nodes or namespaced items across all namespaces. Permissions in Kubernetes are purely additive, which means there are no deny rules.

You link these rules to backup accounts using a RoleBinding or a ClusterRoleBinding. A RoleBinding grants permissions inside one namespace, even when it points to a ClusterRole. A ClusterRoleBinding grants access across the entire cluster for full disaster recovery jobs. Backup tools often need pod subresource rights to freeze databases before taking a snapshot. In Kubernetes RBAC, the API names these subresources with a slash, like pods/exec. Running a hook through pods/exec lets the tool flush disk caches for clean backups.

Why native roles leave a gap during backup operations

Native Kubernetes roles do a great job of controlling which accounts can log in. They let you limit a backup service account to read rules in one namespace. Still, engineers still need direct access to test restores, check logs, and fix broken jobs. When a restore fails, an admin often needs cluster-wide write rights and pods/exec permissions.

Once an engineer gets that role, native RBAC stops checking what they do next. The API server checks who enters the cluster, not what individual commands do in flight. An admin with exec rights can open a shell inside any pod in that scope. They can run destructive commands or read raw database records containing emails and card numbers. Native roles cannot pause a dangerous delete command or rewrite sensitive fields in query results. Teams need a way to inspect every command and protect data during live recovery tasks.

EKS Access Management + Immutable Backups: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

Scoping permissions for safer recovery workflows

You can reduce risk by splitting automated backup roles from human recovery roles. Give your automated backup tool read access to cluster objects and specific snapshot rights. Only grant pods/exec rules in namespaces where databases require a pre-snapshot freeze hook. Keep human engineers out of broad ClusterRoleBinding groups during normal daily operations.

When an engineer must fix a failed backup, review the exact commands they run. Check pod logs through the pods/log subresource before opening an interactive shell session. If a team member queries restored data, apply masking rules to rewrite phone numbers and national IDs. This keeps sensitive user records safe while engineers verify that a backup image works.

Check Kubernetes API requests and responses on the wire

Kubernetes RBAC decides who may call the API server. RBAC does not check response payloads or split listing Secret names from reading Secret values when a role grants get access. With hoop.dev, you run an open-source proxy, a sidecar for your Kubernetes API, inline on the API connection. On every kubectl or API call an AI agent or engineer runs against production, hoop.dev decides to pass, mask, block, or hold it for human review.

On each call, hoop.dev guardrails read the method, normalized path, and headers to refuse bad deletes or full Secret reads with a 403 error, or hold risky calls for approval. On the way back, hoop.dev data masking walks JSON responses by key and masks matched values in Secrets, ConfigMaps, and pod logs in memory. View the project on GitHub.

Frequently asked questions

What is the difference between a Role and a ClusterRole for eks backups?

A Role limits backup permissions to resources inside one single Kubernetes namespace. A ClusterRole can grant backup access to cluster-wide resources or across all namespaces at once.

Why do backup tools ask for pods/exec permissions in Kubernetes?

Backup tools use the pods/exec subresource to run commands inside running database containers. These commands flush memory to disk so the storage snapshot captures clean data.

Can Kubernetes RBAC block specific shell commands during a backup restore?

No, Kubernetes RBAC only checks if a user has permission to call the exec API. It does not inspect or block the actual shell commands run inside the container.

The query runs. The emails stay hidden.