Airbyte Helm

Published
Length
3 min read

Luis Felipe Dias

All posts

You deploy Airbyte on Kubernetes by installing the official airbyte helm chart into your cluster. This chart sets up the web app, state database, workflow engine, and worker pods. Most engineering teams run this chart with shared cluster roles and static database keys. Broad permissions let any worker pod or cluster admin read sensitive tables without oversight. How an airbyte helm setup works An airbyte helm deployment breaks the data platform into several small services inside Kubernetes. T

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

You deploy Airbyte on Kubernetes by installing the official airbyte helm chart into your cluster. This chart sets up the web app, state database, workflow engine, and worker pods. Most engineering teams run this chart with shared cluster roles and static database keys. Broad permissions let any worker pod or cluster admin read sensitive tables without oversight.

How an airbyte helm setup works

An airbyte helm deployment breaks the data platform into several small services inside Kubernetes. The control plane manages sync schedules, user logins, and connector rules through a web app. A workflow service tracks job state and tells worker pods when to start a sync. When a sync begins, the worker pod spins up source and destination containers. These transient containers read records from a source system and write them to a target. An internal database stores job history, sync progress, and secret keys for every data source. Teams pass custom values to the chart to use external storage and external databases. Kubernetes role rules control which pods can start new job containers in the namespace.

What the native chart solves and what it misses

The standard chart makes it easy to scale data pipelines across many cluster nodes. It separates the job scheduler from the worker pods that move heavy data volumes. If a worker pod crashes, the workflow service restarts the job on another node. Still, this cluster setup only controls who can log in to the web app. It does not inspect the actual queries or records moving through the worker pods. Engineers who have cluster access can open pod shells or read stored database keys. Once a sync job starts, the chart cannot mask emails, phone numbers, or card numbers. A bad query or a broad database role can pull restricted rows without warning. Native cluster logs show pod status, but they miss the exact commands run inside databases.

Managing secrets and roles in Kubernetes

Teams often store database keys inside standard Kubernetes secrets when they deploy the chart. Any user or pod with secret read rights in that namespace can view those keys. To reduce risk, platform teams link the chart to an external secret store. They also limit Kubernetes role bindings so only worker pods can launch sync containers. Even with strict pod roles, the database keys themselves often hold broad read rights. A source database sees every sync job coming from the same shared service account. It cannot tell which engineer set up the sync or approved the data transfer. Auditing becomes hard when dozens of pipelines share one static login key.

Govern kubectl and API calls in flight

Identity providers and Kubernetes RBAC decide who logs into a cluster. They cannot redact keys inside returned JSON objects or hold a risky delete call for review. By design, hoop.dev runs as an open-source proxy, a sidecar for your Kubernetes API, checking requests and responses without replacing RBAC.

Helm Chart Security: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

First, hoop.dev guardrails refuse banned methods, paths, or headers before they run. Next, hoop.dev data masking rewrites sensitive keys in returned Secret, ConfigMap, and log responses in memory, and writes each decision to the audit trail. Explore the repo on GitHub.

Frequently asked questions

Why use an external database with the chart?

The default chart runs a database pod inside the cluster for state and job history. An external managed database prevents data loss if the cluster pods restart or fail.

How do worker pods handle high sync volumes?

The workflow engine creates new pods for each sync job up to your set limit. Kubernetes scales these job pods across available nodes based on CPU and memory rules.

Where does the chart store source and destination keys?

By default, the platform writes connector keys to its internal state store or Kubernetes secrets. Teams should configure an external secret manager before moving production data.

Teams that document these role rules and review logs each month catch config drift early. Clear ownership for each role keeps live systems simple to run and easy to audit across the whole team.

The query runs. The emails stay hidden.