Airflow Keycloak Auth Manager
You set up the airflow keycloak auth manager by linking your identity server to Airflow. This setup uses OpenID protocols to log users in and assign their app roles. Engineering teams use this method to centralize user logins across their data tools. Still, teams often map broad identity groups to admin roles inside the scheduler. Broad roles let too many users read secret keys or change live data tasks. How the airflow keycloak auth manager works Airflow uses a modular manager system to hand
Keycloak + GCP Access Context Manager: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You set up the airflow keycloak auth manager by linking your identity server to Airflow. This setup uses OpenID protocols to log users in and assign their app roles. Engineering teams use this method to centralize user logins across their data tools. Still, teams often map broad identity groups to admin roles inside the scheduler. Broad roles let too many users read secret keys or change live data tasks.
How the airflow keycloak auth manager works
Airflow uses a modular manager system to handle user logins and role checks. When you pick Keycloak as your provider, Airflow sends login requests to your realm. The user logs in on the Keycloak page and gets a signed token. Airflow reads this token to check the user identity and group claims.
Next, the manager maps those token claims to local roles inside the app. You can set rules that match a team group to a viewer role. Another rule might map senior engineers to an operator or admin role. The web server checks these roles before a user clicks a button or views logs. Every page load or API call runs through this same permission check.
This workflow keeps user passwords out of the local Airflow database. Your security team manages password rules and multi-factor steps in one place. When someone leaves the team, you block their account in the identity server. They lose their ability to log into the workflow app right away.
What native role mapping solves and what it misses
Native role mapping solves the problem of who can open the web app. It stops unknown users from viewing task graphs or triggering data runs. You can also limit who sees specific workflow views inside the user interface. For basic dashboard use, this identity link works well.
Still, the login manager only checks permissions inside the web app itself. It does not check what happens when a task runs on a worker node. Nor does it watch when an engineer opens a shell into the backend database. Airflow stores database passwords and cloud keys in its own metadata tables. Once a task or user reads those keys, the login manager steps aside.
Keycloak + GCP Access Context Manager: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
This leaves a big gap in your data security setup. A user with high app rights can still pull raw database credentials. Direct access to worker hosts or metadata databases bypasses web role rules. Native setup decides who logs into the app, not what individual queries do in flight. It cannot block a bad query or mask emails and card numbers in results.
Best practices for securing workflow roles and data
Start by giving most users the viewer role by default. Only grant operator rights to people who must restart failed tasks. Keep the admin role for a tiny group or an automated deploy tool. Check your identity group mappings every quarter to remove stale accounts.
Next, look at how engineers debug broken tasks and query the metadata store. Do not share a single root database password with the whole data team. Separate web app login rules from direct database and host session rules. Make sure every query against production data leaves a clear log tied to a person. When results contain phone numbers or national IDs, rewrite those matched fields in results.
Govern statements and responses across databases and APIs
Good config and scoped roles are needed setup. They do not check single queries in flight or mask sensitive fields in returned rows. Across Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, and internal HTTP endpoints, hoop.dev runs beside each supported service as an open-source proxy, a sidecar for your database or API.
First, hoop.dev guardrails block banned queries before they run. Then hoop.dev data masking rewrites matched sensitive values in responses in memory. The audit trail records every statement and verdict per resource. Furthermore, hoop.dev holds SOC 2 Type II and is open source on GitHub.
Frequently asked questions
Does the Keycloak integration manage worker database credentials?
No, it only manages user logins and roles for the web interface and API. Worker nodes still use stored secrets to talk to external databases and cloud tools.
How do group claims map to Airflow permissions?
The identity server puts group names inside the login token during sign-in. The auth manager reads those groups and assigns matching local roles like viewer or admin.
Can native auth roles mask sensitive data in task logs?
No, native roles only control who can open and read the log page. They do not rewrite emails, phone numbers, or card numbers inside the log text itself.