How to Configure DuckDB IAM Authentication
You set up duckdb iam authentication by linking your local engine to a cloud role. The engine then reads your local cloud profile to fetch temporary tokens for storage buckets. Analytics teams often start by sharing static storage keys across laptops and scripts. Broad storage roles let anyone with the key read or delete every file in the bucket. How duckdb iam authentication works in practice DuckDB runs inside your app or on your local machine as an in-process engine. It does not have its o
Your coding agents run with your access.
Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.
You set up duckdb iam authentication by linking your local engine to a cloud role. The engine then reads your local cloud profile to fetch temporary tokens for storage buckets. Analytics teams often start by sharing static storage keys across laptops and scripts. Broad storage roles let anyone with the key read or delete every file in the bucket.
How duckdb iam authentication works in practice
DuckDB runs inside your app or on your local machine as an in-process engine. It does not have its own built-in user table for cloud storage checks. Instead, it relies on official extensions to talk to cloud providers like AWS or GCP. You load the cloud extension and tell the engine to use your local credential chain.
The credential chain checks your machine for active cloud sessions, environment variables, or instance roles. When you run a query on a remote file, the engine asks the cloud provider for a token. The cloud provider checks your IAM role and sends back a short-lived token. DuckDB uses this token to read or write Parquet and CSV files in your bucket.
Setting up roles and credential chains for teams
Good setup starts in your cloud console before you run a single query. You create an IAM role that only grants read rights to specific storage paths. For local work, engineers log in through their cloud command line tool using single sign-on. The engine picks up those local session tokens without needing raw keys in the script.
For servers and automated jobs, you attach the IAM role directly to the compute instance. The engine reads the instance metadata to get fresh tokens on its own. This approach stops developers from pasting static secrets into code or config files. You also need to set a clear session timeout on the IAM role itself. Short timeouts force users to log in again and limit the risk of stolen tokens.
Service-to-Service Authentication + AWS IAM Policies: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
What native IAM roles solve and what keeps the session active
Using IAM roles stops the spread of long-lived storage keys across your team. If an engineer leaves the team, you revoke their cloud role in one place. You also get basic storage logs showing which IAM role opened a specific bucket file.
Still, cloud IAM only checks who can open a file at the storage layer. It does not see or control the actual SQL statements running inside the engine. Once a user has read rights on a Parquet file, they can query every column inside it by default. Cloud roles cannot mask sensitive fields like emails, phone numbers, card numbers, or national IDs. IAM logs also miss the exact SQL query text that pulled those records. Teams still need a way to inspect queries in flight and protect private columns.
Govern statements and responses across databases and APIs
Good config and scoped roles are needed setup. They do not check single queries in flight or mask sensitive fields in returned rows. Across Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, and internal HTTP endpoints, hoop.dev sits inline in the protocol beside each supported service as an open-source proxy, a sidecar for your database or API. For every query or command an AI agent or engineer runs against production, hoop.dev decides to pass, mask, block, or hold it for human approval.
First, hoop.dev guardrails block banned queries before they run or hold risky ones for review, and hoop.dev data masking rewrites matched sensitive values in responses in memory. Furthermore, hoop.dev holds SOC 2 Type II and is open source on GitHub.
Frequently asked questions
Does DuckDB store my cloud IAM keys on disk?
No, the engine holds temporary tokens in memory only for your active session. When you use the credential chain, it reads existing tokens from your cloud login tool.
Can I use IAM roles to restrict specific columns in a Parquet file?
No, cloud IAM rules only work at the bucket and file path level. If a role can read the file, the user can query every column and row inside it by default.
Why do my queries fail after an hour when using IAM roles?
Cloud IAM sessions expire after a set time limit. You must refresh your local cloud login or let the engine fetch a new token to keep running queries.
Related guides
In this post