How to Configure DuckDB Audit Log

Published
Length
3 min read

Luis Felipe Dias

All posts

You set up a duckdb audit log by turning on query profiling or host tracking. Since this engine runs inside your app, it lacks a central server log. Teams often wrap the database in a custom app or use host file rules. When file roles are too broad, anyone with host permissions can read raw files. They can also change local log files before the security team reads them. How a native duckdb audit log works This database is an in-process engine that reads and writes local files. Unlike big clie

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

You set up a duckdb audit log by turning on query profiling or host tracking. Since this engine runs inside your app, it lacks a central server log. Teams often wrap the database in a custom app or use host file rules. When file roles are too broad, anyone with host permissions can read raw files. They can also change local log files before the security team reads them.

How a native duckdb audit log works

This database is an in-process engine that reads and writes local files. Unlike big client-server databases, no central daemon watches every user session. When a developer runs a query, the engine works inside their local process. To build a duckdb audit log, you must turn on built-in profiling features. The engine then writes query timing and execution plans to a local file. You can pick standard formats like JSON or plain text for these outputs.

Because the log sits on the local disk, your team must move it fast. Engineers usually run a background agent on the host machine to collect files. This agent reads the output file and sends each record to a central store. Some teams also use system call trackers on the Linux host itself. These tools watch which OS user opens the database file and when. If you use a cloud audit storage, you can turn on bucket read logs. Bucket logs show which service role pulled a data file from storage.

What native logging solves and what it misses

Local query logs help engineers find slow queries and fix broken data pipelines. They show how the engine plans a join or scans a large table. Host file logs and bucket logs prove when a process opened a specific file. These records help teams track basic usage across shared servers and cloud buckets. If a batch job fails at night, the local output shows the last statement.

Still, local logging cannot stop bad actions while a session is active. Anyone who can write to the folder can also delete the log file. Host logs only show that a user opened the main database file. They do not show which rows or columns the user pulled inside that session. If a table holds emails, phone numbers, or card numbers, raw values return untouched. Native file permissions only decide who can open the file on disk. They cannot block a bad drop statement or rewrite sensitive fields in flight. When multiple users share one service account, local logs cannot tell them apart. You see the shared app role in the log, not the real person.

Audit Log Integrity + End-to-End Encryption: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

Best ways to manage shared analytical sessions

To keep strong records, teams should stop sharing direct file permissions with users. Instead of giving out raw disk roles, run the engine behind a controlled service. Make sure every engineer logs in with their own identity before running queries. This step links every SQL statement to a real person on your team.

Next, send all query records to a remote log server right away. Store those records in a audit storage bucket so no user can erase their tracks. You should also check query results for sensitive fields like national IDs or emails. Masking rules can rewrite matched fields in results before they reach the user screen. Finally, review your query logs every week to spot odd data reads early.

Add runtime guardrails and masking beside your databases and APIs

Login rules, role grants, and CI checks govern who connects to your stack. Yet any allowed script, engineer, or AI agent still runs commands with the full scope of its role. For workloads on Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes, SSH, or internal HTTP services, hoop.dev sits inline in the protocol beside the service as an open-source proxy, a sidecar for your database or API. On every production query an agent or human runs, hoop.dev decides whether to pass, mask, block, or escalate it for review.

Before a query runs, hoop.dev guardrails check the statement against your rules, an AI Analyzer you configure can score intent or hold risky calls for human approval, and hoop.dev data masking rewrites sensitive fields in responses in memory. Explore the MIT-licensed project on GitHub.

Frequently asked questions

Does DuckDB have a built-in server audit log?

No, it runs inside your app process and has no central server daemon. You must use local query profiling outputs or external host tracking tools to record activity.

Can users edit local query log files?

Yes, if the engine runs under their local OS user account, they own the output. A user can edit or delete local log files before an agent ships them out.

How do we track which person ran a query on a shared server?

Local file logs only show the OS account that opened the database file. You must require personal logins before the session starts to tie queries to specific people.

The query runs. The emails stay hidden.