All posts

Audit logging for AI agents on BigQuery

Most audit logging for automated systems fails in the same place: attribution. The log says a query ran, when it ran, and roughly what it touched, but when every agent shares one service-account key, the "who" is a single generic principal doing the work of a dozen. An audit trail that cannot tell your agents apart is a timeline, not an audit. Audit logging for AI agents on BigQuery has to do better. It has to record what each agent did, attribute it to that agent, and produce a trail that hold

Free White Paper

K8s Audit Logging + AI Audit Trails: The Complete Guide

Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.

Free. No spam. Unsubscribe anytime.

Most audit logging for automated systems fails in the same place: attribution. The log says a query ran, when it ran, and roughly what it touched, but when every agent shares one service-account key, the "who" is a single generic principal doing the work of a dozen. An audit trail that cannot tell your agents apart is a timeline, not an audit.

Audit logging for AI agents on BigQuery has to do better. It has to record what each agent did, attribute it to that agent, and produce a trail that holds up because the agent did not write it. That last property is what separates audit logging from an agent's own debug output.

A log and a recording are not the same thing

It helps to separate two ideas. A recording captures the statements in a session for replay. Audit logging is the durable, queryable trail of who did what, across sessions, that you reason over after the fact. You want both, and they share one requirement: a real, per-agent identity attached to every entry.

BigQuery emits job logs, but under a shared key those logs inherit the shared identity, so the audit question collapses to "the service account did it." That is the gap audit logging at the connection layer closes.

What good audit logging answers for a BigQuery agent

A trail earns the name when it can answer a specific question without guesswork: which agent ran which statement against which dataset, when, and was anyone asked to approve it. Hold a candidate trail up to that question. Shared-key job logs answer the statement and the dataset but blur the agent into one principal. An agent's own logs answer everything until you ask whether the agent could have lied.

Continue reading? Get the full guide.

K8s Audit Logging + AI Audit Trails: Architecture Patterns & Best Practices

Free. No spam. Unsubscribe anytime.

Audit logging at the connection layer answers all of it because it sits where identity and statement meet. Every query crosses the boundary, the boundary knows which agent is calling, and it writes the entry itself. The trail you query later is one the audited agents never had a hand in producing.

Producing the trail at the gateway

hoop.dev proxies the connection to BigQuery, so every query the agent runs through the native bq client passes the gateway. hoop.dev logs those queries at the command level and ties each to the calling agent, producing an audit trail outside the agent's control.

With GCP IAM federation on, the session uses a per-user OAuth credential, so the identity in the audit log is the agent itself. Now the trail answers "which agent ran this query against which dataset," which is exactly the question an audit asks.

Steps

  1. Deploy the hoop.dev agent near your GCP project; it connects outbound to the gateway.
  2. Register a BigQuery connection with CLOUDSDK_CORE_PROJECT set and GCP IAM federation enabled.
  3. Turn on recording and auditing for the connection so each query is captured and attributed.
  4. Send the agent's bq traffic through the gateway.
# this query is attributed to the agent in the audit trail
bq query --use_legacy_sql=false \
  'DELETE FROM staging.imports WHERE batch_id = 42'

Verify

Run a mix of read and write queries as different agents, then search the audit trail in hoop.dev by agent identity. Confirm each query maps to the agent that ran it, with timestamps, independent of BigQuery's own job logs.

Pitfalls

  • Do not accept shared-key job logs as your audit trail; they cannot separate agents.
  • Do not store the audit trail where an audited workload can edit it.
  • Do not log only writes. Reads against sensitive datasets are exactly what an exfiltration looks like.

hoop.dev is open source, so you can verify how the audit trail is produced and where it lives. Read the getting started guide and see how auditing builds on session recording for AI agents on BigQuery. Start at github.com/hoophq/hoop and generate a per-agent audit trail from a test connection.

FAQ

How is audit logging different from session recording?

Session recording captures a session's statements for replay. Audit logging is the durable, queryable trail of who did what across sessions. hoop.dev produces both at the gateway, each tied to a per-agent identity.

Why not just use BigQuery's job logs?

Under a shared service-account key, job logs attribute everything to one principal. Audit logging at the gateway, with GCP federation, attributes each query to the specific agent that ran it.

Open source

Save the open-source gateway for agent data access

Hoop is MIT-licensed infrastructure for controlling how AI agents reach production data. Star hoophq/hoop so you can inspect it, deploy it, or share it when your team starts governing agent access.

Star and save the repo →More posts