When autonomous workloads run queries against BigQuery, the lack of a reliable audit trail makes root‑cause analysis and compliance reporting difficult. Without a mechanism that captures the exact SQL text, the response payload, and the identity that issued the request, teams are left with only metadata from native BigQuery logs. That visibility gap hampers security monitoring, and the accountability gap makes it hard to attribute a particular query to a specific pipeline or version of code.
To close those gaps, an access gateway that sits on the data path can inspect every request, record the full interaction as session recording, and store the session for later replay. The gateway validates the caller’s identity, proxies the traffic to BigQuery, and writes an immutable record that auditors can query.
Why session recording matters for autonomous agents
Without a dedicated recording layer, the only logs you get are the standard BigQuery audit logs, which capture request metadata but not the full query text or the exact response payload. That gap leaves two problems:
- Visibility gap: You cannot see the precise SQL that an agent submitted, nor the rows it returned.
- Accountability gap: If a service account is shared among many jobs, you cannot attribute a specific query to a particular pipeline or version of code.
Both gaps undermine security monitoring, breach forensics, and compliance reporting.
Current practice and its blind spot
Many teams grant autonomous agents a shared Google service‑account key. The key lives on the VM or in a secret manager, and the agent authenticates directly to BigQuery. Identity federation can replace the static key with per‑user OAuth tokens, which narrows who can start a session, but the request still travels straight to BigQuery. The gateway that could inspect the traffic is missing, so no session recording occurs and no inline masking or guardrails are applied.
In this state you have:
- Proper authentication (the token or key proves the caller’s identity).
- No enforcement on the data path, because the data path is the direct connection to BigQuery.
- No replayable record of what the agent actually did.
How hoop.dev provides session recording for BigQuery
hoop.dev inserts a Layer 7 gateway between the autonomous agent and the BigQuery service. The gateway runs as a network‑resident agent close to the data store and is configured with the BigQuery credential. The agent that launches the job never sees that credential; it authenticates to hoop.dev with an OIDC token issued by your identity provider.
When the job initiates a query, hoop.dev validates the token, extracts group membership, and then proxies the request to BigQuery. While the request passes through the gateway, hoop.dev records the full SQL statement, the response payload, timestamps, and the identity that originated the call. hoop.dev stores the session record in a log that can be replayed on demand.
Because hoop.dev is the only point where traffic is inspected, it can also enforce additional guardrails, such as masking PII fields or blocking dangerous functions, but the focus of this article is the session recording capability. In practice, hoop.dev:
- Captures each query and result: The complete interaction is persisted for later review.
- Attributes activity to a specific identity: The OIDC token ties the session to a user or service account.
- Provides replayability: You can re‑run the recorded query in a sandbox to verify its impact.
- Supports compliance evidence: Auditors can see who ran what, when, and what data was returned.
All of these outcomes exist only because hoop.dev sits in the data path. The identity verification step alone does not produce a record; the recording happens at the gateway.
Getting started
Deploy the hoop.dev gateway using the Docker Compose quick‑start, or follow the Kubernetes deployment guide if you run in a cluster. Register your BigQuery connection in the hoop.dev console, supply the service‑account credential (or enable GCP IAM federation for per‑user OAuth), and enable the session recording flag for that connection. Once the gateway is running, point your autonomous jobs at the hoop.dev endpoint instead of the native BigQuery endpoint.
For step‑by‑step instructions, see the getting‑started guide and the broader feature overview on the learn page. The open‑source repository contains the full deployment manifests and example configurations.
FAQ
Does hoop.dev store query results?
hoop.dev records the query text and the response payload for the purpose of audit and replay. The storage is separate from BigQuery, and retention policies can be configured to meet your compliance needs.
Can I still use native BigQuery client libraries?
Yes. The client libraries connect to the hoop.dev endpoint just like they would to the standard BigQuery endpoint. No code changes are required beyond updating the host address.
What happens if the gateway is unavailable?
Because hoop.dev is the only path to BigQuery, a temporary outage will block access. This is intentional: it prevents unrecorded queries from slipping through. You can configure high‑availability deployments to minimize downtime.
By placing a recording layer directly in the data path, hoop.dev turns opaque agent activity into a transparent, searchable audit trail.
Explore the source code and contribute on GitHub.
