Putting access controls around ChatGPT: audit trails for AI coding agents (on internal SaaS)

When an internal SaaS lets developers ask ChatGPT to write code, the convenience can hide a costly blind spot. Without audit trails, a single stray secret or a mis‑generated query can leak credentials, violate compliance, or introduce a bug that surfaces weeks later. Teams spend days chasing ghosts, and auditors find no evidence of control.

Most organizations grant a service account an API key and let the application call the LLM directly. The request travels straight from the internal service to the public endpoint, bypassing any corporate gate. Engineers see only the final answer; the platform never logs the prompt, the response, or the context in which the model ran. The result is a “black box” interaction that cannot be audited, cannot be approved, and cannot be masked for sensitive data.

Even when the organization adopts best‑practice identity foundations, OIDC tokens for non‑human actors, least‑privilege service accounts, and role‑based group assignments, the request still reaches the LLM endpoint unmediated. The authentication step tells the LLM who is calling, but it does not provide a place to enforce policies, capture an immutable trail, or scrub secrets before they leave the model. In other words, the setup fixes identity but leaves visibility and control completely open.

Why audit trails matter for AI coding agents

An audit trail is the single source of truth for every prompt, every response, and every decision point in the workflow. For AI‑driven code generation, it provides:

• Forensic evidence that can be examined after a security incident.
• Compliance artifacts that satisfy auditors asking for proof of who generated which piece of code.
• Operational insight that helps developers understand why a particular snippet was produced.
• Risk mitigation by ensuring that any secret that appears in a model response can be identified and removed before it reaches downstream systems.

Without such a trail, organizations are forced to rely on manual logging or hope that the LLM provider retains useful data, both of which are unreliable and often non‑compliant.

How hoop.dev builds the audit trail for ChatGPT

hoop.dev acts as a Layer 7, identity‑aware proxy that sits directly in the data path between the internal service and the ChatGPT endpoint. The gateway verifies the caller’s OIDC token, extracts group and role information, and then forwards the request to the model. While the request is in flight, hoop.dev inspects the protocol, records the full prompt and the model’s response, and applies inline masking rules to any pattern that matches a secret. Because the gateway is the only point where traffic passes, it can also enforce just‑in‑time approvals for high‑risk queries before they are sent to the model.

All of these enforcement outcomes, recording each interaction, masking sensitive fields, and requiring approval, exist only because hoop.dev occupies the data path. The setup layer (identity providers, service accounts, IAM roles) determines who may start a request, but hoop.dev is the mechanism that actually creates the audit trail and enforces the guardrails.

Deploying the gateway for internal LLM use

To protect ChatGPT‑driven coding agents, you deploy hoop.dev near the services that invoke the model. The quick‑start guide walks you through a Docker Compose deployment that includes the gateway, an OIDC configuration, and a default masking policy. Once the gateway is running, you point your internal client at the hoop.dev endpoint instead of the public LLM URL. From that point forward, every prompt and response passes through the gateway, where hoop.dev records the interaction, applies any configured masking, and, if needed, routes the request to a human approver.

Because hoop.dev is open source and MIT‑licensed, you retain full control over the storage of audit records and the policies that govern masking. The recorded sessions can be replayed for investigation, and the logs can be exported to your SIEM or compliance platform.

For a step‑by‑step walkthrough, see the getting started guide. Detailed feature descriptions, including how to define masking patterns and approval workflows, are available in the learn section of the documentation.

FAQ

How does hoop.dev capture audit trails for LLM interactions?

hoop.dev records the full request payload (the prompt) and the full response payload before any masking is applied. These records are stored securely and can be queried later for compliance or forensic analysis.

Does hoop.dev store the ChatGPT API key?

No. The gateway holds the credential only in memory while establishing the outbound connection. The key never appears in the client’s environment, and it is never written to persistent storage.

Can I integrate hoop.dev with existing CI/CD pipelines?

Yes. By routing the pipeline’s LLM calls through the hoop.dev endpoint, the pipeline inherits the same audit‑trail, masking, and approval capabilities without any code changes. The pipeline can continue to use its standard client libraries; hoop.dev acts as a transparent proxy.

Ready to add immutable audit trails to your internal ChatGPT coding agents? Explore the source code and contribute on GitHub.