Putting access controls around Devin: audit trails for AI coding agents (on CI/CD pipelines)

How can you be sure that an AI coding agent like Devin isn’t silently modifying production code during a CI/CD run? Without audit trails, you cannot reconstruct which prompt led to which change. Today many teams drop a large language model into their build pipelines and hand it a service account token that has full write access to repositories and deployment clusters. The token is stored in a shared secret, checked into the pipeline configuration, and any developer with pipeline edit rights can trigger the agent. Because the connection goes straight from the CI runner to the target system, there is no record of which prompt caused which change, no way to mask secrets that the model might echo back, and no gate to stop a destructive command. Auditors see only the final state of the repository, not the step‑by‑step interaction that produced it.

Why audit trails matter for AI coding agents

CI/CD pipelines are the new perimeter for software delivery. When an autonomous agent writes code, creates pull requests, or pushes containers, each action becomes a potential compliance event. Without a verifiable log, a security team cannot answer questions such as:

• Who instructed the agent to change a configuration value?
• Did the agent expose a secret in a log line?
• Was a risky command, such as a force‑push, approved before execution?

Regulatory frameworks and internal policies often require a complete, immutable trail of privileged activity. In the context of AI‑driven automation, the trail must capture not only the final artifact but also the conversational exchange that led to it. That is why a dedicated audit‑trail mechanism is a prerequisite before granting an agent any production‑level permissions.

Implementing audit trails with hoop.dev

Enter hoop.dev. hoop.dev is a Layer 7 gateway that sits in the data path between the CI runner (or any client) and the target infrastructure. By placing the gateway on the network edge, hoop.dev becomes the only place where traffic can be inspected, recorded, and altered before it reaches the downstream system.

Setup begins with standard identity federation. The CI system authenticates to hoop.dev using an OIDC token issued by your organization’s identity provider. That token conveys the service account’s group membership, which hoop.dev uses to decide whether the request may start. The token itself does not grant any direct access to the repository or cluster; it merely proves identity.

Once identity is verified, the request is forwarded to the gateway. The gateway holds the credentials needed to talk to the Git server, the container registry, or the Kubernetes API. Because the credentials never leave the gateway, the AI agent never sees them, satisfying the “agent never sees the credential” principle.

From this point onward, hoop.dev enforces every audit‑trail requirement:

• Session recording: hoop.dev records each request and response byte‑stream, preserving a replayable log of the entire interaction between Devin and the target system.
• Inline masking: when the agent receives a secret, such as a database password, hoop.dev can mask that field in the response before it reaches the agent, preventing accidental leakage into downstream logs.
• Just‑in‑time approval: if a command matches a risk policy (for example, a force‑push to a protected branch), hoop.dev pauses the request and routes it to a human approver. Only after explicit approval does the gateway let the command continue.
• Command blocking: hoop.dev can reject commands that violate policy outright, such as attempts to delete production namespaces, without ever contacting the target.

All of these outcomes exist because hoop.dev is the only component that sees the traffic. The underlying identity provider or the CI system alone cannot provide these guarantees; they lack visibility into the payload.

Getting started

To try this in your own pipelines, follow the getting‑started guide. The guide walks you through deploying the gateway with Docker Compose, registering a Git repository connection, and configuring OIDC authentication. The learn section contains deeper discussions of masking policies, approval workflows, and session replay.

Contributing and extending

hoop.dev is open source and welcomes contributions. Explore the code, file issues, or submit pull requests at the official repository: https://github.com/hoophq/hoop.

FAQ

What if Devin runs in multiple pipelines simultaneously?

Each pipeline establishes its own session through hoop.dev. Because the gateway records every session independently, you can correlate actions across pipelines by filtering on the same service‑account identity or on a shared deployment tag.

Where are the audit logs stored?

hoop.dev records each session in a persistent audit store, providing a replayable log that can be queried for compliance or forensic analysis.

Can I still use existing CI secrets management tools?

Yes. hoop.dev complements secret managers by protecting the secret at the point of use. The gateway retrieves the secret from the manager, injects it into the target connection, and then masks it from the agent’s view, giving you defense‑in‑depth without changing your existing secret‑rotation processes.