All posts

Session recording for AI coding agents on Datadog

An AI coding agent is tasked with automatically generating monitoring dashboards in Datadog based on recent deployment metrics. The agent runs as part of a CI pipeline, authenticates with a hard‑coded Datadog API key, and pushes dashboards without any human review. Because there is no session recording, the team cannot see who triggered which change, and when the pipeline fails, the resulting dashboards contain stale or erroneous queries that flood the team’s alerts. Because the API key is shar

Free White Paper

AI Session Recording + Single Sign-On (SSO): The Complete Guide

Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.

Free. No spam. Unsubscribe anytime.

An AI coding agent is tasked with automatically generating monitoring dashboards in Datadog based on recent deployment metrics. The agent runs as part of a CI pipeline, authenticates with a hard‑coded Datadog API key, and pushes dashboards without any human review. Because there is no session recording, the team cannot see who triggered which change, and when the pipeline fails, the resulting dashboards contain stale or erroneous queries that flood the team’s alerts.

Because the API key is shared across multiple pipelines and even across teams, any compromise gives unrestricted write access to every Datadog resource. There is no record of who triggered which dashboard change, and if an unexpected alert surge occurs, the team cannot trace the exact request that caused it. The organization tries to mitigate risk by rotating the API key periodically, but the rotation process is manual and often delayed, leaving windows of exposure. Engineers also rely on ad‑hoc scripts that embed the key directly in source control, making revocation difficult.

A more disciplined approach would replace the static key with a service account that obtains short‑lived OIDC tokens from the CI system’s identity provider. The token grants the agent permission to call only the Datadog dashboard‑write endpoint. The token is validated by Datadog, and the request still travels straight from the CI runner to the Datadog API. At this point the request is scoped, but there is still no visibility into the exact payload, no ability to pause a suspicious change, and no immutable replay of the interaction.

The missing piece is a layer that sits between the agent and the Datadog API, inspecting every request and recording the full exchange. hoop.dev provides exactly that. It acts as an identity‑aware proxy for Datadog, holding the service‑account credential and presenting a short‑lived token to Datadog on behalf of the agent. Because the proxy is the only component that can see the traffic, it can enforce policies and generate audit evidence.

Why the data path matters for session recording

Setup determines who the request is. In this scenario the CI system authenticates to an OIDC provider, receives a token that identifies the pipeline, and forwards the token to the proxy. That identity step is necessary, but it does not provide any enforcement on its own. The real control point is the data path – the gateway that all HTTP calls to Datadog must traverse.

hoop.dev sits in that data path. It terminates the inbound connection from the AI agent, inspects the HTTP method, URL, and body, and then forwards a sanitized request to the Datadog endpoint. Because the gateway is the sole observer, it can record each request and response verbatim, preserving the exact sequence of API calls made by the agent.

How session recording works for Datadog

When the AI agent initiates a dashboard‑creation call, hoop.dev captures the full request payload, including any JSON that defines the dashboard layout. After the request is forwarded, the proxy also records the Datadog response, which contains the newly created dashboard ID and any error messages. The entire exchange is stored as a replayable session that can be retrieved later for forensic analysis or compliance reporting.

Continue reading? Get the full guide.

AI Session Recording + Single Sign-On (SSO): Architecture Patterns & Best Practices

Free. No spam. Unsubscribe anytime.

Because hoop.dev controls the connection, it can also apply inline masking to redact any secret values that might appear in logs, such as embedded API keys or tokens. The masking policy is defined once in the gateway configuration and is applied automatically to every recorded session.

Integrating the gateway with your CI pipeline

The integration begins with registering a Datadog connection in hoop.dev. The registration includes the Datadog API endpoint, the service‑account credential that the gateway will use, and the scopes that define which Datadog APIs are allowed. The CI pipeline is then pointed at the gateway’s endpoint instead of the raw Datadog URL. From the pipeline’s perspective nothing changes – it still uses the standard Datadog client libraries – but the traffic now flows through the gateway.

Setup also requires configuring the OIDC client that the CI system will use to obtain short‑lived tokens. Those tokens are presented to hoop.dev, which validates them against the identity provider and maps the token’s claims to the appropriate Datadog permission set. This mapping ensures that even if a token is compromised, the gateway will only allow the actions explicitly permitted for that pipeline.

Benefits beyond raw logs

  • Immutable replay: Because hoop.dev records the exact request and response, you can replay a session to understand what changed a dashboard.
  • Policy enforcement: The gateway can block a dashboard creation if it contains disallowed widgets or exceeds size limits, preventing accidental alert storms.
  • Just‑in‑time approval: For high‑risk changes, hoop.dev can route the request to a human approver before forwarding it to Datadog.
  • Audit evidence: The recorded sessions satisfy auditors who need proof of who changed what and when, without relying on Datadog’s internal logs alone.

Getting started

Start with the getting‑started guide to deploy the gateway, register a Datadog connection, and configure OIDC authentication. The guide walks you through the high‑level steps without exposing any low‑level commands.

For deeper details on policy definition, masking, and approval workflows, explore the learn section of the documentation.

FAQ

Does hoop.dev store the Datadog API key?

No. The gateway holds the credential internally and never exposes it to the AI agent or the CI runner. The agent only sees short‑lived OIDC tokens.

Can I view a recorded session after the pipeline finishes?

Yes. hoop.dev provides a searchable UI and API to retrieve any recorded session, allowing you to replay the exact request/response flow.

Is session recording optional?

Recording is enabled by default for every registered connection, because it is the only way to guarantee an immutable audit trail. You can adjust retention policies, but the capture itself cannot be disabled without removing the gateway.

Explore the source code and contribute on GitHub.

Open source

Save the open-source gateway for agent data access

Hoop is MIT-licensed infrastructure for controlling how AI agents reach production data. Star hoophq/hoop so you can inspect it, deploy it, or share it when your team starts governing agent access.

Star and save the repo →More posts