A fully auditable inference pipeline lets you prove to auditors that every request, data transformation, and model output is traceable and governed. SOC 2 expects a complete record of who accessed a system, what operation was performed, and whether any sensitive data was exposed. For inference services, this means capturing the identity that invoked a model, the input payload, the response, and any downstream effects such as database writes or external API calls. The evidence must be immutable, time‑stamped, and linked to the controlling policy so that an auditor can verify both security and processing integrity.
When the evidence is in place, compliance reviewers can see that only authorized service accounts or AI agents performed inference, that each request was approved according to a risk‑based workflow, and that any personally identifiable information (PII) in the response was masked before it left the system. The audit trail also shows that the underlying compute environment remained available and that any failure was logged with sufficient detail to demonstrate proper incident handling. In short, the pipeline becomes a single source of truth for the confidentiality, integrity, and availability criteria that SOC 2 demands.
In many organizations, the reality looks very different. Teams often embed static API keys or long‑lived service credentials inside container images that run the model. Those secrets are shared across multiple environments, copied between developers, and sometimes checked into source control by accident. The inference endpoint is exposed directly on the network, allowing any process with network reach to invoke the model without additional checks. Because the request bypasses a centralized gate, there is no per‑request logging, no real‑time data masking, and no way to enforce just‑in‑time approvals. Auditors are left with a handful of generic server logs that do not tie a specific request to an individual identity or policy decision.
Even when organizations adopt modern identity providers and issue short‑lived OIDC or SAML tokens to their AI agents, the improvement is only partial. The token proves that the caller is allowed to start a connection, but the request still travels straight to the inference server. No component in the path records the exact query, inspects the payload for sensitive fields, or offers a workflow to pause high‑risk calls for manual review. The setup therefore satisfies the authentication piece of SOC 2 but falls short on the logging, masking, and approval requirements that complete the compliance picture.
To close that gap, the control must sit where the request actually flows, between the identity that initiates the call and the inference engine that processes it. By placing a gateway in the data path, every request can be inspected, logged, and, if necessary, altered before it reaches the model. The gateway can enforce just‑in‑time policies, require an approver to sign off on risky inputs, and automatically redact PII from the model’s response. Because the gateway is the only point that sees the clear text of the request and response, it can generate an audit record that ties the operation to a specific identity and policy decision.
hoop.dev fulfills exactly that architectural role. It runs a lightweight agent inside the same network as the inference service and proxies all traffic through a Layer 7 gateway. When a request arrives, hoop.dev validates the caller’s OIDC token, checks the request against configured policies, and records the full session, including input payload, output, and any masking actions. If a policy requires approval, hoop.dev routes the request to a human reviewer before forwarding it to the model. The recorded session can be replayed later for forensic analysis or audit review, providing the concrete evidence SOC 2 auditors expect.
Because hoop.dev is the sole enforcement point, the outcomes it delivers exist only because it is in the data path. hoop.dev masks sensitive fields in the model’s response, blocks commands that violate policy, and requires just‑in‑time approval for high‑risk inputs. It also records each inference session with timestamps and identity metadata, enabling replay and audit without relying on the inference engine’s own logs. Without hoop.dev, the same identity and credential setup would still allow unrestricted access to the model, leaving the compliance gaps unaddressed.
Setting up this architecture starts with deploying the gateway and its network‑resident agent. The official hoop.dev getting‑started guide walks you through a Docker Compose deployment that includes OIDC authentication, policy configuration, and session recording. For deeper policy configuration, see the hoop.dev learn page. Once the gateway is running, you register the inference service as a connection, define the masking and approval policies that match your SOC 2 control objectives, and point your AI agents at the hoop.dev endpoint instead of the raw model address. All of the heavy lifting, credential storage, policy evaluation, and audit logging, remains inside the gateway, keeping the inference service itself simple and focused on model execution.
Because hoop.dev is open source, you can inspect the code, contribute improvements, and align the implementation with your internal security standards. The project’s repository on GitHub provides the full source and a community‑driven issue tracker for ongoing enhancements. Explore the open‑source code on GitHub to see how the gateway captures audit data, applies inline masking, and integrates with your identity provider.
FAQ
How does hoop.dev help with the ‘Processing Integrity’ criterion of SOC 2?
hoop.dev records every inference request and response, including the exact input payload, the model version used, and the output returned. The immutable session log lets auditors verify that the service processed data consistently and that any deviations were captured and investigated.
Can hoop.dev mask PII in model outputs automatically?
Yes. By defining field‑level masking rules, hoop.dev inspects the response before it leaves the gateway and redacts or tokenizes any data that matches the policy, ensuring that sensitive information never reaches downstream systems.
Do I need to change my existing inference code to use hoop.dev?
No. hoop.dev works as a transparent proxy. Your agents simply point to the gateway address instead of the model’s direct endpoint, and the rest of the application code remains unchanged.
