Context windows that exceed policy can leak sensitive data, putting every compliance program at risk.
National Institute of Standards and Technology (NIST) frameworks such as SP 800‑53 and SP 800‑171 require that any system handling controlled information produce verifiable evidence of who accessed what, when, and under what justification. For machine‑learning workloads that ingest prompt history, the same expectations apply: auditors must be able to trace the flow of user‑provided data through the model, see any transformations applied, and confirm that only authorized personnel triggered the request.
In practice, organizations often grant service accounts or API keys direct access to large language model endpoints. The identity layer may be federated through OIDC, and the request can be tied to a user record, satisfying the “who” part of the audit. However, the request travels straight to the model without any intermediary that can observe the payload. That leaves three gaps that NIST explicitly warns against:
- There is no immutable record of the exact prompt that entered the context window.
- Sensitive fields, such as personally identifiable information (PII) or proprietary code, are not masked before they appear in model responses, violating data‑in‑transit protection.
- Any ad‑hoc or emergency use of the model bypasses the formal approval workflow required for high‑impact operations.
These gaps mean that, even with strong identity controls, an organization cannot produce the audit evidence NIST expects. The request still reaches the model directly, and there is no way to prove that the operation complied with policy, that data was protected, or that a privileged approval was obtained.
Why the data path must enforce policy
The only reliable place to enforce masking, approval, and logging is the data path itself. When a gateway sits between the caller and the target, it can inspect each protocol message, apply rule‑based transformations, and capture a record of the exchange. This approach satisfies the NIST control families for audit and accountability (AU‑2, AU‑6), access control (AC‑2, AC‑6), and system and communications protection (SC‑7).
In a typical deployment, the gateway is configured with a service identity that holds the credential for the LLM endpoint. Users authenticate with an external identity provider, and the gateway validates the token before allowing the request to pass. Because the gateway is the sole point where traffic is visible, it can enforce just‑in‑time (JIT) approvals, mask any fields that match a sensitivity policy, and record the full request‑response session for later replay.
How hoop.dev provides the required evidence
hoop.dev is built exactly for this purpose. It sits in the data path for LLM calls, acting as an identity‑aware proxy that applies policy before the request reaches the model. Because hoop.dev controls the flow, it can:
- Record every session. hoop.dev writes a complete log of the prompt, the model’s response, and the identity of the caller. Auditors can retrieve these logs to demonstrate compliance with NIST audit‑trail requirements.
- Mask sensitive data in real time. When a field matches a configured pattern, such as a social security number, hoop.dev replaces it with a placeholder before the model sees it, and it restores the original value only for authorized downstream consumers.
- Require just‑in‑time approval. For high‑risk prompts, hoop.dev routes the request to an approver. The approval decision is stored alongside the session log, giving clear evidence that the operation was authorized.
- Enforce least‑privilege access. The gateway checks the caller’s group membership against a policy that limits which context windows can be used, satisfying NIST’s least‑privilege principle.
Because all of these controls happen inside the gateway, the evidence they produce is trustworthy. If hoop.dev were removed, the raw request would flow directly to the model, and none of the audit, masking, or approval data would exist.
Getting started with hoop.dev for NIST evidence
To adopt this approach, begin with the getting started guide. Deploy the gateway in the same network segment as your LLM endpoint, configure the connection credential, and point your client libraries to the proxy address. Define sensitivity patterns and approval policies in the configuration UI, then enable session recording. The documentation in the learn section walks through policy design, masking rules, and how to query the audit logs for NIST‑required reports.
All of the configuration is declarative, and the source code is openly available on GitHub. Explore the source on GitHub to see how the gateway intercepts LLM traffic and generates the evidence needed for NIST assessments.
FAQ
Does hoop.dev replace the LLM provider’s own logging?
No. hoop.dev complements existing logs by capturing the exact request that reaches the model and the response that leaves it. This extra layer satisfies NIST’s requirement for a complete audit trail.
Can I use hoop.dev with any LLM service?
hoop.dev supports any HTTP‑based LLM endpoint. As long as the model can be reached through a standard proxy, the gateway can enforce the same policies.
How does hoop.dev ensure that masked data is not leaked?
The masking engine runs inside the gateway before the payload is sent upstream. The original value never leaves the gateway, and only authorized downstream processes can request the unmasked data, which is recorded in the session log for audit.
