Without a reliable audit trail, a Retrieval‑Augmented Generation (RAG) deployment can become the weakest link in a financial institution’s regulatory posture.
The FFIEC expects concrete proof that every request to an external model is authorized, logged, and that any sensitive data returned is protected. Auditors look for immutable records of who asked what, when the request was approved, the exact response delivered, and evidence that personally identifiable information (PII) was masked according to policy. Missing any of these artifacts can trigger findings that delay audits or lead to penalties.
Many teams treat RAG as a black‑box service. Engineers store prompts in application logs, capture responses in temporary files, and rely on downstream monitoring to spot anomalies. This approach satisfies the need for a quick prototype but falls short of FFIEC’s “audit trail” and “data protection” requirements. Teams often overwrite logs, erase history, and lose the link between a request and the user who issued it. The logs also lack user identity and provide no mechanism to enforce approval before a risky query reaches the model.
ffiec evidence requirements for RAG
To meet the FFIEC framework, an organization must produce three core artifacts for every RAG interaction:
- Access record: a timestamped entry that ties the request to a specific user or service account, including the authentication method used.
- Approval log: evidence that any request flagged as high‑risk (for example, queries that could extract confidential customer data) was reviewed and approved by an authorized reviewer before execution.
- Response audit: a tamper‑evident record of the model’s answer, with any regulated PII masked or redacted according to the institution’s data‑handling policy.
These artifacts must be retained for the period defined by the institution’s data‑retention schedule and must be retrievable in a format that auditors can review without additional tooling.
Why a gateway is the only reliable control point
Identity and least‑privilege provisioning are necessary foundations. An OIDC or SAML identity provider can confirm who is making the request, and role‑based policies can limit which users may call the RAG service. However, without a control point that sits on the data path, the request can still bypass any policy enforcement. The request travels directly from the client to the LLM endpoint, leaving no place to inject approval checks, masking, or immutable logging.
Placing a Layer 7 gateway between the identity layer and the model creates the single place where every packet can be inspected, altered, or blocked.
How hoop.dev captures the audit trail
hoop.dev is an open‑source identity‑aware proxy that sits on the data path for RAG workloads. When a user authenticates through an OIDC provider, hoop.dev validates the token, extracts group membership, and then forwards the request to the LLM only after applying the configured policies.
During the forward, hoop.dev records the full request payload, the authenticated user ID, and the exact time of the call. If the request matches a high‑risk policy, the gateway routes it to a human approver and logs the approval decision before the query reaches the model. hoop.dev intercepts the response, scans it for regulated fields, and masks any PII in real time. hoop.dev stores the masked response and the original payload as a session record that can be replayed for audit purposes.
Because hoop.dev owns the credential used to talk to the LLM, the client never sees the secret, eliminating the risk of credential leakage. hoop.dev performs all enforcement outcomes – approval, masking, blocking, and session recording – inside the gateway, ensuring that the evidence cannot be altered by the client or the model.
Artifacts that satisfy the FFIEC audit
When a RAG request passes through hoop.dev, the system automatically produces the following evidence:
- Session log: a chronologically ordered record that includes user ID, request body, approval status, and masked response.
- Approval audit: a separate entry that captures who approved a high‑risk query, the justification provided, and the timestamp of the decision.
- Data‑masking report: a diff that shows which fields were identified as PII and how they were redacted, proving compliance with data‑protection policies.
- Access summary: aggregated metrics that demonstrate who accessed the RAG service, how often, and under what policy conditions.
hoop.dev stores the logs in a storage backend configured by the deployment, preserving the audit trail for the required retention period. Auditors can query the store directly or export the logs in CSV or JSON for review, fulfilling the FFIEC requirement for “auditability of access and data handling”.
Integrating hoop.dev with existing RAG pipelines
Because hoop.dev proxies standard HTTP traffic, teams insert hoop.dev in front of any existing RAG client without code changes. Teams simply point their HTTP endpoint to the gateway’s address, and hoop.dev handles authentication, policy enforcement, and logging. hoop.dev leaves the existing model‑serving infrastructure untouched, preserving performance characteristics while adding the compliance layer.
You can find a quick‑start guide in the getting‑started documentation. The learn section guides you through policy definition, masking rules, and approval workflow configuration.
FAQ
What evidence does hoop.dev produce for FFIEC?
It generates session logs, approval records, masking reports, and access summaries that map directly to the FFIEC audit‑trail, access‑control, and data‑protection requirements.
Can hoop.dev work with my existing RAG service?
Yes. hoop.dev acts as a transparent HTTP proxy, so any RAG client that can reach an HTTP endpoint can be routed through the gateway without modifying application code.
How does hoop.dev ensure that PII is masked?
The gateway applies configurable masking rules to the model’s response before it is returned to the caller. The masking operation and the fields affected are recorded in the masking report, providing verifiable evidence of data‑protection compliance.
To explore the source code, contribute, or run a self‑hosted instance, visit the hoop.dev GitHub repository.
