Inference and FFIEC Compliance

Why FFIEC expects concrete evidence from inference pipelines

Imagine a data science contractor who finishes a project and leaves the organization, yet the API key used to call a machine‑learning inference service remains active. A nightly CI job continues to invoke the same endpoint, and an over‑privileged service account can pull raw predictions without any human review. When a regulator asks for proof that the bank’s predictive models were accessed only for approved purposes, the answer is often “we have logs somewhere,” but the logs are scattered, incomplete, or missing entirely.

The Federal Financial Institutions Examination Council (FFIEC) requires financial institutions to demonstrate that critical systems – including AI and inference engines that feed credit decisions – are accessed under strict controls. Auditors look for immutable records of who invoked a model, what data was supplied, and what output was returned. They also expect evidence that any sensitive fields in the response (for example, personally identifiable information) were protected, and that privileged commands could be blocked or escalated for approval.

Many teams rely on static credentials stored in configuration files or environment variables. The request travels directly from the client to the inference server, bypassing any central policy point. The architecture provides no place to enforce just‑in‑time approval, no real‑time data masking, and no reliable session recording. In practice, the organization can claim “we have an IAM role” but cannot prove that the role was exercised in a controlled, auditable way.

How hoop.dev creates the audit trail required by FFIEC

hoop.dev is a Layer 7 gateway that sits between identities and the inference endpoint. It does not replace the identity provider; instead, it consumes OIDC or SAML tokens from the existing IdP, verifies the user’s groups, and then decides whether the request may proceed. The gateway is the only place where enforcement can happen, because all traffic to the model server is forced through it.

Once a request reaches hoop.dev, the system records the full session – the request payload, the response, timestamps, and the identity that initiated the call. This recording satisfies FFIEC’s demand for a reliable log of every inference operation. Because the gateway sits in the data path, it can also apply inline masking to any fields that match a PII pattern before the response leaves the controlled zone. The masking happens in real time, ensuring that downstream systems never see raw personal data.

When a request matches a high‑risk pattern – for example, a bulk prediction that could affect many accounts – hoop.dev can pause the call and route it to a human approver. hoop.dev captures the approval workflow in the same audit record, providing a clear justification chain that auditors can review. If a command is deemed unsafe, hoop.dev blocks it outright, preventing accidental or malicious misuse.

Implementation overview

The setup phase begins with provisioning an OIDC client in the organization’s identity provider. The organization grants engineers the minimum groups needed to request inference, and it assigns narrowly scoped roles to service accounts. This setup decides who may start a request, but it does not enforce any policy on the request itself.

The data path is the hoop.dev gateway deployed as a Docker Compose service or a Kubernetes pod near the inference server. An agent runs inside the same network segment, holding the credential that the model server requires. Users never see that credential; they only present their identity token to the gateway.

hoop.dev produces enforcement outcomes exclusively. It records each inference session, masks sensitive response fields, requires just‑in‑time approval for risky calls, and blocks disallowed commands. hoop.dev logs all of these actions in a central audit store that can be exported for FFIEC reporting.

Why the gateway model matters for compliance

• Evidence generation: hoop.dev’s session logs give a complete, query‑level view of every inference request, satisfying the FFIEC requirement for detailed access evidence.
• Least‑privilege enforcement: The identity provider defines who can request a model, while hoop.dev enforces what they can do, ensuring that over‑scoped tokens cannot bypass controls.
• Data protection: Inline masking prevents raw PII from leaving the controlled environment, aligning with FFIEC’s data‑privacy expectations.
• Human oversight: Approval workflows add a documented decision point for high‑impact predictions, which auditors can trace back to a responsible individual.
• Replay capability: hoop.dev can replay recorded sessions for forensic analysis if a regulator asks for a deeper investigation.

All of these capabilities are described in the official documentation. To get started quickly, follow the getting‑started guide. For deeper insight into masking, approval, and audit features, explore the learn section of the site.

FAQ

Does hoop.dev replace my existing identity provider?

No. hoop.dev relies on the tokens issued by your IdP. It only consumes the identity information to make access decisions.

Can I use hoop.dev with any inference framework?

hoop.dev proxies standard wire‑protocol connections, so any model server that speaks HTTP, gRPC, or a database‑like API can be fronted without code changes.

How long are the audit logs retained?

Retention policies are configured in the backend store you choose. hoop.dev ensures each session is recorded; you decide how long to keep the records for compliance purposes.

Explore the source code and contribute to the project at https://github.com/hoophq/hoop.