Shadow ai risk from uncontrolled function calls lets large language models act on production resources without any trace.
Most teams hand LLMs a static API key and a hard‑coded endpoint, then trust the model to respect internal policies. The model can invoke storage APIs, spin up compute, or modify databases while the engineering team remains unaware of what was actually executed. Because the call travels directly from the model to the target service, there is no audit log, no opportunity to scrub sensitive fields, and no gate that can pause a risky request for human review.
This situation is the core of the shadow ai problem. The precondition we need to address is simple: we must be able to restrict which functions an AI can invoke, but the request still reaches the backend service unchanged. Without a dedicated enforcement point, the request bypasses any approval workflow, any masking of returned secrets, and any replayable record of the interaction.
Why a gateway is required to stop shadow ai
Only a data‑path component can inspect the wire‑protocol of a function call before it reaches the target. Identity providers and token issuance (the setup phase) decide who the caller is, but they cannot rewrite or block the payload once it is in flight. Placing policy enforcement in the data path guarantees that every request is subject to the same guardrails, regardless of the originating model or service.
When a gateway sits between the LLM and the function endpoint, it can enforce three essential controls: it records the full request and response, it masks any sensitive values that appear in the response, and it requires just‑in‑time approval for operations that exceed a predefined risk threshold.
How hoop.dev enforces policy on function calls
hoop.dev acts as that mandatory data‑path gateway. It receives each function‑calling request, evaluates the caller’s identity, and then applies configurable policies before forwarding the request to the actual service.
Session recording
hoop.dev writes a complete log of every function invocation, including input parameters, timestamps, and the identity that initiated the call. The log is recorded and searchable, giving security teams evidence of every AI‑driven action.
Inline data masking
hoop.dev scans responses for patterns that match secrets, tokens, or personally identifiable information and replaces them with placeholder values before the data reaches the model. This prevents the LLM from learning or leaking sensitive data in subsequent prompts.
Just‑in‑time approval
hoop.dev can route high‑risk calls, such as those that write to a production database or modify IAM policies, to a human approver. The request is paused until an authorized reviewer explicitly grants permission, eliminating accidental or malicious changes.
All of these enforcement outcomes exist only because hoop.dev sits in the data path. Without that gateway, the setup layer would still authenticate the model, but no policy could block, mask, or record the call.
Defining policies for function calls
Policies are expressed in terms of the function name, the caller’s groups, and the risk level of the operation. For example, a policy might allow any model belonging to the "dev‑ops" group to invoke read‑only analytics functions, but require approval for any write‑back operation. Another policy could automatically mask any field that matches a credit‑card regex in the response payload.
Because hoop.dev evaluates policies at the protocol layer, the same rules apply whether the request originates from a custom LLM integration, an automation script, or an AI‑assisted developer tool.
Getting started with the gateway
Deploy the gateway using the official Docker Compose quick‑start, configure OIDC authentication, and register the function endpoint you want to protect. The documentation walks you through defining policies for recording, masking, and approval.
For detailed steps, see the getting‑started guide and the broader learn section. The product page at hoop.dev provides an overview of all supported connectors and guardrails.
When you are ready to explore the codebase, explore the open‑source repository on GitHub. All configuration examples are available there.
FAQ
- Does this approach add latency to function calls? The gateway adds a small, predictable overhead for policy evaluation and logging. In most environments the impact is negligible compared to the network latency of the underlying service.
- Can I use existing identity providers? Yes. hoop.dev works with any OIDC or SAML provider, so you can continue to use your organization’s IdP for authentication.
- What happens to calls that are denied? hoop.dev returns a clear error to the caller and records the denial event, giving you a complete audit trail of attempted shadow ai activity.
- Is it possible to version policies? Policies are stored as configuration files that can be version‑controlled in Git. Updating a policy takes effect immediately for new sessions while existing sessions continue under the policy that was active when they started.
- How does masking affect model performance? Masking occurs on the response stream before it reaches the model, so the model never sees the raw secret. This has no impact on downstream inference performance.
