When shadow AI is fully contained, every prompt sent to a self‑hosted model is logged, approved, and any sensitive output is filtered before it reaches downstream systems. In that ideal state, engineers can trust that no hidden process is siphoning proprietary data or generating unapproved responses.
Today many teams expose their models through static API keys or long‑lived service accounts that sit in code repositories, CI pipelines, or container images. Those credentials are shared across dozens of services, and any developer with access can invoke the model at will. Because the request travels directly to the model endpoint, there is no central log of who asked what, no way to enforce that only approved prompts are executed, and no guarantee that the model’s response does not contain confidential information. The result is a fertile ground for shadow AI – an unsanctioned instance that silently consumes the model, learns from it, and potentially leaks data.
Why shadow ai threatens self‑hosted models
The core issue is the lack of a control point between the identity that initiates a request and the model that fulfills it. Without a gateway, the following gaps remain:
- Standing access: a token that never expires gives perpetual rights.
- No audit trail: the model sees a request but the organization cannot later prove who made it.
- No inline protection: sensitive fields in the model’s response are sent unfiltered to the caller.
- No approval workflow: risky prompts can be issued without human oversight.
Even if you introduce non‑human identities, role‑based tokens, or federated login, the request still reaches the model directly, leaving the above gaps untouched. The setup alone decides who may start a session, but it does not enforce policy on the data path.
Introducing an identity‑aware gateway
To close the gap, the enforcement point must sit on the network path that carries the request. hoop.dev is built exactly for that purpose. It acts as a Layer 7 proxy that terminates the client connection, validates the caller’s OIDC or SAML token, and then forwards the request to the self‑hosted model only after applying policy.
Setup begins with provisioning an OIDC client in your identity provider, creating service accounts for automated agents, and assigning least‑privilege groups that reflect the intended use of the model. Those steps decide who may attempt a connection, but they do not yet guarantee safe execution.
Data‑path enforcement by hoop.dev
Once the request reaches the gateway, hoop.dev becomes the sole place where enforcement can happen. It records each session, stores a replay‑able log, and can replay the interaction for forensic review. Before a prompt is forwarded, hoop.dev can require a just‑in‑time approval from a designated reviewer, ensuring that only vetted queries are processed. If the request contains a command that matches a risky pattern, hoop.dev blocks it outright.