An offboarded contractor still has a copy of a service account key that points at a self‑hosted inference model. A nightly CI job runs with a long‑lived token that can push new weights, restart the container, and read raw predictions. An over‑scoped token baked into a configuration file grants every developer full control over the model runtime. In each case the credential never expires, is stored in plain text, and is used directly against the model without any visibility into who invoked it or what they did. Teams often assume that just-in-time access will automatically solve the risk, but without a dedicated enforcement point the request still lands on the model unchanged, leaving no audit trail, no masking of sensitive outputs, and no way to intervene if a command looks dangerous.
Why just-in-time access matters for self‑hosted models
Self‑hosted models are attractive because they keep data and inference inside the organization’s perimeter. That same isolation, however, makes it easy to fall back on static secrets that circulate among engineers, scripts, and CI pipelines. Just-in-time access promises three core improvements: credentials exist only for the duration of a specific request, the privilege level is limited to the exact operation, and the action is recorded for later review. When applied correctly, the approach reduces the blast radius of a compromised secret, limits lateral movement, and provides the evidence auditors need to verify that only authorized personnel touched the model.
Even with a policy that mandates temporary credentials, the request still travels straight to the model server. The model itself cannot distinguish a legitimate, time‑boxed request from a rogue one because the enforcement logic lives outside the model. Without a gate that inspects the protocol, the system cannot block destructive commands, mask personally identifiable information in responses, or require a human approver for high‑risk operations. The missing piece is a data‑path control that sits between the identity provider and the model.
Putting the enforcement point in the data path
hoop.dev provides that missing control surface. It is a Layer 7 gateway that intercepts every connection to a self‑hosted model, inspects the wire‑protocol, and applies policy before the traffic reaches the target. The gateway runs alongside the model inside the same network, so it never exposes credentials to the user or to the calling process.
Setup – Identity is handled by an OIDC or SAML provider. Engineers obtain short‑lived tokens that convey group membership and intent. hoop.dev validates the token, extracts the identity, and maps it to a set of allowed actions for the requested model. No standing credentials are passed to the model; the gateway supplies its own service credential only for the duration of the approved session.
The data path – All traffic to the model flows through hoop.dev. Because the gateway sits at the protocol layer, it can enforce command‑level guards, require just‑in‑time approvals, and apply real‑time masking to any response that contains sensitive fields such as raw training data or user identifiers.
Enforcement outcomes – hoop.dev records each session, stores a replayable log, and tags every command with the originating identity. It blocks commands that match a deny list, pauses execution for manual approval when a high‑risk operation is detected, and masks fields that match a data‑privacy rule before they leave the model. Without a data‑path gateway you lose the ability to block commands, mask outputs, and create an audit trail. These outcomes exist only because the gateway is in the data path; removing hoop.dev would revert the system to the original state of unchecked, permanent credentials.
Because the gateway is open source, teams can inspect the code, extend the policy engine, or integrate it with existing CI/CD pipelines. The getting started guide walks through deploying the gateway with Docker Compose, configuring OIDC, and registering a model connection. The feature documentation details how to define masking rules, approval workflows, and command‑level policies without writing a single line of code.
FAQ
- Can I use just-in-time access without hoop.dev? You can issue short‑lived tokens, but without a data‑path gateway you lose the ability to block commands, mask outputs, and create an audit trail. hoop.dev supplies the enforcement point that turns temporary credentials into a complete security control.
- Does hoop.dev store any secrets? The gateway holds the service credential needed to talk to the model only for the length of an approved session. Engineers never see the secret, and the credential is never written to disk in clear text.
- How does this help with compliance audits? hoop.dev generates evidence for access‑control standards by logging who accessed the model, when, and what they did. The logs are replayable, searchable, and can be exported to meet audit requirements.
Explore the source code on GitHub.
