An offboarded contractor still has a personal API key baked into a CI pipeline that pushes prompts to a self‑hosted LLM. This is a classic case of data exfiltration. The pipeline runs nightly, sending raw customer queries and receiving model completions that are written to a shared bucket. When the contractor’s access is revoked, the token remains active, and the job continues to exfiltrate proprietary data without anyone noticing.
Self‑hosted models give organizations direct control over inference, but they also inherit the same privilege‑escalation and credential‑leakage problems that plague traditional services. A single over‑scoped token, a hard‑coded secret, or an unattended agent can become a conduit for data exfiltration. Because the model itself does not enforce who can see which response, the responsibility falls to the surrounding infrastructure.
Typical mitigation steps focus on rotating secrets, tightening IAM policies, and scanning logs for anomalies. While necessary, those steps leave a critical gap: the actual data stream between the user (or automation) and the model is never inspected. Without a control point that can see each request and response, an organization cannot reliably block or mask sensitive fields, nor can it produce an immutable record of what was sent and received.
Why data exfiltration is a hidden threat in self‑hosted deployments
Self‑hosted AI workloads often sit behind internal networks, behind firewalls, and behind CI/CD pipelines. Engineers treat the model endpoint like any other internal service, reachable via a TCP socket or HTTP request. This perception leads to three risky patterns:
- Credentials are stored in code repositories or environment files that persist after staff turnover.
- Automation scripts run with broad permissions, allowing any downstream service to invoke the model.
- Audit trails capture only high‑level metrics (request count, latency) but not the payload itself.
Each pattern creates an opportunity for data exfiltration. An attacker who compromises a CI runner can issue a series of prompts that embed confidential identifiers, then capture the model’s answers. Because the payload travels uninspected, the organization has no chance to mask personally identifiable information (PII) or proprietary code before it leaves the network.
What an effective control plane must provide
To stop data exfiltration at the source, the control plane must sit directly on the data path. It needs to:
- Authenticate every request using a federated identity provider, ensuring the caller is known.
- Enforce just‑in‑time (JIT) approvals for high‑risk prompts, requiring a human to sign off before the request reaches the model.
- Apply inline masking to responses, stripping or redacting fields that match sensitive patterns.
- Record each session with full request and response payloads, creating a replayable audit log.
- Allow policy updates without redeploying the model itself, keeping enforcement independent of the inference engine.
Only a gateway that intercepts traffic can guarantee that every piece of data passes through these checks. A pure IAM or secret‑management solution cannot see the payload, so it cannot block or mask it.