When a contractor leaves a project, the API key they used for a nightly RAG job often remains in a CI secret store, breaking the segregation of duties that should separate data access from model execution. The job continues to pull confidential documents, combine them with a large language model, and write summaries to a shared bucket. In each case the separation between who can access raw data and who can influence model output is blurred, opening the door to accidental leaks or intentional misuse.
Understanding segregation of duties in RAG workflows
Segregation of duties is a control principle that requires distinct individuals or processes to perform separate steps of a critical workflow. For Retrieval‑Augmented Generation the workflow can be broken into three logical stages: data retrieval, prompt construction, and model execution. Each stage touches different risk surfaces – the retrieval stage exposes raw corporate documents, the prompt stage may add context that influences output, and the execution stage produces the final content that may be published.
Why the traditional perimeter model falls short
Most organizations rely on network firewalls and static credentials to protect data stores. Those mechanisms stop traffic from unauthorized IP ranges, but they do not enforce who can combine a document with a model prompt. A single token that grants both read access to a knowledge base and write access to a generation endpoint defeats the purpose of duty separation. Even if identity providers issue short‑lived tokens, the token itself still carries all privileges, and there is no point where the request can be inspected before it reaches the target system.
Setting up the identity foundation
The first layer of defense is to provision identities that are purpose‑built. Create one service account that only has read rights on the document store, and a separate account that can invoke the LLM endpoint. Use OIDC or SAML to federate these accounts into a central directory, and assign them to narrowly scoped groups. This step decides who is allowed to start a request, but it does not stop a compromised credential from being used to issue a dangerous command.
Placing enforcement in the data path
The decisive control point must sit on the traffic path between the identity layer and the backend systems. By inserting a Layer 7 gateway that proxies every RAG request, the organization gains a single place to evaluate policy, mask sensitive fields, require human approval for high‑risk prompts, and record the entire session. Because the gateway sees the full protocol exchange, it can block a prompt that contains prohibited keywords, redact personally identifiable information from responses, and log who approved each step.
How hoop.dev provides the required data‑path controls
hoop.dev implements exactly this gateway model. It authenticates users and service accounts via OIDC, then routes RAG traffic to the configured retrieval and generation endpoints. While the request flows through hoop.dev, the platform can enforce segregation of duties by:
- Allowing only the read‑only identity to reach the document store.
- Requiring an explicit approval workflow before any generation request is forwarded.
- Masking confidential fields in retrieval responses so downstream prompts never see raw secrets.
- Recording the full interaction for replay and audit, giving evidence that duty separation was respected.
All of these outcomes exist because hoop.dev sits in the data path; removing it would return the system to the insecure direct‑connect state described earlier.
Practical steps to adopt the model
- Define the roles needed for your RAG pipeline: a “retriever” role with read‑only access to source stores, and a “generator” role with permission to call the LLM service.
- Register each target (document store, LLM endpoint) in hoop.dev, assigning the appropriate credential to the gateway so that downstream users never see secrets.
- Configure policy rules that bind the retriever role to the retrieval path and the generator role to the generation path. Enable inline masking for fields like SSNs or API keys.
- Set up an approval workflow for any prompt that exceeds a risk threshold, such as those that request more than a certain number of tokens or contain regulated terminology.
- Enable session recording and store logs in a secure audit‑ready location. Use the recorded evidence to satisfy audit requirements for segregation of duties.
Getting started
The quickest way to try this approach is to follow the getting‑started guide on hoop.dev. The documentation walks you through deploying the gateway, defining connections, and applying role‑based policies. For deeper insight into policy features, visit the learn section of the site.
FAQ
Does segregation of duties eliminate all data leakage risk? No. It reduces the attack surface by ensuring no single credential can perform both retrieval and generation, but you still need proper secret management and monitoring.
Can I enforce duty separation without a gateway? You could build custom middleware, but without a single data‑path enforcement point you lose the guarantee that every request is inspected. hoop.dev provides that guarantee out of the box.
Is the solution compatible with existing CI pipelines? Yes. The gateway presents the same network endpoint that your CI jobs already target, so you only need to update the connection string to point at hoop.dev.
Ready to explore the code? View the open‑source repository on GitHub and start building a segregation‑aware RAG pipeline today.
