How can you prove that your long‑term memory layer never leaks PHI? Auditors will ask for concrete evidence that every request, every piece of protected health information, and every decision point is visible, controlled, and retained for the required retention period.
What PHI compliance expects from a long‑term memory system
Protected health information (PHI) is subject to strict federal rules. The core expectations revolve around four pillars:
- Identity‑driven access: Only authorized identities may invoke the memory service, and the identity must be recorded.
- Just in time (JIT) authorization: Access is granted for a limited window and must be approved when the request is high‑risk.
- Audit‑ready logging: Every query, response, and policy decision is logged in a record that cannot be altered for the retention window.
- Data minimization and masking: Sensitive fields are hidden from the caller unless explicitly allowed.
Regulators will examine logs for who accessed what, when, and why. They will also look for evidence that PHI was never exposed in clear text to an unauthorized consumer.
Why long‑term memory makes compliance hard
Large language models (LLMs) and vector stores often act as a “black box” cache. A request can retrieve a snippet that contains a patient’s name, diagnosis, or medication without a clear audit trail. Typical deployments rely on static API keys or service accounts that have broad, standing permissions. This setup leaves several gaps:
- No per‑request identity capture – the service sees only its own credential.
- No inline policy enforcement – dangerous prompts are executed before any review.
- No session replay – if a breach is suspected, there is no way to reconstruct the exact interaction.
- No inline masking – the response may return PHI directly to the caller.
Because the enforcement point is inside the application, auditors cannot verify that the required controls were actually applied.
Embedding the controls in the data path
The reliable way to satisfy the PHI pillars is to place a gateway at the only point where traffic leaves the trusted network and reaches the memory service. That gateway becomes the sole place where identity, policy, and logging intersect.
When a user or an automated agent initiates a query, the request first passes through the gateway. The gateway validates the OIDC or SAML token, checks group membership, and decides whether the request is allowed, needs approval, or must be blocked. If the request is approved, the gateway forwards it to the memory backend using its own credential, ensuring the backend never sees the caller’s secret.
Because the gateway sits in the data path, it can:
- Record every request and response, preserving a record that cannot be altered.
- Apply inline masking rules to strip PHI from the response before it reaches the caller.
- Trigger a human‑in‑the‑loop approval workflow for high‑risk queries.
- Replay the entire session for forensic analysis.
hoop.dev as the PHI ready data‑path gateway
hoop.dev implements exactly this architectural pattern. It is an open source Layer 7 gateway that proxies connections to infrastructure, including long‑term memory services such as vector stores or LLM inference endpoints. The solution is built around three distinct responsibilities:
Setup – identity and least‑privilege provisioning
The organization provisions non‑human identities in its IdP (Okta, Azure AD, Google Workspace, etc.). Those identities receive short‑lived OIDC tokens that hoop.dev validates. The token tells hoop.dev who the caller is and what groups they belong to, allowing the system to enforce fine‑grained policies without ever exposing static credentials.
The data path – the only place enforcement occurs
All traffic to the long‑term memory backend traverses hoop.dev. Because the gateway sits between the caller and the backend, it is the sole point where masking, approval, and command blocking can be applied. The backend never sees the caller’s token, and the caller never sees the backend’s credential.
Enforcement outcomes – evidence generated by hoop.dev
hoop.dev records each session, masks protected fields in real time, requires just in time approval for risky queries, and blocks disallowed operations before they reach the memory store. Those outcomes become the audit evidence auditors demand for PHI compliance. The logs are centrally stored, searchable, and can be exported for any regulatory review.
By placing hoop.dev in the data path, organizations gain a single source of truth for who accessed what PHI, when, and under what policy conditions. The gateway’s replay capability lets a compliance officer reconstruct an entire interaction, demonstrating that PHI never left the controlled environment unmasked.
Getting started with hoop.dev
Deploying the gateway is straightforward. The official getting‑started guide walks you through a Docker Compose deployment, OIDC configuration, and connection registration for a vector store. Once the gateway is running, you point your existing client libraries at the hoop.dev endpoint and gain the full suite of PHI‑focused controls without changing application code.
For deeper technical details on masking policies, approval workflows, and session replay, explore the learn section. The documentation explains how to define PHI‑specific field masks and how to configure just in time approval pipelines that align with your organization’s risk appetite.
FAQ
- Does hoop.dev store PHI itself? No. hoop.dev only proxies traffic and records metadata about the interaction. The actual PHI remains in the downstream memory store, and any masked fields are removed before they reach the caller.
- How does hoop.dev help with PHI audit logs? Because hoop.dev is the sole data‑path gateway, every request and response is logged with identity, timestamp, and policy decision. Those logs satisfy the audit‑ready requirement for PHI compliance.
- Can I use hoop.dev with an existing LLM service? Yes. You register the LLM’s endpoint as a connection in hoop.dev, and all calls to the model will flow through the gateway, gaining masking, approval, and recording without modifying the model or client code.
Ready to add PHI‑ready controls to your long‑term memory layer? Visit the open‑source repository and start building a compliant data‑path today.
