Imagine a team that can run chain‑of‑thought prompts on a large language model while a shadow AI silently audits every step, redacts sensitive answers, and blocks risky commands before they reach the model. In that world, developers get the creative benefits of multi‑step reasoning without ever worrying that a stray prompt might leak secrets or violate policy.
Why chain‑of‑thought is vulnerable today
Chain‑of‑thought prompting asks an LLM to think aloud, breaking a problem into a series of intermediate steps. The technique improves accuracy, but it also expands the attack surface. Every intermediate answer is a new data point that can contain credentials, personal identifiers, or proprietary logic. Most teams run these prompts directly against the model, trusting that the model itself will not expose anything. In practice, there is no built‑in audit log, no real‑time redaction, and no way to require a human to approve a particularly risky reasoning branch. The result is a blind spot where policy violations can slip through unnoticed.
What a shadow AI adds – and what it still leaves open
Introducing a shadow AI means attaching a silent observer to the LLM request pipeline. The observer watches the chain‑of‑thought exchange, flags disallowed content, and can request approval before the next step proceeds. This fixes the lack of visibility and the inability to intervene. However, the original request still travels straight to the model service. The model itself does not enforce any of the shadow AI’s rules, and there is still no guaranteed place where the policy check is irrevocably applied. Without a dedicated gateway, a compromised client could bypass the observer, and no immutable session record would exist.
hoop.dev as the enforcement gateway for shadow AI
hoop.dev sits in the data path between the caller and the LLM service. By proxying the request, hoop.dev guarantees that every chain‑of‑thought exchange passes through a single, controllable point. At that point hoop.dev runs the shadow AI logic, applies inline masking to any sensitive fields, records the full session for replay, and can pause execution for a just‑in‑time human approval. Because the gateway is the only route to the model, the enforcement outcomes exist only because hoop.dev is present in the path.
In practice, a user authenticates with an OIDC identity provider. hoop.dev validates the token, extracts group membership, and then forwards the request to the LLM only after the shadow AI has cleared each step. If a step contains a secret, hoop.dev masks it before it is logged. If a step attempts a prohibited operation, hoop.dev blocks it and optionally routes it to an approver. hoop.dev stores the entire interaction as an audit record that can be replayed for compliance checks.