The servers sat silent, deep inside the VPC. No public route. No open ports. No one gets in unless you build the bridge yourself.
Deploying open source models inside a VPC private subnet demands that bridge. A proxy deployment is often the cleanest, safest way to make these models available without exposing your infrastructure to the internet. Done right, it lets you serve requests, run inference, and scale while keeping your data path locked down.
The pattern is simple: the open source model runs in a container or VM in the private subnet. A lightweight proxy—Nginx, Envoy, HAProxy—sits between the model and any client. The proxy runs in a public subnet or a controlled ingress point with strict security groups. Traffic flows in, through the proxy, into the secure subnet. Private subnets keep the model isolated. The proxy handles TLS termination, request routing, and rate limits.
AWS VPC setups make this pattern common. You define subnets, route tables, and security groups to control all ingress and egress. The model service has no internet access. The proxy has only the ports you choose. For outbound calls, use NAT gateways or VPC endpoints. This reduces attack surface while still integrating with external APIs.
Kubernetes-based deployments can map this same pattern. The model runs on private nodes with no inbound rules. An ingress controller or service mesh gateway acts as the proxy. With Istio, Linkerd, or native ingress, you can enforce mTLS, authentication, and load balancing without exposing the model pods to the public.
When using an open source model like LLaMA, Falcon, or Mistral, the proxy becomes more than a security measure—it’s also a performance checkpoint. Cache frequent responses, handle concurrent requests, and manage resource-intensive inference jobs. This architecture separates scaling concerns: you scale the proxy for throughput, the model for compute.
Logging and monitoring stay outside the public blast radius. Send proxy logs to CloudWatch, Prometheus, or your SIEM. Store model metrics privately. Alerts should trigger on anomalies in request rate, latency, or proxy errors.
A well-built VPC private subnet proxy deployment for open source models weighs almost nothing in maintenance but pays back with security and control. You decide the exact flow, the cost boundaries, and the exposure level.
See this pattern live, deployed in minutes, running securely inside your own VPC—visit hoop.dev and launch it yourself.