The cluster went dark. No public IPs, no open ports, no way in except through the gates you built. This is where Databricks access control lives or dies—inside a VPC, private subnets locked down, traffic flowing only through a hardened proxy.
When you deploy Databricks in a private network, you’re taking control of every byte. The control plane stays out on the managed side. Your data plane sits in your AWS or Azure VPC. Isolation is not just a setting—it’s architecture. Private subnets keep worker nodes invisible to the public internet. A proxy handles all outbound traffic. No direct exposure. Every connection is intentional. Every access request is filtered.
To make it work, start with strict network policies. Configure your VPC with private subnets for all Databricks clusters. Route public access only through a secure bastion or proxy. Block 0.0.0.0/0 at the subnet level for inbound traffic. Control outbound routes with NAT or a custom proxy layer. This is not theoretical—misconfigurations here open direct paths to your data.
Access control is layered. Use workspace-level permissions and SCIM-based identity sync for user roles. Enforce cluster ACLs so only approved principals can start jobs or attach notebooks. Wrap it in table ACLs and Unity Catalog for fine-grained data governance. The narrower the permission scope, the smaller the blast radius.
The proxy is where control meets enforcement. It’s the only bridge from private subnets to the outside world. Centralize logging here. Use TLS for all routes. Terminate certificates only inside the VPC. Monitor traffic in real time and trigger automated guardrails on patterns that drift from baseline. The proxy is not a single point of failure—it’s your single point of truth for what gets in and out.
Combined, VPC isolation, private subnet placement, and proxy-based control create a Databricks environment that is locked down yet functional. Jobs run without disruption, but the outside can’t touch the inside. Engineers can move fast without moving recklessly.
If you want to see this kind of secure, proxy-driven private deployment in action without spending weeks wiring it up from scratch, try it live on hoop.dev. In minutes you can watch a Databricks access control setup—VPC, private subnets, proxy included—up and running, ready to handle production traffic the right way from day one.