Databricks Customer Managed VPC
You set up a databricks customer managed vpc by building your own cloud network first. Then you link that network to your workspace during setup. Most platform teams do this to control IP ranges and firewall rules. They want to place compute nodes inside their own cloud account. Still, network rules only fence the boundary. Overly broad database roles inside the workspace still let users run risky queries. How a databricks customer managed vpc works By default, the platform creates and runs t
Customer-Managed Encryption Keys + GCP VPC Service Controls: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You set up a databricks customer managed vpc by building your own cloud network first. Then you link that network to your workspace during setup. Most platform teams do this to control IP ranges and firewall rules. They want to place compute nodes inside their own cloud account. Still, network rules only fence the boundary. Overly broad database roles inside the workspace still let users run risky queries.
How a databricks customer managed vpc works
By default, the platform creates and runs the compute network for you. That default setup works well for quick tests. Still, enterprise network teams often need strict control over cloud traffic. They want to own the subnets, route tables, and security groups.
In a customer managed setup, you build the virtual network in your cloud account. You size the subnets to fit your expected cluster growth. You also set up outbound routes so compute nodes reach the control plane. Next, you register this network with the platform account console. When you launch a workspace, you pick your registered network profile. The platform then boots all spark clusters inside your private subnets.
This design lets you peer the workspace network with other internal systems. You can route traffic through your own firewalls and inspection tools. You can also block public internet routes from the cluster nodes. Network admins keep full visibility into packet flows and IP allocations.
What network boundaries solve and what they miss
A custom network solves the perimeter problem for cloud data platforms. It stops random outside hosts from reaching your compute clusters. It also helps you meet strict cloud rules for private routing. Your network team can log every packet that leaves a subnet.
Still, network rules do not inspect what happens inside an active user session. Once an engineer or app logs in, the firewall steps aside. The network setup decides where clusters run and which hosts connect. It does not check what individual SQL statements or commands do in flight.
Customer-Managed Encryption Keys + GCP VPC Service Controls: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Inside the workspace, teams often share broad service roles to move fast. A developer with a valid session can query raw tables with sensitive fields. Nothing at the subnet level rewrites emails, phone numbers, or card numbers in results. Network logs show IP packets, not who ran a drop table statement. Similarly, automated scripts and AI agents use static tokens inside the private network. If a script runs a bad update query, the custom subnet allows it. Routing rules cannot pause a risky command and ask a manager for approval.
Why teams pair network rules with query controls
Relying only on subnet design forces teams into a tough trade-off. You either lock engineers out of data or give them full table rights. Building dozens of tiny network zones for each team adds huge operational toil. Subnet masks are a blunt tool for data security.
Modern data teams separate network plumbing from runtime data governance. They use the custom network to isolate cloud compute and meet routing rules. Then they add statement-level checks to govern the actual queries and users. This split keeps the network simple while protecting sensitive records in real time. Engineers keep fast query workflows, and security teams get clear audit logs.
Add runtime guardrails and masking beside your databases and APIs
Login rules, role grants, and CI checks govern who connects to your stack. Yet any allowed script, engineer, or AI agent still runs commands with the full scope of its role. For workloads on Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes, SSH, or internal HTTP services, hoop.dev sits beside the service as an open-source proxy, a sidecar for your database or API.
Before a query runs, hoop.dev guardrails check the statement against your rules. An AI Analyzer you configure can score intent or hold risky calls for human review. Next, hoop.dev data masking rewrites sensitive fields in responses, and each verdict is logged per listener. Explore the MIT-licensed project on GitHub.
Frequently asked questions
When should a team use a databricks customer managed vpc?
Use one when you need custom IP ranges, private links, or strict outbound firewalls. You must set it up before you create the workspace. You cannot move an existing workspace to a new network later without rebuilding it.
Does a custom network hide sensitive data in query results?
No, subnets and security groups only control packet routing between cloud hosts. They do not inspect SQL queries or mask fields like emails and card numbers. You need runtime data controls to rewrite sensitive fields in query outputs.
How do you audit user queries inside a private workspace network?
Cloud network flow logs only record source IPs, destination IPs, and byte counts. To see every SQL statement and who ran it, you need query-level logging. This ties each command back to a specific user identity.