Jump Server Architecture
You build a jump server architecture by placing one hardened Linux host in a public subnet. Engineers log in to this relay host via SSH before reaching private servers. Teams set up this layout to keep database and app nodes off the public internet. Still, broad SSH keys and shared system roles often give users too much power inside the network. How a jump server architecture works A standard jump server architecture splits your cloud network into two clear zones. You place the relay host in
SSH Bastion Hosts / Jump Servers + Zero Trust Architecture: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You build a jump server architecture by placing one hardened Linux host in a public subnet. Engineers log in to this relay host via SSH before reaching private servers. Teams set up this layout to keep database and app nodes off the public internet. Still, broad SSH keys and shared system roles often give users too much power inside the network.
How a jump server architecture works
A standard jump server architecture splits your cloud network into two clear zones. You place the relay host in a public zone with a strict firewall rule. This firewall only allows inbound SSH traffic from known office IP addresses. All other backend servers sit inside a private subnet with no public IP addresses.
Private servers only accept SSH traffic that comes from the internal IP of the relay host. When an engineer needs to fix a backend server, they first log in to the relay host. Or they use SSH agent forwarding to bounce through the host in one step. The relay host checks their public key against a local list of allowed users.
Once the user passes that check, they open a second SSH tunnel to the private server. System admins manage this flow by putting user keys on both the relay host and target machines. Some teams also link the relay host to a central directory to check user roles. All network traffic must pass through this single choke point before reaching any private system.
What the native setup solves and what it misses
This classic network layout solves a major problem by shrinking your public attack surface. Attackers on the internet cannot scan or reach your private databases and app servers. You only have to patch and watch one public entry point instead of dozens. Network logs on the relay host also show which IP address logged in and when.
Yet this network setup leaves a wide gap once a user logs in. A relay host only decides who can cross the network perimeter at the start. It cannot inspect or control the commands an engineer runs inside the final SSH session. Once a user reaches a private host, their shell role dictates everything they can do.
If an engineer has root rights on the target machine, they can delete files or dump tables. The jump server sees encrypted packets passing through, not the exact shell commands or database queries. Also, if a user stores private keys on the relay host, a breach there exposes every backend server. Moving SSH keys across many machines takes hours of manual work when people leave the team.
SSH Bastion Hosts / Jump Servers + Zero Trust Architecture: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Hardening your relay host setup
You can reduce some of these risks by tightening rules on the relay host itself. First, never store private SSH keys on the jump server disk. Have engineers forward their local keys from their own laptops for each session. Second, turn off root login and password checks so every user needs a strong key.
Third, remove compilers, package managers, and extra shell tools from the relay host operating system. A user should only bounce through the host, not run local workloads or store files on it. Finally, ship system auth logs off the box right away to a secure log server. This stops a bad actor from wiping their login trail if they break into the host.
Even with these steps, network perimeter controls cannot manage live command risk or hide sensitive data. When an engineer queries a database through an SSH tunnel, raw emails and card numbers return in plain text. Only controls that work at the command and query level can rewrite those fields in results.
Check database and API queries on the wire
Identity tools, roles, and network rules decide who opens a session. Setup alone cannot check what each query does or mask sensitive values in a response once a session starts. When workflows touch Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, or internal HTTP APIs, hoop.dev runs beside those services as an open-source proxy, a sidecar for your Postgres or API. It inspects requests coming in and results going back on the wire.
Your login details and client tools stay as they are while hoop.dev guardrails refuse bad deletes or drops before they run. On responses, hoop.dev data masking rewrites matched sensitive values in memory. Furthermore, hoop.dev holds SOC 2 Type II, is MIT licensed, and writes every query and decision to an audit trail that generates evidence for HIPAA, GDPR, and PCI DSS. See the source on GitHub.
Frequently asked questions
What is the difference between a jump server and a firewall?
A firewall filters network packets by IP address and port number. A jump server is an actual host where users log in to reach private machines.
Should engineers store SSH keys on the relay host?
No, storing private keys on the relay host creates a major security risk. Anyone who breaches that single machine can steal the keys and reach every private server.
Can a standard SSH relay host block dangerous Linux commands?
No, a standard SSH relay host only forwards network traffic to the target server. It does not read or block individual commands inside the encrypted shell session.