Checkmk High Availability
You set up checkmk high availability by pairing two servers in an active-passive cluster. A shared virtual IP address and synced storage move tasks when one node fails. Many teams build this cluster to keep alerts running during host outages. Yet admins often share broad root keys across both nodes to speed up repair work. Wide permissions let one bad command break both the active node and the standby node. How checkmk high availability works in practice A standard cluster uses two Linux host
High Availability: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You set up checkmk high availability by pairing two servers in an active-passive cluster. A shared virtual IP address and synced storage move tasks when one node fails. Many teams build this cluster to keep alerts running during host outages. Yet admins often share broad root keys across both nodes to speed up repair work. Wide permissions let one bad command break both the active node and the standby node.
How checkmk high availability works in practice
A standard cluster uses two Linux hosts or dedicated appliances in the same network. One host runs as the active node and polls all monitored systems. The second host waits as the passive node and watches the active server health. Both nodes link to a shared storage volume or sync disk blocks over the network.
Cluster software manages the virtual IP address and the monitoring service state. Users and agents always send data to the virtual IP address. When the active node stops responding, the cluster manager shifts the virtual IP. It then mounts the shared data folder on the standby host and starts the service. This switch takes seconds and keeps historical check data and alert rules intact.
For teams with many sites, distributed setups offer another layer of resilience. A central site reads status from remote sites, where each remote site runs a local cluster. If the central viewer goes down, remote clusters still run checks and store metrics. Once the central host returns, it pulls the history from the remote nodes.
Why native cluster roles leave an operational gap
Native cluster setups solve hardware failure and keep your alert checks online during maintenance. Built-in web roles also limit who can edit check rules or view host groups. You can map team groups to read-only or admin roles inside the web interface. This stops regular users from changing alert settings through the browser.
Still, managing the cluster itself requires shell logins on the underlying Linux nodes. Native web roles only decide who logs in to the monitoring dashboard. They do not inspect what shell commands or database queries an engineer runs in flight. When an admin logs in via SSH to fix a split-brain disk issue, they hold full root power.
High Availability: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Shared Linux accounts make it hard to see which person ran a destructive command. Also, monitoring outputs and state logs often contain sensitive details like emails or phone numbers. Standard OS roles do not rewrite matched fields in results when engineers inspect raw log files. Teams need a way to check individual commands and mask sensitive fields like card numbers or national IDs during live sessions.
Best practices for safer cluster operations
Start by separating web dashboard duties from Linux cluster maintenance tasks. Give most engineers read-only roles in the web app so they can check host status. Restrict OS shell logins on the cluster nodes to a small group of platform admins. Never use the same shared SSH key across active and passive servers.
Test your failover workflow on a regular schedule during normal work hours. Pull the network link on the active node to confirm the standby node takes over. Check that disk sync finishes cleanly before you move services back to the primary host. Log every shell command run during failover tests so your team can review each step.
Check database and API queries on the wire
Identity tools, roles, and network rules decide who opens a session. Setup alone cannot check what each query does or mask sensitive values in a response once a session starts. When workflows touch Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, or internal HTTP APIs, hoop.dev runs beside those services as an open-source proxy, a sidecar for your Postgres or API. It inspects requests coming in and results going back on the wire.
Your login details and client tools stay as they are while hoop.dev guardrails refuse bad deletes or drops before they run. On responses, hoop.dev data masking rewrites matched sensitive values in memory. Furthermore, hoop.dev holds SOC 2 Type II, is MIT licensed, and writes every query and decision to an audit trail that generates evidence for HIPAA, GDPR, and PCI DSS. See the source on GitHub.
Frequently asked questions
Does Checkmk use active-active or active-passive clustering?
It uses an active-passive model for a single monitoring site. One node runs all checks while the second node waits on standby to take over on failure.
How do remote sites behave during a central node outage?
Remote sites keep polling hosts and saving performance data on their own local disks. When the central server comes back online, it syncs the current status from each remote site.
Why do cluster admins need stricter controls than web users?
Web users only interact with host views and alert rules inside the browser app. Cluster admins use root shell logins where one wrong disk command can wipe shared storage.