Load balancers sit in front of nearly every production web service, quietly routing traffic between backend servers so that no single node gets overwhelmed – and when they misbehave, the failure often looks like a mystery outage rather than an obvious load balancer problem. Monitoring load balancers properly means watching health checks, traffic distribution, and failover behavior together, because a load balancer that looks “up” can still be silently sending users to dead backends or draining one node into oblivion while another sits idle.
Why load balancer monitoring gets overlooked
Most teams monitor the servers behind the load balancer religiously – CPU, memory, disk – and then treat the load balancer itself as a black box that “just works.” That’s a mistake. A load balancer is a single point of failure by design, even when you run it in a pair with a virtual IP. If its health check logic is misconfigured, it can keep routing traffic to a backend that’s returning 500s, or worse, pull a perfectly healthy node out of rotation because of a flaky check interval.
A common scenario: an app server starts running slow garbage collection pauses under load, health checks start timing out intermittently, and the load balancer flaps that node in and out of the pool every few minutes. Users see random slowness and occasional errors, but every individual server graph looks fine because the problem is in the routing decisions, not the servers themselves.
Health checks – the foundation everything else depends on
Health checks are how a load balancer decides whether a backend deserves traffic. Get them wrong and every other metric you collect becomes less trustworthy.
Things worth verifying regularly:
Check depth – A TCP-level check that just confirms a port is open will happily pass even when the application behind it is deadlocked or returning garbage. Prefer an HTTP check hitting a real endpoint, ideally one that touches the database or a critical dependency, not just a static health.txt file.
Check interval and thresholds – Too aggressive (checking every second with a one-failure threshold) causes flapping during brief GC pauses or deploys. Too lenient (30-second intervals with five failures required) means users hit a broken backend for over two minutes before it’s pulled. Most production setups land somewhere around 5-10 second intervals with 2-3 consecutive failures required.
Timeout values – If your health check timeout is longer than your actual request timeout, the load balancer will consider a backend healthy right up until real users start timing out on it.
It helps to track health check pass/fail rates as a metric over time, not just the current pool status. A backend that fails and recovers ten times in an hour is telling you something even if it’s “healthy” right now – similar to the patterns described in web server health metrics monitoring, where request-level signals matter more than a simple up/down state.
Traffic distribution – catching the imbalance before users do
The whole point of a load balancer is even distribution, so uneven distribution is itself an incident signal. Watch for:
Request count per backend – If one node is consistently taking 60% of traffic while others sit at 20%, check the balancing algorithm. Round-robin assumes equal-capacity backends; if one server is smaller or slower, weighted algorithms are usually the fix.
Session persistence side effects – Sticky sessions (session affinity) are notorious for creating hot spots. If a marketing campaign drives a traffic spike and sessions are already pinned from an earlier, smaller pool, new capacity added to the pool won’t actually absorb the surge for existing users.
Connection counts, not just request counts – A backend can look fine on requests-per-second but be quietly accumulating open connections because of slow downstream calls, eventually hitting a connection limit and refusing new work.
Comparing latency per backend, not just an aggregate average, is what usually catches this early. If one node’s p95 latency is climbing while the others stay flat, that’s a targeted problem, not a general capacity issue – something covered in more depth in guidance on monitoring network latency across multiple locations, which applies just as well to backend-to-backend comparisons behind a single load balancer.
Failover – the part that only gets tested during a real incident
Failover is the scenario everyone assumes works until it doesn’t. Two failure modes matter here, and they’re different problems:
Backend failover – When a node fails health checks and gets pulled from rotation, does the remaining capacity actually absorb the load without degrading? A pool of four servers running at 70% average utilization loses a lot of headroom the moment one drops out and the other three jump to over 90%.
Load balancer failover – If you’re running an active-passive pair (common with HAProxy plus keepalived, or a cloud load balancer with a standby), does the virtual IP actually move, and how long does that take? DNS-based failover can add minutes of propagation delay that a VIP-based setup avoids.
Test failover deliberately, on a schedule, rather than waiting to discover the gaps during a real outage. Pulling a backend out of rotation manually during a maintenance window and watching how traffic redistributes is a cheap way to validate assumptions. This ties directly into reducing mean time to recovery – failover that’s been tested and monitored resolves incidents in seconds; failover that’s untested often turns a single-node problem into a full outage because nobody trusted the automatic recovery and had to intervene manually.
Common misconception: “if it’s up, it’s fine”
A load balancer reporting all backends as healthy is not the same as traffic being handled well. This is probably the most persistent myth in load balancer monitoring. A pool can be 100% healthy by check status while still serving elevated error rates, because the health check endpoint and the actual application logic real users hit aren’t the same code path. Always correlate health check status with actual error rates and latency from the backends themselves – the two data sources tell different parts of the story, and neither one alone is sufficient.
Setting up alerts that actually help
Alert on the conditions that predict an outage, not just the outage itself:
– Backend pool capacity dropping below a safe threshold (e.g., fewer than 2 of 4 nodes healthy)
– Sustained traffic imbalance between backends beyond a defined percentage
– Health check failure rate trending upward even if the node hasn’t been pulled yet
– Failover events themselves, so someone reviews what happened rather than just being glad it recovered
Keep the thresholds tuned to avoid noise – a single failed health check during a deploy shouldn’t page anyone, but three consecutive failures across two backends should. Overly sensitive alerting on load balancer events tends to get muted fast, which defeats the purpose; the same tuning principles from smarter alert thresholds apply directly here.
Frequently asked questions
How often should load balancer health checks run?
Most production environments use a 5-10 second interval with 2-3 consecutive failures required before pulling a backend, and 2-3 consecutive successes before adding it back. Faster intervals catch problems sooner but increase the risk of flapping during brief slowdowns like GC pauses or deploys.
Can a load balancer show as healthy while users still see errors?
Yes, and it’s one of the most common blind spots. This happens when the health check endpoint doesn’t exercise the same code path as real requests – for example, checking a static file instead of an endpoint that touches the database. Always monitor real request error rates alongside health check status.
What’s the difference between load balancer failover and backend failover?
Backend failover is the load balancer removing an unhealthy server from its pool and redistributing traffic among the rest. Load balancer failover is the load balancer itself going down and a standby instance or virtual IP taking over. Both need to be tested and monitored separately, since a healthy backend pool doesn’t help if the load balancer in front of it is the thing that failed.
Treat the load balancer as a monitored component in its own right, not just plumbing between users and servers. Health check accuracy, traffic distribution, and tested failover together are what turn a load balancer from a single point of failure into the reason an outage never happened in the first place.