Quick Summary
- Load balancers distribute incoming traffic across multiple servers to improve reliability and performance
- Round-robin sends each new request to the next server in sequence
- Health checks automatically remove failed servers from the pool and add them back when they recover
Why Load Balancing Matters
Imagine a popular restaurant with only one waiter. At lunchtime, that single waiter is overwhelmed -- orders pile up, food arrives cold, and customers leave frustrated. Now imagine three waiters, each handling a portion of the tables. Customers get served quickly, and if one waiter calls in sick, the other two can cover. That is load balancing in a nutshell: distributing work across multiple workers so no single one becomes a bottleneck.
In networking, a load balancer sits between clients and a pool of backend servers. When a client sends a request, the load balancer decides which server should handle it. This provides three critical benefits: scalability (add more servers to handle more traffic), reliability (if one server fails, traffic goes to the others), and performance (each server handles fewer requests, so response times stay low).
Load balancing is everywhere on the modern internet. Every major website -- Google, Amazon, Netflix -- uses it. Even small applications benefit from running behind a load balancer, because it enables zero-downtime deployments (update servers one at a time while the load balancer routes traffic to the remaining ones) and provides a single entry point for monitoring and security.
Interactive: Round-Robin Load Balancer
Click "Send Request" to watch the load balancer distribute traffic across three servers. Toggle "Server 2 Down" to simulate a health check failure and see the load balancer skip the dead server.
Load Balancing Algorithms
Round-Robin
The simplest algorithm: requests are sent to each server in turn. Server 1 gets the first request, Server 2 the second, Server 3 the third, then back to Server 1. Weighted round-robin assigns different proportions -- a powerful server with weight 3 gets three times as many requests as a server with weight 1. Round-robin works well when all requests take roughly the same time to process and all servers have similar capacity.
Least Connections
Each new request goes to the server with the fewest active connections. This naturally adapts to varying request complexity -- if one server is processing a slow database query, it accumulates connections, and new requests go elsewhere. Least connections is a better default than round-robin for most real-world applications where request processing times vary significantly.
IP Hash
The client's IP address is hashed to determine which server handles the request. This ensures the same client always reaches the same server -- a form of session persistence without cookies. The downside is that distribution can become uneven if certain IP ranges generate more traffic than others. IP hash is commonly used when server-side session state cannot be shared across the pool.
Health Checks
Health checks are not an algorithm but a critical feature that works alongside any algorithm. The load balancer periodically sends test requests (HTTP GET to a /health endpoint, a TCP connection attempt, or a simple ping) to each server. If a server fails to respond within a timeout or returns an error, it is marked unhealthy and removed from the rotation. When it recovers, it is automatically added back. Health checks are the reason load balancers provide high availability.
Layer 4 vs. Layer 7 Load Balancing
Load balancers operate at different layers of the network stack, and the layer determines what information they can use to make routing decisions. A Layer 4 (L4) load balancer works at the transport layer -- it sees IP addresses and port numbers but not the content of the request. It makes fast decisions based on connection-level information. HAProxy in TCP mode and AWS NLB are Layer 4 load balancers.
A Layer 7 (L7) load balancer operates at the application layer and can inspect the full content of HTTP requests. This enables powerful routing rules: send requests for /api to the API server pool, send requests for /images to the CDN, route mobile users to a different backend. L7 load balancers can also perform SSL termination (decrypting HTTPS at the load balancer so backend servers handle only HTTP), add security headers, and cache responses. NGINX, HAProxy in HTTP mode, and AWS ALB are Layer 7 load balancers.
The tradeoff is speed versus intelligence. L4 load balancers are faster and handle more connections per second because they do less processing per packet. L7 load balancers are more flexible but add latency because they must fully parse each request. Most production architectures use both -- an L4 load balancer at the edge for raw speed and an L7 load balancer behind it for intelligent routing.
Real-World Load Balancing
Cloud Load Balancers
AWS ALB/NLB, Google Cloud Load Balancing, and Azure Load Balancer are managed services that scale automatically. They handle millions of requests per second, distribute traffic across availability zones, and integrate with auto-scaling groups.
Software Load Balancers
NGINX and HAProxy are open-source software load balancers used by companies of all sizes. They run on commodity hardware and can be configured for both L4 and L7 load balancing. Envoy Proxy, built by Lyft, is popular in microservice architectures.
DNS-Based Load Balancing
DNS can distribute traffic by returning different IP addresses for the same domain. Cloudflare, Route 53, and other DNS providers support weighted and geographic routing at the DNS level. This is the simplest form of load balancing but offers the least control.
Common Pitfalls
Session Stickiness Problems
Sticky sessions (sending the same user to the same server) break load distribution. If Server 1 accumulates all the long-lived sessions, it becomes overloaded while Server 2 sits idle. Store sessions in Redis or a shared database instead.
No Health Checks
Without health checks, the load balancer happily sends traffic to dead servers. Users see errors for one-third of their requests (if one of three servers is down). Always configure health checks with appropriate intervals and timeouts.
Single Load Balancer
A single load balancer is itself a single point of failure. Production deployments use pairs of load balancers in active-passive or active-active configurations, with failover handled by VRRP or cloud-provider redundancy.
Frequently asked questions about load balancing
What is a load balancer?
A load balancer sits between clients and a pool of backend servers and distributes incoming requests across them. It provides scalability (add more servers to handle more traffic), reliability (skip dead servers via health checks), and a single stable entry point for clients.
What is the difference between Layer 4 and Layer 7 load balancing?
A Layer 4 load balancer routes based on TCP/UDP connection info — IPs and ports — and is very fast. A Layer 7 load balancer parses the HTTP request and can route by hostname, URL path, headers, or cookies, which is slower but enables smarter routing and SSL termination.
What is round-robin load balancing?
Round-robin sends each new request to the next server in sequence: server 1, server 2, server 3, then back to server 1. Weighted round-robin lets you assign capacity weights so a beefier server takes a larger share. It works well when all requests cost about the same to handle.
How do health checks work?
The load balancer periodically probes each backend — typically an HTTP GET on a /health path or a TCP connect — and tracks pass/fail. When a backend fails N checks in a row it is removed from rotation, and once it passes again it is added back automatically.
What are sticky sessions and when should I use them?
Sticky sessions (session affinity) pin a given client to one backend, usually by cookie or source IP hash. They are needed when servers keep session state in local memory, but they break even load distribution and complicate failover. Storing session state in Redis or a shared database is usually better.
Try load balancing
You just learned how round-robin, health checks, and L4/L7 work. Now put a Layer 4 load balancer in front of web servers, trace a connection through it, and see why the backends only ever see the balancer's address. The lab has no health checks, so it shows distribution, not failover.
Launch the load balancing lab →