Load balancing
🏠The everyday picture
A bank with several open counters and a token machine: each new customer is sent to the next free counter. No counter is overwhelmed while others are idle, and if one counter closes, customers go to the rest.
📘What it really is
- A load balancer spreads incoming requests across several copies of your app (horizontal scaling, Module 70).
- Where: Nginx
upstreamblocks (Module 66), cloud load balancers (Module 68), or platform features (PaaS, Kubernetes). - Methods:
- round robin: each server in turn;
- least connections: the server with the fewest active requests;
- weighted: stronger servers get more;
- hashing (by user or IP): the same user goes to the same server; avoid relying on this ("sticky sessions") because it makes scaling and failures harder.
- Health checks: the balancer checks each copy's
/healthand stops sending traffic to unhealthy ones (Module 70). -
Stateless apps are essential (Module 70): sessions in the database or Redis, files in object storage, so any copy can serve any request.
-
Layers: Layer 4 (by IP and port, fast) and Layer 7 (by HTTP content, can route
/apiand/staticdifferently, terminate TLS). -
Beyond one region: DNS-based or global load balancing sends users to the nearest healthy region; useful for international users or disaster recovery (Module 96).
-
Autoscaling (Module 70) adds and removes copies by load; set sensible minimums and maximums.
🖱️Try it
Run three copies of the booking API with Compose, behind Nginx using least_conn and passive health checks (max_fails,
fail_timeout). Run your k6 load test, then stop one copy during the test and watch errors (there should be few or none) and
latency. Confirm the app is stateless by logging in on one copy and continuing on another.
✅Quick check
Why are sticky sessions best avoided?
Show the answer
They tie users to one server, making scaling and recovery from failures harder; stateless apps don't need them.
What does a load balancer's health check do?
Show the answer
It stops traffic going to copies that aren't healthy.