CD2: Running many containers together
Several copies behind one address
🎯You'll be able to
Run several copies of an app behind a load balancer, and explain how new copies are found and why the app must not keep state in memory.
🏠The everyday picture
A railway station has one "Tickets" sign and several counters under it. Passengers queue at the sign and go to whichever counter is free. If a counter closes, the queue moves to the others. A counter can't keep your half-finished booking in its drawer, because next time you may reach a different one.
📘What it really is
-
A load balancer receives every request at one address and passes it to one of several copies (also called instances, replicas or backends). Benefits: more capacity, and one copy can fail or be replaced while the others serve.
-
In RouteRani, nginx is the load balancer. Its configuration (
infra/nginx.conf) does three important things: resolver 127.0.0.11 valid=2sandset $app http://app:3000; proxy_pass $app;make nginx look upappagain every 2 seconds. Without the variable, nginx looks the name up once at start and never notices new copies;proxy_next_upstream error timeout http_502 http_503retries a request on another copy if one fails or is shutting down;-
proxy_next_upstream_tries 3limits the retries. -
Stateless copies: each request must carry or look up everything it needs (in the database, a cache or a token). If a copy keeps sessions or uploaded files in memory or on its own disk, users lose them when they reach a different copy.
-
Sticky sessions (sending one user to the same copy) hide state problems but break when that copy is replaced. Prefer shared state.
-
Health checks decide membership: a copy that isn't ready shouldn't get traffic (CD1 Lesson 4). Cloud load balancers do this with their own checks; RouteRani uses Docker health checks plus nginx retries.
-
How many copies? At least two for any service that matters, so one can fail or be replaced. More for capacity: measure one copy's limit with a load test and divide (CD9).
📝A worked example
Input: after Project 1, run node tools/load.js http://localhost:8080/api/routes 20 10 with 2 copies, then set app_count = 3,
terraform apply, and run it again.
Expected output: the load tool prints per_second, p95 and failed: 0 each time, and the x-served-by header shows 2, then 3 different
copies. On the 4-core test machine, nginx with RouteRani served about 1,100 to 1,160 requests per second (measured 1 November 2026); your
numbers depend on your computer, and more copies help only while the computer has spare CPU.
🌍Where it's used
- Every high-traffic site: e-commerce sales, exam results, cricket score apps.
- Cloud load balancers: AWS Application Load Balancer, Google Cloud Load Balancing, Azure Load Balancer.
- Kubernetes Services spread traffic over pods the same way (CD5).
📝Case scenario
A shopping site stored each user's cart in the memory of the copy that served them. When they went from one copy to four for a festival sale, users' carts kept "emptying" as requests landed on different copies. Moving carts into Redis made any copy able to serve any request, and the team could add copies freely.
🖱️Try it
With 2 or more copies running, stop one with docker stop while node tools/load.js http://localhost:8080/api/routes 20 10 runs. Success:
failed stays at 0 because nginx retries on the other copy, and docker ps shows the remaining copy still healthy.
⚠️Common mistakes
- nginx
proxy_pass http://app:3000;with no variable. Sign: new copies get no traffic until nginx restarts. Fix: aresolverand a variable, or an orchestrator's service. - Keeping sessions in memory. Sign: users signed out at random after scaling. Fix: shared session storage or tokens.
- One copy "because traffic is low". Sign: every deploy or crash is an outage. Fix: run at least two.
🛠️For your project
Project 1 test 5 needs at least two healthy copies answering through the load balancer, and Project 3's autoscaler changes the count.
✅Quick check
Why does RouteRani's nginx use a variable in proxy_pass?
Show the answer
So it looks the name app up again regularly and finds copies added or removed later.
Why must copies be stateless?
Show the answer
Any request can reach any copy, so state kept in one copy is missing on the others.