Load Balancing Algorithms

Why this matters

Round-robin sounds fair until one backend's GC pause makes it the slowest in the pool — round-robin happily keeps sending it 1/N of traffic anyway. Picking the wrong algorithm means your p99 latency is set by your slowest replica even when 9 out of 10 are healthy.

Demo

Configure and inspect different LB algorithms on a managed cloud load balancer.

Try it yourself

1. Switch an ALB target group from round-robin to least-outstanding and run a load test where one backend sleeps 200ms — observe traffic redistribute.
2. Set up weighted routing 95/5 between two target groups and confirm with 1000 curls that the split lands within 1% of expected.
3. Toggle SourceIP distribution and verify the same client always hits the same backend across 50 sequential requests.

Prompt your AI

Use these three in order. Each builds on the one before.

1. Basics & terminology

In one paragraph, explain the four common load balancing algorithms (round-robin, least-connections, IP hash, weighted) like I'm new to load balancing.

2. Why it works (the mechanism)

Walk me through how a least-outstanding-requests algorithm makes its routing decision per request, and what state it has to track per-backend.

3. Advanced — application & what's next

Given a stateful WebSocket service with sticky sessions and a planned rolling deploy, choose between IP hash and consistent hashing — defend the choice.

References

• AWS — ALB routing algorithms
• Envoy — Load balancing types