What Is a Load Balancer? How Load Balancing Works

Load Balancer at a Glance

A load balancer distributes client traffic across a pool of healthy backend resources so that applications can scale horizontally, tolerate backend failures, and provide a more consistent service endpoint.

Backend resources may include:

• Physical servers
• Virtual machines
• Cloud instances
• Kubernetes Pods
• Container workloads
• API services
• Applications running in different data centers

Users normally do not connect directly to these backend systems.

They access a single hostname or IP address, while the load balancer decides which backend should handle each connection or request.

What Is a Load Balancer?

A load balancer is a system that receives incoming traffic and selects an appropriate healthy backend according to routing rules, health information, and load-balancing algorithms.

Its purpose is to make multiple backend resources appear to users as a single service.

This abstraction allows infrastructure teams to:

• Add new capacity without changing the public application endpoint.
• Remove unhealthy servers without changing the client configuration.
• Perform maintenance with reduced service interruption.
• Scale application capacity horizontally.
• Apply centralized traffic-management policies.

What Does a Load Balancer Do?

The primary role of a load balancer is to make available backend capacity accessible to users in a controlled, scalable, and resilient way.

Common functions include:

• Preventing a single backend from receiving excessive traffic
• Supporting horizontal application scaling
• Detecting unavailable backend systems
• Reducing downtime during maintenance
• Supporting controlled application deployments
• Routing traffic between regions or data centers
• Centralizing TLS certificate handling
• Applying application-level routing policies

Why Is Load Balancing Needed?

Single-Server Capacity Is Limited

Every server has finite:

• CPU
• Memory
• Disk I/O
• Network capacity
• Connection capacity
• Application processing capacity

A load balancer allows application capacity to grow by adding new backend resources instead of relying only on vertical server upgrades.

Server Failures Are Inevitable

Hardware, operating systems, networks, and applications can fail.

A load balancer using health checks can stop sending new traffic to an unhealthy backend and continue using the remaining healthy resources.

Maintenance Can Be Performed with Less Downtime

A backend server can be removed from the active pool before maintenance while other systems continue serving users.

Horizontal Scaling Becomes Practical

Vertical scaling makes an existing server larger.

Horizontal scaling adds additional servers or application instances.

The load balancer makes this distributed capacity available through one logical service endpoint.

How Does a Load Balancer Work?

A load balancer accepts an incoming connection or request, identifies the appropriate backend pool, excludes unhealthy targets, selects a backend according to its algorithm, and forwards the traffic to that target.

A simplified flow is:

1. The client connects to the application's hostname or IP address.
2. The request reaches the load-balancing layer.
3. The load balancer determines the correct backend pool.
4. Unhealthy backends are excluded according to health-check status.
5. The configured algorithm selects a backend.
6. The connection or request is forwarded to that backend.
7. The response returns to the client according to the forwarding architecture.
8. The load balancer continues monitoring health and traffic conditions.

Is a Load Balancer Always a Reverse Proxy?

No. Many Layer 7 load balancers operate as reverse proxies, but not every load-balancing architecture is technically a reverse proxy.

In a reverse-proxy architecture, the client connection terminates on the load balancer and the load balancer establishes a separate connection to the backend.

Other load-balancing architectures may use:

• NAT-based forwarding
• Direct server return
• Layer 4 flow forwarding
• Anycast-based distribution

Reverse proxy and load balancer are therefore related concepts, but they are not exact synonyms.

What Are the Main Load-Balancing Algorithms?

A load-balancing algorithm defines how the load balancer chooses a healthy backend for a new connection or request.

Round Robin

Requests are distributed sequentially across backend servers.

Round Robin is simple and predictable and can work well when backend capacity and request duration are relatively similar.

Weighted Round Robin

Different weights are assigned to different backends.

A higher-capacity server can therefore receive a larger share of traffic.

Least Connections

A new connection is sent to the backend with fewer active connections.

This can be useful when connection duration varies significantly.

Weighted Least Connections

Backend capacity and current connection counts are considered together.

Least Response Time

Supported implementations can use backend response time and active-connection information to select a target.

The exact behavior depends on the load-balancing platform.

IP Hash

Client IP information is used to calculate a backend preference.

This can be used for session persistence.

However, if the backend pool changes or the selected backend becomes unavailable, the client may still be routed elsewhere.

Consistent Hashing

Consistent hashing attempts to minimize how many clients or keys need to move to another backend when the backend pool changes.

It can be useful in caching, sharding, and certain state-sensitive architectures.

How Do You Choose a Load-Balancing Algorithm?

There is no single best load-balancing algorithm. The correct method depends on backend capacity, request duration, connection behavior, application state, and traffic patterns.

What Is a Load Balancer Health Check?

A health check is a test used by the load balancer to determine whether a backend is currently suitable to receive traffic.

Health checks can operate at different levels:

• TCP port checks
• HTTP requests
• HTTPS requests
• Application-specific health endpoints
• Custom service-health protocols

For example:

/health

can be designed to verify not only that the web server process is listening, but also whether critical application dependencies are working.

Active vs Passive Health Checks

Active Health Check

The load balancer sends dedicated test requests to backend systems at regular intervals.

After a configured number of failures, a backend can be removed from the active pool.

Passive Health Check

The load balancer uses signals from real client traffic, such as:

• Connection failures
• Timeouts
• HTTP errors

to infer backend health.

Advanced architectures may combine both models.

Why Are Health-Check Settings Important?

An excessively aggressive health-check policy can interpret temporary latency as a backend failure.

A policy that is too relaxed can continue sending users to an unhealthy application.

Important parameters include:

• Check interval
• Timeout
• Healthy threshold
• Unhealthy threshold
• Expected HTTP status
• Application health-endpoint behavior

What Is Connection Draining?

Connection draining stops new traffic from being sent to a backend that is being removed from service while allowing existing connections or requests to complete for a defined period.

During planned maintenance:

1. The backend is placed into a draining state.
2. New requests are sent to other backends.
3. Existing connections are allowed to complete.
4. The backend can then be safely stopped or updated.

Connection draining can reduce user-facing errors during deployments, maintenance, and scale-in operations.

What Is Session Persistence or a Sticky Session?

Session persistence, also known as a sticky session, attempts to route subsequent requests from the same user or session to the same backend.

Common methods include:

• Cookie-based persistence
• Source-IP affinity
• Hash-based routing

Sticky sessions can be useful for legacy applications that store session state inside application-server memory.

What Are the Disadvantages of Sticky Sessions?

Session persistence reduces some of the flexibility of load balancing and can create uneven backend utilization.

For example, source-IP persistence may direct a large number of users behind the same NAT gateway to one backend.

Modern applications often improve scalability by storing session state in:

• A database
• A distributed cache
• An external session store

instead of individual application servers.

This helps application backends remain more stateless and easier to scale.

Layer 4 vs Layer 7 Load Balancer: What Is the Difference?

A Layer 4 load balancer routes traffic primarily using transport-layer information such as IP addresses, ports, and protocols, while a Layer 7 load balancer can make decisions using application-layer information such as HTTP hostnames, paths, headers, and cookies.

What Can a Layer 7 Load Balancer Do?

A Layer 7 load balancer can inspect HTTP-level information and apply rules such as:

• Hostname-based routing
• Path-based routing
• Header-based routing
• Cookie-based routing
• Redirects
• HTTP response manipulation
• TLS termination

For example:

• api.example.com can be routed to API servers.
• /images can be routed to media backends.
• /checkout can be routed to payment services.

Internal vs External Load Balancer

External Load Balancer

An external load balancer receives traffic from the internet or another external network.

It may be used for:

• Public websites
• Customer-facing applications
• Public APIs

Internal Load Balancer

An internal load balancer is reachable only from a private network.

It may distribute traffic between:

• Frontend and backend services
• Internal APIs
• Application and database proxy layers
• Private microservice endpoints

Local vs Global Load Balancing

Local load balancing distributes traffic across resources within the same data center or region. Global load balancing directs traffic across multiple regions, cloud environments, or data centers.

Global traffic management can consider:

• User geography
• Network latency
• Regional health
• Backend capacity
• Routing policy
• Data-residency or compliance requirements

What Is GSLB - Global Server Load Balancing?

Global Server Load Balancing - GSLB - directs users between services hosted in different data centers or regions using information such as health, location, latency, and routing policy.

GSLB can be implemented through:

• DNS-based traffic management
• Anycast
• Global proxy architectures

In DNS-based architectures, failover speed can be influenced by DNS TTL and resolver caching.

Therefore, GSLB should not automatically be interpreted as instant failover under every condition.

How Does a Load Balancer Support High Availability?

A load balancer supports high availability by allowing multiple application instances to be presented as a single service and by redirecting new traffic away from unavailable backends.

However, backend redundancy alone is not enough.

High-availability architecture may also need redundancy across:

• Load balancers
• Application servers
• Networks
• Storage
• Databases
• DNS
• Power infrastructure

Can the Load Balancer Become a Single Point of Failure?

Yes. If all application traffic depends on one load balancer, the load balancer itself can become a Single Point of Failure.

Critical architectures may therefore use:

• Active-passive load balancers
• Active-active load balancers
• Multi-zone managed load balancers
• Anycast-based traffic distribution
• Provider-managed high availability

Active-Active vs Active-Passive Load Balancing

Active-Passive

One load balancer actively handles traffic while a secondary system remains available as a standby.

If the active system becomes unavailable, traffic is moved to the standby instance.

Active-Active

Multiple load balancers actively handle traffic at the same time.

Potential benefits include:

• Additional capacity
• Better resource utilization
• Greater resilience to a load-balancer failure

The design can also introduce additional complexity around traffic distribution and state synchronization.

How Does Load Balancing Relate to Disaster Recovery?

Disaster recovery does not end when systems start successfully in the recovery environment. User and application traffic must also reach that environment.

A failover plan may need to coordinate:

• Applications
• Databases
• DNS
• Firewalls
• Load balancers
• Certificates
• Routing

For the broader recovery architecture, see What Is DRaaS?.

What Is TLS Termination?

TLS termination means the encrypted client connection is terminated at the load balancer instead of at the backend application server.

The load balancer can:

• Perform the TLS handshake.
• Present the server certificate.
• Decrypt incoming traffic.
• Establish a separate backend connection.

Potential advantages include:

• Centralized certificate management
• Reduced TLS processing on application backends
• Layer 7 routing
• Integration with WAF and application-security controls

What Are TLS Passthrough and TLS Re-Encryption?

TLS Passthrough

The load balancer forwards the encrypted connection without decrypting the application traffic.

TLS termination takes place on the backend.

TLS Re-Encryption

The client TLS connection terminates at the load balancer, but the load balancer establishes a new TLS connection to the backend.

This allows Layer 7 processing at the load balancer while keeping backend traffic encrypted.

How Is the Client IP Preserved Behind a Load Balancer?

In proxy-based architectures, the backend may see the load balancer's address instead of the original client IP.

Client information can be passed using mechanisms such as:

• X-Forwarded-For
• Forwarded header
• PROXY Protocol
• Source-IP preservation

Preserving the correct client identity can be important for:

• Security logging
• Rate limiting
• Fraud detection
• Geographic policies

Why Are Load Balancer Timeouts and Retries Important?

Incorrect timeout and retry policies can transform a small backend problem into a wider application outage or resource-consumption problem.

Common timeout categories include:

• Connection timeout
• Request timeout
• Response timeout
• Idle timeout
• Keepalive timeout

Excessive retry behavior can create retry storms and put even more pressure on unhealthy systems.

Timeout and retry policies should therefore reflect the actual behavior of the application.

Load Balancer vs CDN: What Is the Difference?

A load balancer distributes traffic across backend compute resources. A Content Delivery Network - CDN - focuses on delivering content from distributed edge locations closer to users and reducing origin demand.

CDN and load balancing are commonly used together.

A request can first reach a CDN edge location. If the request cannot be served from cache, dynamic traffic can continue to the origin infrastructure, where a load balancer distributes it across backend systems.

For the edge-distribution architecture, see What Is a CDN?.

Load Balancer vs WAF: What Is the Difference?

A load balancer decides where allowed traffic should go. A Web Application Firewall - WAF - analyzes application traffic against security policies and can block malicious requests.

A WAF can help address risks such as:

• SQL injection
• Cross-site scripting
• Malicious request patterns
• Bot and rate-limit policies

WAF and load-balancing capabilities may be delivered by the same platform, but their logical roles are different.

Is a Load Balancer the Same as a Firewall?

No. A firewall determines whether traffic should be allowed according to security policy. A load balancer determines which backend should receive allowed traffic.

Load Balancer vs API Gateway

A load balancer primarily focuses on traffic distribution, availability, and backend selection, while an API Gateway adds API-specific policy and lifecycle capabilities.

API Gateway capabilities may include:

• Authentication
• API key management
• Rate limiting
• Quotas
• Request transformation
• API versioning

An architecture may use an API Gateway and one or more load balancers together.

Does a Load Balancer Protect Against DDoS Attacks?

A load balancer can distribute increased traffic across available capacity, but it should not be treated as a complete DDoS-protection solution.

Large attacks may require:

• Network-level mitigation
• Anycast scrubbing
• Rate limiting
• CDN
• WAF
• Dedicated DDoS mitigation

For the broader defense architecture, see What Are DDoS Attacks?.

Why Is Load Balancer Security Important?

A load balancer can become an internet-facing entry point for critical application traffic.

Security design should therefore include:

• Restricted management-interface access
• Secure TLS configuration
• Certificate lifecycle management
• Software patching
• Administrative access control
• Audit logging
• Strong administrator authentication

Hardware vs Software vs Managed Cloud Load Balancers

A managed cloud load balancer should not automatically be assumed to scale infinitely or instantly.

Managed services may still have:

• Service quotas
• Scaling behavior
• Connection limits
• Regional dependencies

How Does Load Balancing Work in Kubernetes?

Kubernetes uses Service resources to provide stable network access to changing groups of Pods, while external load balancers can be integrated through supported infrastructure and Service type LoadBalancer.

Kubernetes Service types include:

• ClusterIP
• NodePort
• LoadBalancer

HTTP and application-routing layers may additionally use:

• Ingress
• Gateway API

For the wider orchestration and networking model, see What Is Kubernetes?.

How Is Load Balancing Used in Microservices?

Microservice architectures commonly need traffic distribution at two different levels.

North-South Traffic

Traffic entering the application environment from users or external systems.

East-West Traffic

Communication between internal microservices.

East-west traffic may use:

• Service discovery
• Client-side load balancing
• Service mesh

depending on the architecture.

Client-Side vs Server-Side Load Balancing

Server-Side Load Balancing

The client connects to one load-balancer endpoint and the load balancer chooses the backend.

Client-Side Load Balancing

The application or client library knows the available backend endpoints and selects one directly.

Client-side approaches can appear in microservice, service-discovery, and service-mesh architectures.

Which Load Balancer Metrics Should Be Monitored?

Load balancer observability should include traffic, backend health, latency, errors, connections, and resource saturation rather than only up/down status.

Useful KPIs include:

• Requests per second
• Connections per second
• Active connections
• Healthy backend count
• Unhealthy backend count
• P50 response time
• P95 response time
• P99 response time
• 4xx error rate
• 5xx error rate
• Connection error rate
• TLS handshake errors
• Bytes in and out
• Backend saturation

How Do You Size Load Balancer Capacity?

Request count alone is not enough for capacity planning.

Teams should evaluate:

• Peak requests per second
• Concurrent connections
• Average connection duration
• New connections per second
• Payload size
• TLS handshake volume
• Backend response time
• WebSocket and other long-lived connections
• Expected traffic growth

Can a Load Balancer Handle WebSocket Traffic?

Yes.

However, WebSocket connections can remain open for long periods, which makes the following settings important:

• Idle timeout
• Connection limits
• Health checks
• Session behavior
• Graceful shutdown

Request-per-second metrics alone may not represent load accurately for long-lived connection workloads.

Which Questions Should You Ask When Designing a Load Balancer?

• Is the traffic TCP, UDP, HTTP, HTTPS, or gRPC?
• Do we need Layer 4 or Layer 7 routing?
• What are peak request and connection volumes?
• Do backend systems have equal capacity?
• What should the health check actually validate?
• Is session persistence required?
• Can the application become stateless?
• Where should TLS terminate?
• Should backend traffic be re-encrypted?
• Does the backend need the original client IP?
• What should the connection-draining period be?
• Is multi-zone redundancy required?
• Is multi-region or DR failover required?
• Where do WAF and DDoS protection sit in the architecture?
• How will the load-balancing layer be monitored?

Common Load Balancer Design Mistakes

1. Using Only One Load Balancer

Redundant backends do not provide complete high availability if all traffic depends on one load balancer.

2. Checking Only Whether a Port Is Open

A port can be reachable while the application behind it is failing.

3. Using Overly Aggressive Health Checks

Temporary latency can cause unnecessary backend removal and failover.

4. Depending Too Heavily on Sticky Sessions

Session affinity can create uneven load and still cannot guarantee that one backend remains available forever.

5. Ignoring Connection Draining

Active user requests can be interrupted during maintenance and deployments.

6. Ignoring TLS Capacity

Large volumes of TLS handshakes can create meaningful CPU and connection overhead.

7. Assuming All Backends Have Equal Capacity

Different hardware or workload profiles may require weighted algorithms.

8. Monitoring Only Uptime

A backend can technically remain up while latency and error rates become unacceptable.

9. Treating Load Balancing as DDoS Protection

Large volumetric attacks may overwhelm upstream capacity before backend load distribution becomes relevant.

10. Forgetting Traffic Redirection in Disaster Recovery

Recovery infrastructure is not useful if users and integrations cannot reach it.

How Do Load Balancing, CDN, and Disaster Recovery Work Together?

A large web architecture may use a traffic path such as:

1. The client resolves the application endpoint through DNS.
2. Traffic reaches a CDN or edge layer.
3. Cacheable content can be served from the edge.
4. Dynamic requests continue to the origin infrastructure.
5. A load balancer selects a healthy backend.
6. If the primary environment fails, global traffic management may redirect users to a recovery region.

CDN, load balancing, and disaster recovery therefore solve different problems at different layers of the same service architecture.

Why Does a Carrier-Neutral Data Center Matter for Load-Balanced Applications?

Load balancing distributes traffic between application backends, but it does not solve the quality or resilience of the network path between users and the data center.

A carrier-neutral data center can support:

• Multiple ISP options
• Route diversity
• Reduced dependency on one carrier
• Alternative network paths

This provides an additional resilience layer around the application architecture.

For the connectivity perspective, see What Is a Carrier-Neutral Data Center?.

How Is Load Balancing Used in Hybrid Cloud?

In hybrid infrastructure, application backends may run across:

• Private cloud
• Public cloud
• On-premises infrastructure
• Colocation

Traffic management then becomes more than a local load-balancing decision.

Network latency, route quality, cloud connectivity, security boundaries, and data location also matter.

For controlled cloud connectivity, see Direct Cloud Access with DE-CIX.

High Availability and Traffic Management with Ixpanse

A high-availability architecture requires more than a load balancer.

Compute, network, storage, connectivity, monitoring, and disaster recovery need to operate as one architecture.

Ixpanse's Private Cloud provides dedicated and scalable cloud infrastructure for organizations that require greater control over modern application environments.

For organizations operating physical infrastructure, Ixpanse's Colocation service provides carrier-neutral connectivity, multiple network options, and power availability of up to 20KW+ per rack.

At the network layer, Ankara IX provides Direct Internet Access, Cloud Interconnect, and managed point-to-point connectivity.

Continuous infrastructure operations can be supported through Managed Services, including 24/7 monitoring, virtual-server management, replication management, and cloud-operations support.

When applications also need protection against site or regional outages, load-balancing and traffic-routing architecture should be designed together with DRaaS and disaster-recovery failover.

To evaluate high availability, private cloud, connectivity, and traffic-management architecture for your applications, contact the Ixpanse expert team.

Conclusion

A load balancer is the traffic-management layer that distributes connections or requests across multiple healthy backend resources and forms a key part of modern high-availability architecture.

• A load balancer does more than divide traffic equally.
• Backend health can influence traffic decisions.
• Not every load balancer is a reverse proxy.
• Round Robin, Least Connections, weighted, and hash-based algorithms suit different workloads.
• Health-check quality directly affects failover behavior.
• Connection draining protects active requests during maintenance and deployment.
• Sticky sessions may be required, but stateless application design is generally easier to scale.
• Layer 4 and Layer 7 load balancers make decisions using different traffic information.
• TLS termination, passthrough, and re-encryption create different security models.
• The load-balancing layer itself should also be redundant.
• GSLB can route traffic across regions and data centers.
• A load balancer is not a substitute for DDoS protection.
• Kubernetes and microservice environments can use load balancing at multiple layers.
• Disaster recovery also requires traffic to be redirected to the recovery environment.

The right question when designing a load-balancing architecture is not only:

"How many servers can it distribute traffic across?"

A more useful question is:

"How reliably can the architecture manage traffic when a backend fails, demand increases, an application is deployed, a network path degrades, or an entire data center becomes unavailable?"

Frequently Asked Questions About Load Balancers

What is a load balancer?

A load balancer is a traffic-management component that directs incoming network or application traffic across multiple healthy backend servers or application instances.

How does a load balancer work?

A load balancer accepts a connection or request, evaluates backend health, selects a target according to a load-balancing algorithm, and forwards the traffic to that backend.

Is a load balancer the same as a reverse proxy?

No. Many Layer 7 load balancers operate as reverse proxies, but Layer 4 forwarding, NAT, direct-routing, and other load-balancing architectures also exist.

What is Round Robin load balancing?

Round Robin distributes new requests sequentially across available backend servers.

What is Least Connections load balancing?

Least Connections directs a new connection to a backend that currently has fewer active connections.

What is a load balancer health check?

A health check determines whether a backend is currently suitable to receive new traffic.

What is connection draining?

Connection draining stops new traffic from reaching a backend being removed from service while allowing existing connections to complete for a defined period.

What is a sticky session?

A sticky session or session-persistence policy attempts to keep subsequent requests from the same client or session on the same backend.

What is a Layer 4 load balancer?

A Layer 4 load balancer routes TCP or UDP traffic using transport-layer information such as IP addresses, ports, and protocols.

What is a Layer 7 load balancer?

A Layer 7 load balancer can use application information such as HTTP hostnames, paths, headers, and cookies to make routing decisions.

What is an internal load balancer?

An internal load balancer distributes traffic between services that are reachable only within a private network.

What is GSLB?

GSLB - Global Server Load Balancing - manages traffic across services in different data centers or regions using health, geography, latency, or routing policy.

What is TLS termination?

TLS termination means the encrypted client connection ends at the load balancer, which then establishes a separate backend connection.

What is TLS passthrough?

TLS passthrough forwards encrypted traffic to the backend without decrypting it at the load balancer.

Does a load balancer protect against DDoS attacks?

A load balancer can distribute increased traffic across backend capacity, but comprehensive DDoS defense usually requires additional network, CDN, WAF, rate-limiting, and mitigation layers.

What is the difference between a load balancer and a CDN?

A load balancer distributes traffic across backend resources, while a CDN focuses on delivering content from distributed edge locations and reducing origin demand.

Is a load balancer the same as a firewall?

No. A firewall decides whether traffic is allowed according to security policy. A load balancer decides which backend should receive allowed traffic.

What is the difference between a load balancer and an API Gateway?

A load balancer primarily handles traffic distribution and availability. An API Gateway focuses more on API authentication, rate limiting, quotas, transformation, and lifecycle policies.

Does a load balancer provide high availability?

A load balancer can improve high availability by failing traffic away from unhealthy backends, but the load-balancing layer itself must also be resilient.

Does Kubernetes use load balancers?

Yes. Kubernetes Services can distribute traffic to Pods, and supported infrastructure can integrate external load balancers through Service type LoadBalancer.

Does a small website need a load balancer?

Not always. If one server provides enough capacity and downtime is acceptable, the additional complexity may not be justified. Load balancing becomes more valuable as availability and growth requirements increase.

How many servers do you need before using a load balancer?

There is no fixed threshold. Even two backend servers can justify a load balancer when the application has low tolerance for downtime.

How does Ixpanse support load-balanced architectures?

Ixpanse supports high-availability application architectures through Private Cloud, Colocation, Ankara IX, Managed Services, connectivity, monitoring, and DRaaS infrastructure layers.

Related Content

• What Is a CDN?
• What Is DRaaS?
• What Are DDoS Attacks?
• What Is Kubernetes?
• What Is a Carrier-Neutral Data Center?
• Direct Cloud Access with DE-CIX
• Private Cloud
• Colocation
• Ankara IX
• Managed Services

Technical Resources

• NGINX - HTTP Load Balancing 
• HAProxy - Health Checks 
• HAProxy - Session Persistence 
• AWS - Load Balancer Health Checks