Hackforge Academy

Explanation

Load balancing distributes incoming requests across multiple instances of a service. In a microservices architecture, a single service may run on multiple servers or containers to handle more traffic and improve availability. Instead of sending every request to one instance, a load balancer distributes requests across available instances. Common load balancing strategies include round robin, least connections, weighted routing, and health-based routing. Load balancing is especially useful when a service receives high traffic or needs high availability.

Code:

# Example architecture
#
#                 Client
#                   |
#                   v
#             Load Balancer
#              /    |    \
#             /     |     \
#            v      v      v
#       Service-1 Service-2 Service-3
#        :8001     :8002     :8003

Explanation

A FastAPI microservice can run multiple instances on different ports. Each instance provides the same API and functionality. A load balancer can then distribute incoming requests across these instances. This approach allows the application to handle more concurrent requests and continue operating if one instance becomes unavailable, provided the load balancer performs health checks.

Code:

# product_service.py

from fastapi import FastAPI

app = FastAPI()


@app.get('/products')
def get_products():
    return {
        'service': 'Product Service',
        'message': 'Response from Product Service'
    }


# Start multiple instances:
# uvicorn product_service:app --port 8001
# uvicorn product_service:app --port 8002
# uvicorn product_service:app --port 8003

Explanation

Round robin is one of the simplest load balancing strategies. Requests are distributed sequentially across available service instances. For example, the first request can go to Service 1, the second to Service 2, the third to Service 3, and the fourth back to Service 1. This works well when all service instances have similar capacity and requests have roughly similar processing requirements.

Code:

import itertools

services = [
    'http://localhost:8001',
    'http://localhost:8002',
    'http://localhost:8003'
]

service_cycle = itertools.cycle(services)


for request_number in range(1, 10):
    service = next(service_cycle)
    print(
        f'Request {request_number} -> {service}'
    )

# Output pattern:
# Request 1 -> 8001
# Request 2 -> 8002
# Request 3 -> 8003
# Request 4 -> 8001

Explanation

Nginx can act as a reverse proxy and load balancer in front of multiple FastAPI instances. The upstream section defines the available backend servers, and proxy_pass forwards incoming requests to them. Nginx can distribute requests using strategies such as round robin, least connections, or IP hash. This approach is commonly used when deploying FastAPI microservices because Nginx can also handle TLS termination, static files, connection management, and reverse proxying.

Code:

# nginx.conf

upstream product_service {
    server 127.0.0.1:8001;
    server 127.0.0.1:8002;
    server 127.0.0.1:8003;
}

server {
    listen 80;

    location /products/ {
        proxy_pass http://product_service;
    }
}

# FastAPI instances:
# uvicorn product_service:app --port 8001
# uvicorn product_service:app --port 8002
# uvicorn product_service:app --port 8003

Explanation

A production load balancer should know whether service instances are healthy before sending traffic to them. FastAPI can expose a simple health endpoint such as /health. The load balancer can periodically call this endpoint and remove unhealthy instances from the request pool. When the service becomes healthy again, it can be added back. Health checks, multiple instances, and load balancing together improve availability and resilience in a microservices architecture.

Code:

from fastapi import FastAPI

app = FastAPI()


@app.get('/health')
def health_check():
    return {
        'status': 'healthy'
    }


@app.get('/products')
def get_products():
    return {
        'products': []
    }


# Health check URL:
# GET http://localhost:8001/health
# GET http://localhost:8002/health
# GET http://localhost:8003/health
#
# Load balancer sends traffic only to healthy instances.