Hackforge Academy

Explanation

Distributed logging is the practice of collecting logs from multiple microservices so that application activity can be monitored across the entire system. In a microservices architecture, one user request may travel through an API Gateway, User Service, Order Service, and Payment Service. Each service produces its own logs. Centralized logging makes it easier to search, analyze, and troubleshoot these logs from one location. Python's built-in logging module can be used to generate structured application logs.

Code:

import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

logger = logging.getLogger('order-service')

logger.info('Order request received')
logger.info('Validating order')
logger.warning('Payment service response is slow')
logger.error('Payment service failed')

Explanation

A request ID is a unique identifier assigned to a request as it travels through multiple services. Including the same request ID in every service log makes it possible to find all log messages related to one request. This is especially useful when debugging distributed applications. The API Gateway can generate the request ID and pass it to downstream services through an HTTP header such as X-Request-ID.

Code:

import uuid
import logging
from fastapi import FastAPI, Request

app = FastAPI()

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger('order-service')


@app.get('/orders')
async def get_orders(request: Request):
    request_id = request.headers.get(
        'X-Request-ID',
        str(uuid.uuid4())
    )

    logger.info(
        'Request received: %s',
        request_id
    )

    return {
        'request_id': request_id,
        'orders': []
    }

Explanation

Distributed tracing tracks the complete journey of a request across multiple services. A trace represents the entire request, while spans represent individual operations performed by services. For example, a request may create a trace containing an API Gateway span, Order Service span, Product Service span, and Payment Service span. Tracing helps developers identify slow services, failed operations, and performance bottlenecks that are difficult to understand from individual service logs.

Code:

# Example distributed trace
#
# Trace ID: abc123
#
# API Gateway
# |------ 20 ms
# |
# +-- Order Service
#     |------ 80 ms
#     |
#     +-- Product Service
#     |     |------ 30 ms
#     |
#     +-- Payment Service
#           |------ 500 ms
#
# Total request time: ~630 ms
#
# The trace shows that Payment Service
# is the slowest operation.

Explanation

OpenTelemetry is an open-source observability framework used to collect traces, metrics, and logs. It can instrument FastAPI applications and export telemetry data to compatible observability platforms. OpenTelemetry helps standardize distributed tracing across multiple Python microservices. A trace can then be viewed in systems such as Jaeger, Zipkin, Grafana Tempo, or other OpenTelemetry-compatible backends.

Code:

# Install packages
# pip install opentelemetry-api
# pip install opentelemetry-sdk
# pip install opentelemetry-instrumentation-fastapi
# pip install opentelemetry-exporter-otlp

from fastapi import FastAPI
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor

app = FastAPI()

FastAPIInstrumentor.instrument_app(app)


@app.get('/orders')
def get_orders():
    return {
        'orders': []
    }


# OpenTelemetry automatically creates
# tracing information for FastAPI requests.

Explanation

In a production microservices architecture, logs and traces are collected from all services and sent to centralized observability systems. Logs provide detailed application events, while traces show how a request moves through the system. A common architecture uses structured Python logging, OpenTelemetry for tracing, and centralized platforms such as Elasticsearch, Loki, Jaeger, Grafana, or cloud observability services. Combining logs and traces allows developers to search for a trace ID and quickly identify which service caused a failure or performance problem.

Code:

# Distributed Observability Architecture
#
#                 Client
#                    |
#                    v
#              API Gateway
#                    |
#          +---------+---------+
#          |         |         |
#          v         v         v
#       User      Order     Payment
#      Service   Service    Service
#          |         |         |
#          +---------+---------+
#                    |
#                    v
#             OpenTelemetry
#                    |
#          +---------+---------+
#          |                   |
#          v                   v
#       Logs Store        Trace Backend
#          |                   |
#          v                   v
#      Grafana/Loki        Jaeger/Tempo
#
# Key concepts:
# - Log Level
# - Request ID
# - Trace ID
# - Span ID
# - Centralized Logging
# - Distributed Tracing
# - Observability