Category: python
What are Timeout and Retry in Microservices?
Published on 22 Aug 2026
Explanation
Timeout and retry are important resilience patterns in microservices. A timeout prevents a service from waiting indefinitely when another service is slow or unavailable. A retry allows a failed request to be attempted again when the failure may be temporary, such as a network interruption. These patterns are commonly used when services communicate through REST APIs. Retries should be used carefully because repeatedly calling an unhealthy service can increase traffic and make the failure worse.
Code:
import httpx
async def get_product():
async with httpx.AsyncClient() as client:
response = await client.get(
'http://localhost:8001/products/101',
timeout=5.0
)
return response.json()
Explanation
A timeout defines how long a service should wait for another service to respond. HTTPX allows developers to configure request timeouts using the timeout parameter. If the remote service does not respond within the configured time, HTTPX raises a timeout-related exception. Handling the timeout allows the calling service to return a meaningful response instead of remaining blocked indefinitely.
Code:
import httpx
from fastapi import FastAPI, HTTPException
app = FastAPI()
@app.get('/products/{product_id}')
async def get_product(product_id: int):
try:
async with httpx.AsyncClient() as client:
response = await client.get(
f'http://localhost:8001/products/{product_id}',
timeout=3.0
)
return response.json()
except httpx.TimeoutException:
raise HTTPException(
status_code=504,
detail='Product Service timed out'
)
Explanation
Retry logic allows a failed request to be attempted again. A retry is useful when failures are temporary, such as connection resets or short network interruptions. The number of retries should be limited, and a delay should normally be added between attempts. In this example, the request is attempted up to three times. If all attempts fail, the service returns an error instead of continuing indefinitely.
Code:
import asyncio
import httpx
async def get_product():
max_retries = 3
async with httpx.AsyncClient() as client:
for attempt in range(max_retries):
try:
response = await client.get(
'http://localhost:8001/products/101',
timeout=3.0
)
response.raise_for_status()
return response.json()
except (httpx.TimeoutException, httpx.RequestError):
print(f'Attempt {attempt + 1} failed')
if attempt < max_retries - 1:
await asyncio.sleep(1)
raise Exception('Product Service unavailable')
Explanation
Exponential backoff increases the delay between retry attempts. Instead of immediately sending another request after a failure, the client waits progressively longer. For example, the delays can be 1 second, 2 seconds, 4 seconds, and 8 seconds. This reduces pressure on an unhealthy service and gives it time to recover. Production systems commonly combine exponential backoff with a maximum retry count and optional jitter to prevent many clients from retrying at exactly the same time.
Code:
import asyncio
import httpx
async def get_product():
max_retries = 4
async with httpx.AsyncClient() as client:
for attempt in range(max_retries):
try:
response = await client.get(
'http://localhost:8001/products/101',
timeout=3.0
)
response.raise_for_status()
return response.json()
except (httpx.TimeoutException, httpx.RequestError):
if attempt == max_retries - 1:
break
delay = 2 ** attempt
print(f'Retrying in {delay} seconds...')
await asyncio.sleep(delay)
raise Exception('All retry attempts failed')
Explanation
A production microservice should combine timeouts, limited retries, error handling, and meaningful HTTP responses. A timeout protects the service from waiting too long, while retries can recover from temporary failures. However, not every error should be retried. For example, retrying a 400 Bad Request usually does not help because the request itself is invalid. Retry logic is more appropriate for temporary network failures and selected server-side failures. For larger systems, these patterns can be implemented using resilience libraries, API gateways, or service-mesh infrastructure.
Code:
import asyncio
import httpx
from fastapi import FastAPI, HTTPException
app = FastAPI()
async def call_product_service(product_id: int):
max_retries = 3
async with httpx.AsyncClient() as client:
for attempt in range(max_retries):
try:
response = await client.get(
f'http://localhost:8001/products/{product_id}',
timeout=3.0
)
# Do not retry client errors such as 400 or 404
if 400 <= response.status_code < 500:
return response
response.raise_for_status()
return response
except (httpx.TimeoutException, httpx.RequestError):
if attempt == max_retries - 1:
raise
delay = 2 ** attempt
await asyncio.sleep(delay)
@app.get('/orders/product/{product_id}')
async def get_product(product_id: int):
try:
response = await call_product_service(product_id)
return response.json()
except httpx.TimeoutException:
raise HTTPException(
status_code=504,
detail='Product Service timeout'
)
except httpx.RequestError:
raise HTTPException(
status_code=503,
detail='Product Service unavailable'
)