Mock LLM API Client with Error Handling

easyPython

Lesson

Robust API Error Handling and Retry Logic

When building applications that depend on external APIs, you'll inevitably encounter various types of failures: network timeouts, rate limiting, server errors, and malformed responses. Robust error handling and retry logic are essential patterns for creating reliable applications.

Common API Failure Scenarios

Rate Limiting (HTTP 429): APIs often limit how many requests you can make per minute or hour. When you exceed this limit, the server responds with a 429 status code. The proper response is to wait and retry, not to immediately fail.

Network Timeouts: Network requests can hang indefinitely due to connectivity issues. Setting timeouts prevents your application from freezing, but you might want to retry if the timeout was temporary.

Malformed Responses: Even successful HTTP responses (200 status) might contain invalid JSON or unexpected data formats that break your parsing logic.

Exponential Backoff Strategy

When retrying failed requests, it's important not to immediately retry at full speed. Exponential backoff gradually increases wait times between retries: first wait 1 second, then 2 seconds, then 4 seconds, etc. This gives temporary issues time to resolve and prevents overwhelming an already struggling server.

Graceful Error Handling

Effective error handling distinguishes between recoverable and non-recoverable errors. Rate limits and timeouts are often temporary and worth retrying. Malformed responses or authentication errors usually aren't worth retrying and should be reported immediately.

The key is to make your error handling predictable - callers should know what exceptions to expect and when retry logic has been exhausted.

Example
1import requests 2import time 3 4def fetch_weather_data(city, max_retries=3): 5 url = f"https://api.weather.com/data/{city}" 6 7 for attempt in range(max_retries + 1): 8 try: 9 response = requests.get(url, timeout=10) 10 11 if response.status_code == 200: 12 return response.json() 13 elif response.status_code == 429: 14 if attempt == max_retries: 15 raise Exception("Rate limited - max retries exceeded") 16 17 # Exponential backoff 18 wait_time = 2 ** attempt 19 time.sleep(wait_time) 20 continue 21 else: 22 response.raise_for_status() 23 24 except requests.exceptions.Timeout: 25 if attempt == max_retries: 26 raise 27 time.sleep(2 ** attempt) 28 29 return None
L6Loop through retry attempts, including the initial attempt (attempt 0)
L13Check if we've exhausted retries before waiting and retrying
L16Exponential backoff: wait 1s, then 2s, then 4s, etc.

Key Takeaways

  • •Use exponential backoff to gradually increase wait times between retries
  • •Distinguish between temporary errors (retry) and permanent errors (fail fast)
  • •Always set request timeouts to prevent your application from hanging indefinitely
Loading...