When building applications that depend on external APIs, you'll inevitably encounter various types of failures: network timeouts, rate limiting, server errors, and malformed responses. Robust error handling and retry logic are essential patterns for creating reliable applications.
Rate Limiting (HTTP 429): APIs often limit how many requests you can make per minute or hour. When you exceed this limit, the server responds with a 429 status code. The proper response is to wait and retry, not to immediately fail.
Network Timeouts: Network requests can hang indefinitely due to connectivity issues. Setting timeouts prevents your application from freezing, but you might want to retry if the timeout was temporary.
Malformed Responses: Even successful HTTP responses (200 status) might contain invalid JSON or unexpected data formats that break your parsing logic.
When retrying failed requests, it's important not to immediately retry at full speed. Exponential backoff gradually increases wait times between retries: first wait 1 second, then 2 seconds, then 4 seconds, etc. This gives temporary issues time to resolve and prevents overwhelming an already struggling server.
Effective error handling distinguishes between recoverable and non-recoverable errors. Rate limits and timeouts are often temporary and worth retrying. Malformed responses or authentication errors usually aren't worth retrying and should be reported immediately.
The key is to make your error handling predictable - callers should know what exceptions to expect and when retry logic has been exhausted.
1import requests
2import time
3
4def fetch_weather_data(city, max_retries=3):
5 url = f"https://api.weather.com/data/{city}"
6
7 for attempt in range(max_retries + 1):
8 try:
9 response = requests.get(url, timeout=10)
10
11 if response.status_code == 200:
12 return response.json()
13 elif response.status_code == 429:
14 if attempt == max_retries:
15 raise Exception("Rate limited - max retries exceeded")
16
17 # Exponential backoff
18 wait_time = 2 ** attempt
19 time.sleep(wait_time)
20 continue
21 else:
22 response.raise_for_status()
23
24 except requests.exceptions.Timeout:
25 if attempt == max_retries:
26 raise
27 time.sleep(2 ** attempt)
28
29 return None