OpenAI SDK timeouts, retries and request IDs
A timeout can happen in the SDK, a proxy or the upstream service. Capture the layer first, then change one policy at a time.
Separate three failures before changing a timeout
A client timeout, an HTTP 504 and a structured API error are different failures. Record what the client received before changing retry settings.
| Observed result | What it proves | First check |
|---|---|---|
Python APITimeoutError or Node timeout error, no HTTP status | The SDK stopped waiting before it received a complete HTTP response | SDK timeout, network path and whether an automatic retry ran |
| HTTP 504 | A gateway or proxy did not receive a timely upstream response | Response headers, body, request ID and gateway logs |
| HTTP 401 or 429 with JSON | The service returned an application response | Authentication or rate-limit details; a longer timeout will not fix either |
RFC 9110 defines 504 at the gateway layer. The OpenAI Python SDK documents APITimeoutError as a client exception. They can happen during the same incident, but they are not interchangeable labels.
Verify the route with a cheap request
Call the model list before testing a long generation. It gives you a short request that separates basic routing and authentication from generation latency.
curl -sS -D /tmp/monkey-headers.txt \ https://monkeytheme.com/v1/models \ -H "Authorization: Bearer ***" \ -o /tmp/monkey-models.json sed -n '1,20p' /tmp/monkey-headers.txt
On August 3, 2026, an unauthenticated production probe returned structured HTTP 401 from this route. That confirms the public path reaches the authentication layer. It does not prove that a specific key or model is healthy.
Set Python timeouts without hiding retries
The official Python SDK currently documents a ten-minute default timeout and two automatic retries for connection errors, 408, 409, 429 and responses at or above 500. A timed-out request is retried by default. During diagnosis, disable retries so one test means one SDK attempt.
import os
import httpx
import openai
from openai import OpenAI
client = OpenAI(
api_key=os.environ["MONKEYAI_API_KEY"],
base_url="https://monkeytheme.com/v1",
max_retries=0,
timeout=httpx.Timeout(60.0, connect=5.0, read=45.0, write=10.0),
)
try:
models = client.models.list()
except openai.APITimeoutError:
print("SDK timeout: no completed HTTP response")
except openai.APIStatusError as exc:
print("HTTP status:", exc.status_code)
print("Request ID:", exc.request_id)Use client.with_options(timeout=..., max_retries=...) when one operation needs a different policy. Model discovery and a long streaming request usually should not share the same read timeout.
Configure Node.js in milliseconds
The official Node SDK also documents a ten-minute default and two automatic retries for the same broad error classes. Its timeout value is in milliseconds.
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.MONKEYAI_API_KEY,
baseURL: "https://monkeytheme.com/v1",
maxRetries: 0,
timeout: 60_000,
});
try {
await client.models.list();
} catch (err) {
console.error(err.status ?? "no HTTP status", err.request_id ?? "no request id");
}Do not copy Python's seconds directly into Node's millisecond field. That quiet unit mismatch can turn a planned 60-second test into 60 milliseconds.
Log evidence without logging secrets
Keep the timestamp, endpoint family, model ID, elapsed time, HTTP status and request ID. Do not log the bearer token or full request body. Both official SDKs expose _request_id on successful object responses when the server sends x-request-id; failed status responses expose a request ID on the error when available.
Retry only when the operation is safe
The SDK defaults are useful for transient faults, but retries repeat requests. For a read-only model-list call that is usually straightforward. For a write or any operation with an external side effect, check the endpoint's idempotency behavior before retrying. This guide does not claim that every OpenAI-compatible endpoint is idempotent.
For 401, billing-limit errors or a wrong model ID, fix the cause instead of adding retries. For 429, follow Retry-After when the service provides it. For 500, 503 or 504, use a bounded backoff and keep the request evidence.
Sources and related guides
FAQ
Is an SDK timeout the same as HTTP 504?
No. An SDK timeout means the client stopped waiting without receiving a complete response. HTTP 504 is a response returned by a gateway or proxy.
Do the official OpenAI SDKs retry timeouts?
Their current READMEs document two automatic retries by default for connection errors and selected HTTP statuses. Timed-out requests are retried by default.
What timeout unit does the Node SDK use?
The Node SDK timeout is configured in milliseconds. The Python SDK accepts seconds as a float or an httpx.Timeout object.
Will a longer timeout fix 401 or 429?
No. A 401 is an authentication response. A 429 describes a rate, credit or limit condition; inspect the error code and Retry-After header instead.
What should I send to support?
Send the timestamp, endpoint family, model ID, elapsed time, HTTP status, safe response details and request ID when present. Never send the API key in logs or screenshots.