OpenResponses Configuration with AI Gateway
The OpenResponses API lets you configure AI Gateway behavior using providerOptions. The gateway namespace gives you control over provider routing, fallbacks, and restrictions.
Set up automatic fallbacks so if your primary model is unavailable, requests route to backup models in order. Use the models array to specify the fallback chain.
const apiKey = process.env.AI_GATEWAY_API_KEY;
const response = await fetch('https://ai-gateway.vercel.sh/v1/responses', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: 'anthropic/claude-opus-5',
input: [
{
type: 'message',
role: 'user',
content: 'Tell me a fun fact about octopuses.',
},
],
providerOptions: {
gateway: {
models: ['openai/gpt-6-astra', 'google/gemini-3.1-pro-preview'],
},
},
}),
});
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());import json
import os
import urllib.request
request = urllib.request.Request(
'https://ai-gateway.vercel.sh/v1/responses',
data=json.dumps({'model': 'anthropic/claude-opus-5', 'input': [{'type': 'message', 'role': 'user', 'content': 'Tell me a fun fact about octopuses.'}], 'providerOptions': {'gateway': {'models': ['openai/gpt-6-astra', 'google/gemini-3.1-pro-preview']}}}).encode(),
headers={'Content-Type': 'application/json', 'Authorization': "Bearer " + os.environ["AI_GATEWAY_API_KEY"]},
)
with urllib.request.urlopen(request) as response:
print(json.load(response))curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-opus-5",
"input": [
{
"type": "message",
"role": "user",
"content": "Tell me a fun fact about octopuses."
}
],
"providerOptions": {
"gateway": {
"models": [
"openai/gpt-6-astra",
"google/gemini-3.1-pro-preview"
]
}
}
}'Control the order in which providers are tried using the order array. AI Gateway will attempt providers in the specified order until one succeeds.
const apiKey = process.env.AI_GATEWAY_API_KEY;
const response = await fetch('https://ai-gateway.vercel.sh/v1/responses', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: 'google/gemini-3.1-pro-preview',
input: [
{
type: 'message',
role: 'user',
content: 'Explain quantum computing in one sentence.',
},
],
providerOptions: {
gateway: {
order: ['google', 'vertex'],
},
},
}),
});
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());import json
import os
import urllib.request
request = urllib.request.Request(
'https://ai-gateway.vercel.sh/v1/responses',
data=json.dumps({'model': 'google/gemini-3.1-pro-preview', 'input': [{'type': 'message', 'role': 'user', 'content': 'Explain quantum computing in one sentence.'}], 'providerOptions': {'gateway': {'order': ['google', 'vertex']}}}).encode(),
headers={'Content-Type': 'application/json', 'Authorization': "Bearer " + os.environ["AI_GATEWAY_API_KEY"]},
)
with urllib.request.urlopen(request) as response:
print(json.load(response))curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "google/gemini-3.1-pro-preview",
"input": [
{
"type": "message",
"role": "user",
"content": "Explain quantum computing in one sentence."
}
],
"providerOptions": {
"gateway": {
"order": [
"google",
"vertex"
]
}
}
}'Restrict requests to specific providers using the only array, which is useful when a compliance requirement limits which providers can handle your requests. only removes failover to unlisted providers: if none of the listed providers can serve the request, it fails rather than routing elsewhere. To optimize for cost, use sort: 'cost' instead, which ranks providers by price while keeping every provider eligible for failover. For data-handling requirements, zero data retention and disallow prompt training restrict routing to providers that meet those policies without naming providers individually.
const apiKey = process.env.AI_GATEWAY_API_KEY;
const response = await fetch('https://ai-gateway.vercel.sh/v1/responses', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: 'anthropic/claude-sonnet-5',
input: [
{
type: 'message',
role: 'user',
content: 'What makes a great cup of coffee?',
},
],
providerOptions: {
gateway: {
only: ['anthropic', 'bedrock'],
},
},
}),
});
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());import json
import os
import urllib.request
request = urllib.request.Request(
'https://ai-gateway.vercel.sh/v1/responses',
data=json.dumps({'model': 'anthropic/claude-sonnet-5', 'input': [{'type': 'message', 'role': 'user', 'content': 'What makes a great cup of coffee?'}], 'providerOptions': {'gateway': {'only': ['anthropic', 'bedrock']}}}).encode(),
headers={'Content-Type': 'application/json', 'Authorization': "Bearer " + os.environ["AI_GATEWAY_API_KEY"]},
)
with urllib.request.urlopen(request) as response:
print(json.load(response))curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-5",
"input": [
{
"type": "message",
"role": "user",
"content": "What makes a great cup of coffee?"
}
],
"providerOptions": {
"gateway": {
"only": [
"anthropic",
"bedrock"
]
}
}
}'You can set per-provider timeouts for BYOK credentials to trigger fast failover when a provider is slow to respond. Pass providerTimeouts in providerOptions.gateway:
"providerOptions": {
"gateway": {
"providerTimeouts": {
"byok": { "anthropic": 3000, "bedrock": 5000 }
}
}
}For full details, limits, and response metadata, see Provider Timeouts.
Use caching: 'auto' in the request body to let AI Gateway automatically add cache markers for providers that require them (like Anthropic). For full details, supported providers, and examples, see Automatic Caching.
Was this helpful?