AI Gateway Regional Inference

AI Gateway lets you choose the region where your model's provider runs inference, where the provider supports it. This may help with your data residency, compliance, and latency requirements. You can pin one of these regions, or leave inference on the default:

• 🇺🇸 United States (us): available models
• 🇪🇺 European Union (eu): available models
• 🇦🇺 Australia (au): available models
• 🇯🇵 Japan (jp): available models
• 🌐 Any region (global, the default): AI Gateway picks whichever region can serve the model

AI Gateway routes your data to the provider in a data center in the region you choose, and to the extent the provider stores data, it's configured to store it there. If AI Gateway can't honor the chosen region, the request fails instead of silently running somewhere else. Tools that AI Gateway runs for you, such as web search, are an exception (see where your data goes).

There are two ways to pin a region:

• Team-wide: Pin one or more regions for the whole team from the AI Gateway Dashboard Settings tab. AI Gateway then restricts the team's requests to providers that can serve a selected region. See team-wide region pinning.
• Per-request: Set inferenceRegion in providerOptions for individual requests. This gives you fine-grained control over which requests you pin and where.

A per-request inferenceRegion takes precedence over the team-wide pin, and it must stay inside it: a request that asks for a region the team hasn't pinned fails with HTTP 400 instead of routing outside the pin. See how team-wide and per-request pinning interact.

While your team has a team-wide pin, AI Gateway charges the team-wide pinning surcharge on every successful request from the team, including:

• Requests that set their own inferenceRegion, because the team-wide pin still checks that the region is inside it
• Requests that use your BYOK keys

AI Gateway doesn't charge it on failed requests, including a request that fails because it asked for a region outside the pin. On WebSocket connections, AI Gateway charges the surcharge once per session instead of once per response. You also pay the provider's rate for the region that serves the request (see regional pricing).

To stop the surcharge, remove the team-wide pin: uncheck every region under Region Pinning in the AI Gateway Dashboard Settings tab, then click Save.

Set inferenceRegion on a request and the provider runs inference in that region. To the extent it stores anything at rest, it stores it in the same region. One setting covers both:

What varies by provider is whether anything is stored at rest at all. Many providers store nothing for inference, and zero data retention has the provider delete prompts and responses after the request (see where your data goes).

On Pro and Enterprise plans, a team can pin regions for every request it sends, so callers don't have to set inferenceRegion themselves. This is useful when residency is a team-level policy instead of a per-call decision. Only team Owners can change the pin, and teams on a Pro trial can't turn it on.

To pin regions team-wide:

1. Go to the AI Gateway Dashboard.
2. Open the Settings tab.
3. Under Region Pinning, check the regions you want to allow.
4. Drag the selected regions into your preference order, then click Save.
5. If the team had no regions pinned before, review the surcharge in the Enable Region Pinning dialog and click Enable Region Pinning.

To remove the pin, uncheck every region and click Save. With no regions selected, the team is unpinned, which is the default: requests may use providers in any region.

The order of the selected regions is the fallback priority. A provider that can serve more than one of your regions gets an attempt in each, in the order listed. By default, AI Gateway tries your BYOK keys before system credentials, and applies the region order within each group. With us then eu selected, that means your keys in us, then your keys in eu, then system credentials in us, then system credentials in eu. Without BYOK keys, AI Gateway tries every provider in us before any provider in eu. Either way, if your first region fails, the request falls back to the next selected region instead of failing.

Once you pin regions:

• AI Gateway drops providers that can't serve any selected region from routing. If no provider remains for the requested model, the request fails instead of routing out of region.
• Only models with regional endpoints work while the pin is on (see keeping a request in-region to find them). AI Gateway doesn't have regional endpoints for image, video, speech, transcription, or realtime models yet, so requests and sessions for those models fail.
• An authenticated request to /v1/models with include_availability sets model_eligibility.reason to inference_region_unavailable on models that no provider serves in a pinned region, so you can check the pin's effect before sending a request. See team availability annotations.
• Responses still report the region that served the request, so you can verify residency the same way (see confirming where a request ran).

Team-wide pinning carries a surcharge. See pricing.

A per-request inferenceRegion wins over the team-wide pin, within the bounds of the pin:

Set providerOptions.gateway.inferenceRegion on the request, and AI Gateway routes inference to that region:

• The default (no inferenceRegion) is global, which means AI Gateway picks whichever region can serve the model, based on availability and latency. It can resolve to a different region on each request, and that region may be outside your users' jurisdiction. Residency is opt-in: pin a region if where inference runs matters to you.
• zone requires geoRegion (us, eu, au, or jp). If you omit it, the request fails.
• If your team has a team-wide pin, { scope: 'global' } fails with HTTP 400 because it asks to route outside the pin. Leave inferenceRegion out to route within the pin.
• You set the region in the request body or config only. There's no HTTP header for it.

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'openai/gpt-6-astra',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      inferenceRegion: {
        scope: 'zone',
        geoRegion: 'us',
      },
    },
  },
});
 
console.log(text);

import asyncio
import ai
 
async def main():
    model = ai.get_model("openai/gpt-6-astra")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'openai/gpt-6-astra',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  // AI Gateway extension fields are not included in the upstream SDK types.
  ...{
    providerOptions: {
      gateway: {
        inferenceRegion: {
          scope: 'zone',
          geoRegion: 'us',
        },
      },
    },
  },
});
 
console.log(response.choices[0]?.message.content);

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="openai/gpt-6-astra",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}},
)
 
print(response.choices[0].message.content)

curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "openai/gpt-6-astra",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "gateway": {
      "inferenceRegion": {
        "scope": "zone",
        "geoRegion": "us"
      }
    }
  }
}'

import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'openai/gpt-6-astra',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        inferenceRegion: {
          scope: 'zone',
          geoRegion: 'us',
        },
      },
    },
  },
});
 
for (const block of response.content) {
  if (block.type === 'text') console.log(block.text);
}

import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="openai/gpt-6-astra",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}},
)
 
for block in response.content:
    if block.type == "text":
        print(block.text)

curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "openai/gpt-6-astra",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "gateway": {
      "inferenceRegion": {
        "scope": "zone",
        "geoRegion": "us"
      }
    }
  }
}'

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'openai/gpt-6-astra',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        inferenceRegion: {
          scope: 'zone',
          geoRegion: 'us',
        },
      },
    },
  },
});
 
console.log(response.output_text);

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="openai/gpt-6-astra",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}},
)
 
print(response.output_text)

curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "openai/gpt-6-astra",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "gateway": {
      "inferenceRegion": {
        "scope": "zone",
        "geoRegion": "us"
      }
    }
  }
}'

Most callers set one region and stop. If more than one provider can serve a single request, you can set a default plus per-provider overrides. The top-level value applies to every provider unless you list that provider under providers:

{
  "inferenceRegion": {
    "scope": "zone",
    "geoRegion": "us",           // default for all providers
    "providers": {
      "bedrock": { "scope": "zone", "geoRegion": "eu" },  // Bedrock only: EU
      "anthropic": null                                   // Anthropic: no region pinned
    }
  }
}

This reads as: AI Gateway routes your data to every provider in the US, except it routes Bedrock traffic to the EU and leaves Anthropic unconstrained. AI Gateway ignores unknown provider keys.

AI Gateway routes your data to the provider in the region you pin. To the extent that provider stores your request data at rest, it's configured to store it in that region. Each provider's own terms set what it keeps and for how long, so check the provider's documentation for the specifics.

Tools that AI Gateway runs for you aren't pinned. When a model calls web search tools such as Perplexity Search, Exa Search, or Parallel Search, AI Gateway calls the tool vendor's API wherever the vendor runs it, whether the region comes from the request or a team-wide pin, so search queries built from your prompt can leave the region.

Abuse and safety monitoring is separate. Regardless of your pinned region or retention settings, a provider handles and retains requests it flags for abuse or safety review under its own policies, which can fall outside your region. Edgenivo doesn't control that or guarantee it beyond what the provider documents, so check the provider's terms.

To limit retention, add zero data retention (ZDR), where available. With ZDR, the provider deletes your prompts and responses after the request rather than retaining them. Some provider models do not have ZDR. For example, Anthropic's fable-5 is ineligible for ZDR on every provider, so a request with ZDR configured fails rather than route to it.

Not every provider serves every region, so coverage varies by provider and model (see Current limits). Don't assume a request ran where you asked; confirm the resolved region from the response (see Confirming where a request ran).

Pinning a region can raise what a request costs. Providers often price regional inference above their default routing, and one region can cost more than another for the same model. The provider sets that price.

AI Gateway passes the provider's regional price straight through. Setting inferenceRegion on a request adds no AI Gateway markup on top of it: you pay the provider's rate for the region you picked.

AI Gateway bills team-wide pinning separately from the provider's rate. See pricing.

To see what a model charges per region, open it from the model list and hover the price in the Providers table. The Price by region card shows each region's rate as a percentage difference from the global price, for example EU +10%. A region that costs the same as global reads Same as global.

The https://ai-gateway.vercel.sh/v1/models endpoint returns the same rates as absolute per-token numbers, under pricing.regional:

{
  "pricing": {
    "input": "0.000005",
    "regional": {
      "eu": { "input": "0.0000055" },
      "us": { "input": "0.0000055" }
    }
  }
}

To see what a specific request cost, check the cost reported in the response metadata.

You pay a region's rate only when the request's inferenceRegion or your team-wide pin selects that region. With neither (the global default), you pay the provider's standard rate.

Bringing your own key doesn't opt you out of residency. When you use BYOK credentials, AI Gateway applies inferenceRegion the same way it does with system credentials:

• AI Gateway routes your key's requests to in-region endpoints. Your key material is untouched; AI Gateway regionalizes how it's used, calling the provider's in-region endpoint with your key.
• An explicit region overrides a region saved on the credential. If your Vertex credential is saved with a location and the request asks for a different region, the request's region wins. If you don't set a region, the credential's saved location applies where it has one.
• Failure behavior is the same. If no provider can honor the region with your credentials, the request fails with HTTP 400. AI Gateway doesn't fall back to an out-of-region endpoint, even when the credentials are your own.
• Verification is the same. Responses report the resolved region for BYOK requests too, so you can confirm residency from the response metadata.
• Zero data retention skips BYOK keys by default. If you pair a pinned region with ZDR, AI Gateway routes around your own credentials unless you mark a key as ZDR-compliant, since those keys run under your provider agreements rather than Edgenivo's. See ZDR and BYOK.

Set inferenceRegion and the provider processes your prompts and completions, and stores anything it keeps at rest, only in your chosen region. If it can't, the request fails instead of running elsewhere.

To limit retention, add zero data retention and disallow prompt training, so the provider deletes your data after the request and doesn't train on it. Whether a provider stores anything by default is provider-dependent (see where your data goes).

Region coverage varies by model. To find models that support a region, filter the model list:

• Models available in the US
• Models available in the EU
• Models available in Australia
• Models available in Japan

The /v1/models endpoint returns the same coverage as a regions array per model (for example, ["eu", "us"]); a model with no regions field doesn't support regional routing.

This pins where the provider processes and stores your data. It doesn't pin the network path your request takes to reach the provider.

• If you request no region, AI Gateway routes your data globally. This isn't an error.
• If AI Gateway can't honor the region, the request fails with HTTP 400 (invalid_request_error). AI Gateway doesn't silently fall back to another region.
• There's no automatic downgrade from a preferred region to a broader one.

When you pin a region, every response reports where AI Gateway actually ran inference, so you can verify residency instead of assuming it. The resolved region is in the gateway routing metadata, on the provider attempt that served the request.

With the AI SDK, read it from providerMetadata.gateway. On the OpenAI-compatible endpoints, the same object sits on each choice's message as provider_metadata.gateway:

{
  "gateway": {
    "routing": {
      "finalProvider": "openai",
      "modelAttempts": [
        {
          "providerAttempts": [
            {
              "provider": "openai",
              "inferenceEndpoint": {
                "slug": "geo-us",
                "scope": "zone",
                "geoRegion": "us"
              }
            }
          ]
        }
      ]
    }
  }
}

Read inferenceEndpoint.geoRegion (here us) and compare it against the region you asked for; finalProvider names the provider that served the request. When you don't pin a region, inferenceEndpoint is null, matching the global default, which doesn't pin a region.

If GDPR is the driver for pinning a region, consider your compliance needs: where personal data is processed is one obligation among many. Regional routing answers the location question; it doesn't by itself make a workload GDPR-compliant.

• The regions available today are us, eu, au, and jp, and not every model supports all of them. Check a model's regions in the model list or from https://ai-gateway.vercel.sh/v1/models.
• Region selection is opt-in; the default is global, both per request and team-wide.
• There's no cross-region fallback: a region that AI Gateway can't honor fails the request. Team-wide pinning falls back across the regions you selected, in order, but never outside them.
• Regional routing doesn't apply to web search tools that AI Gateway runs for you (see where your data goes).
• Image, video, speech, transcription, and realtime models don't have regional endpoints yet, so they don't work with a region pinned.

Pin an inference region on a virtual model to route all of its system-credential traffic to that region. The virtual model's region wins over the request's. If your team has a team-wide pin, the virtual model's region must be inside it, or requests to the virtual model fail with HTTP 400.