Gateway
Gateway Quickstart
Route your first LLM request through the Inference Gateway and see it land in the dashboard.
This page is the gateway-focused quickstart. Point your SDK at https://api.inference.net/v1, add a couple of headers, and Inference platform captures every request with cost, latency, and full request/response payloads. If you'd rather see the higher-level Get Started flow, start with Record your first LLM call.
The example below uses OpenAI. For other providers (Anthropic, Vertex AI, Amazon Bedrock, Gemini, OpenRouter, Cerebras, Groq, LangChain, ElevenLabs), see the Gateway overview.
Choose a setup path
Installing with AI is the quickest. Use the manual flow if you want to wire it up yourself.
Use the Inference CLI to launch a coding agent like Claude Code, OpenCode, or Codex to scan your codebase, update your LLM clients, and add the routing headers.
Install the CLI and authenticate
Install the Inference CLI globally and log in. Your browser will open to authenticate.
npm install -g @inference/cli && inf auth loginRun gateway instrumentation in your project
From your project root, run instrumentation in gateway mode.
cd /path/to/your/project && inf instrument --mode gatewayThe command guides you through the following workflow:
- Select a coding agent: Claude Code, OpenCode, or Codex.
- Scan your codebase for LLM clients such as OpenAI, Anthropic, LangChain, etc.
- Redirect base URLs to the Inference Gateway.
- Add routing headers so requests are authenticated, forwarded, and tagged.
- Add task IDs so each call site is grouped automatically in the dashboard.
- Review the generated changes before applying them.
Pick both instead of gateway to also install the tracing SDK in the same pass. Run inf instrument --dry-run to preview changes without modifying any files.
Run your app
Run your application how you normally would. Requests now flow through Gateway and appear in the dashboard.
View your results
Open the dashboard to see request details and analytics.
Want the full canonical guide for this workflow? See Install with AI.
Use this path if you want to wire it up yourself. The example below uses OpenAI. For Anthropic, Vertex AI, Amazon Bedrock, Gemini, OpenRouter, Cerebras, Groq, LangChain, and ElevenLabs, see the per-provider guides linked from the Gateway overview.
Get your API keys
You need two keys:
- Inference project API key from your dashboard under API Keys
- Provider API key (in this example, OpenAI) from your OpenAI account
Set them as environment variables:
export INFERENCE_API_KEY=<your-project-api-key>
export OPENAI_API_KEY=<your-openai-api-key>Update your code
Point your SDK at https://api.inference.net/v1 and use your Inference project API key as the SDK's apiKey. Your provider's API key goes in the x-inference-provider-api-key header so the gateway can forward it. The gateway adds roughly 10ms of latency and forwards your requests to the provider as-is.
Send a request
Run the snippet above. Once the request completes, Inference platform captures it automatically.
View your results
Open the dashboard to inspect the request and metrics.
Need a different provider? See the Gateway overview for per-provider guides, or use any OpenAI-compatible endpoint through the x-inference-provider-url header.
That's it. Every request now flows through Gateway and gets captured automatically.
Headers
These headers control routing, authentication, and how the request gets tagged in the dashboard. The only one required for every request is Authorization. Add the others as needed.
| Header | Required | Description |
|---|---|---|
Authorization | Yes | Bearer <your-project-api-key>. Authenticates the request to the Inference platform and selects the project scope. For OpenAI-compatible SDKs, set this as the SDK's apiKey. |
x-inference-provider-api-key | When proxying a provider | Your downstream provider's API key (OpenAI, Bedrock, Groq, Cerebras, etc.). The gateway forwards it as bearer auth so your code never has to. For Anthropic's native /v1/messages route, use x-api-key instead. |
x-inference-provider | Optional | Forces routing to a specific provider (openai, anthropic, groq, cerebras, vertex-ai, gemini). When omitted, the gateway infers the provider from the SDK path or base URL. Set this only when you want to override that inference. |
x-inference-provider-url | Optional | Routes to any OpenAI-compatible provider by base URL, even one without a dedicated integration. For third-party OpenAI-compatible URLs, the gateway infers OpenAI automatically. Pair with x-inference-provider only when you want to force a specific provider name. |
x-inference-environment | Optional | Tags the request with an environment name like production, staging, or development. Filterable in the dashboard. |
x-inference-task-id | Optional | Groups requests under a logical task such as summarize-docs or chat-support. Useful for filtering, analytics, and building datasets. |
x-inference-metadata-* | Optional | Attach arbitrary metadata to a request. The x-inference-metadata- prefix is stripped to form the key (e.g., x-inference-metadata-chat-id: abc123 stores chat-id: abc123). Filter inferences and create datasets in the dashboard using these keys. |
Provider base URLs
The base URL you point your SDK at determines which provider the gateway forwards to. Most providers don't need an explicit x-inference-provider header, the gateway figures it out from the URL.
| Provider | Base URL | Note |
|---|---|---|
| OpenAI | https://api.openai.com/v1 | Default routing, no provider header needed. |
| OpenRouter | https://openrouter.ai/api | No provider header needed unless you want to force openai. |
| Amazon Bedrock | https://bedrock-mantle.{region}.api.aws/v1 | Use a Bedrock bearer/API key in x-inference-provider-api-key. |
| Anthropic | https://api.anthropic.com/v1 | No provider header needed for /v1/messages or api.anthropic.com. |
| Google Gemini | https://generativelanguage.googleapis.com | Use /v1beta/models/* native paths, or /v1beta/openai for OpenAI-compatible calls. |
| Vertex AI | https://aiplatform.googleapis.com/v1/projects/{project}/locations/global/endpoints/openapi | Set x-inference-provider: vertex-ai. |
| Azure OpenAI | https://{resource}.openai.azure.com/openai/deployments/{deployment} | |
| Groq | https://api.groq.com/openai/v1 | |
| Together AI | https://api.together.xyz/v1 | |
| Fireworks AI | https://api.fireworks.ai/inference/v1 | |
| Perplexity | https://api.perplexity.ai | |
| Mistral | https://api.mistral.ai/v1 | |
| DeepSeek | https://api.deepseek.com/v1 | |
| Cerebras | https://api.cerebras.ai/v1 | |
| Inference.net | https://api.inference.net/v1 |
What gets captured
Once traffic is flowing, Inference platform records:
- The full request and response payloads
- Cost per call and aggregate spend
- Latency, including time to first token (TTFT) and tokens per second
- Token counts (input and output)
- Error rates and status codes
- Model and provider
Where to find your data
- Metrics Explorer for cost, latency, errors, and usage across all your LLM calls
- Inference Viewer to browse and filter individual requests and responses
Next steps
Gateway overview
Routing headers, supported providers, and the full set of OpenAI-compatible base URLs.
Connect more providers
Set up Anthropic, Vertex AI, Amazon Bedrock, Gemini, OpenRouter, Cerebras, Groq, and more.
Organize with tasks
Group LLM calls by feature or objective to track metrics separately.
Build a dataset
Turn captured traffic into datasets for evals and training.