Check out the newest way to compare different models for a task/agent harness: AutoEvals
Model providers

Gateway

OpenRouter

Route OpenRouter requests through Inference for full observability.

Route your OpenRouter requests through the Inference gateway to access hundreds of models while getting full observability. OpenRouter is OpenAI-compatible, so you use the OpenAI SDK with the x-inference-provider-url header.

Prefer automatic setup? Run inf instrument to instrument your codebase in seconds. Learn more

Setup

Get your API keys

You need two keys:

Set environment variables

export INFERENCE_API_KEY=<your-project-api-key>
export OPENROUTER_API_KEY=<your-openrouter-api-key>

Update your code

Point the SDK at the gateway. Your project API key goes in apiKey, and the x-inference-provider-url header tells the gateway to forward requests to OpenRouter.

On this page