API Quickstart

The Inference.net API is OpenAI-compatible, so you can use the OpenAI SDK or plain HTTP to make requests. There are three ways to use it:

1. Call a model serverless: call open-source models and popular closed-source models (GPT, Claude, Gemini) with just your Inference API key. Usage is billed per token to your credit balance.
2. Proxy through Inference Gateway: route requests to any provider (OpenAI, Anthropic, etc.) through Inference Gateway with your own provider API key.
3. Call your custom model: hit a model you've fine-tuned and deployed on the platform.

All three paths go through Inference Gateway, so you get the same metrics, cost tracking, and eval-readiness whichever one you use.

Get an API Key

export INFERENCE_API_KEY=<your-api-key>

1. Call a Model Serverless

Call models with just your Inference API key. No provider API key is needed. This works for two kinds of models:

• Open-source models hosted on Inference.net, such as glm-5.2.
• Popular closed-source models, such as claude-haiku-4-5, gpt-5-mini, and gemini-3.5-flash. Inference.net routes the request to the provider for you and bills the usage per token to your credit balance.

Browse available models at inference.net/models, or list them with GET https://api.inference.net/v1/models.

Prefer the Anthropic SDK? The API also supports the Anthropic Messages format. See Anthropic SDK.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "glm-5.2",
  messages: [{ role: "user", content: "What is the meaning of life?" }],
  stream: true,
});

for await (const chunk of response) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

The same code works for every serverless model. Set model to the model id you want, for example claude-haiku-4-5. This includes our purpose-built Schematron models for structured data extraction.

2. Proxy Through Inference Gateway

Route requests to any LLM provider (OpenAI, Anthropic, Groq, etc.) through Inference Gateway. You keep your existing provider API key, and the provider bills you directly. The gateway adds observability, cost tracking, and eval-readiness with roughly 10ms of added latency.

Use this path when you want a model that is not in the serverless catalog, or when you want usage billed to your own provider account. The captured metrics are the same as for serverless calls.

Your Inference project API key authenticates with the gateway. Your provider API key is forwarded to the provider via the x-inference-provider-api-key header.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY,
  defaultHeaders: {
    "x-inference-provider-api-key": process.env.OPENAI_API_KEY,
    "x-inference-provider": "openai",
  },
});

const response = await client.chat.completions.create({
  model: "gpt-4.1",
  messages: [{ role: "user", content: "What is the meaning of life?" }],
});

console.log(response.choices[0].message.content);

3. Call Your Custom Model

Hit a model you've fine-tuned and deployed on Inference.net. The model path is your team slug followed by the deployment name, shown on your deployment's detail page in the dashboard.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.net/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "your-team/your-model",
  messages: [{ role: "user", content: "Hello, world!" }],
});

console.log(response.choices[0].message.content);

Headers Reference

Supported Request Parameters

The API supports the standard OpenAI chat completions parameters:

Next Steps