Check out the newest way to compare different models for a task/agent harness: AutoEvals
Tracing

Get Started

Capture Your First Trace

Inference Tracing captures the full execution of your AI apps and agents: LLM calls, tool calls, framework steps, and any custom spans you add. Drop the SDK into your app, point it at the Inference platform, and traces start flowing.

This guide gets you from zero to a captured trace. The example uses OpenAI because it is the smallest end-to-end setup. The same flow works for Anthropic, LangChain, LangGraph, the Vercel AI SDK, Vercel Eve, OpenAI Agents, LiveKit Agents, Pi Agent, legacy PI AI, Pydantic AI, and the other supported integrations.

Either path below installs and wires up the Inference Tracing SDK: @inference/tracing on npm for TypeScript, or inference-tracing on PyPI for Python.

To get started with the Inference platform, create a free account at inference.net.

Choose a setup path

Installing with AI is the quickest. Use the manual flow if you want to review each change yourself.

Use the Inference CLI to launch a coding agent like Claude Code to install the tracing SDK, configure export, and wire up your existing LLM clients.

Install the CLI and authenticate

Install the Inference CLI globally and log in. Your browser will open to authenticate.

npm install -g @inference/cli && inf auth login

Run tracing instrumentation in your project

From your project root, run instrumentation in tracing mode.

cd /path/to/your/project && inf instrument --mode tracing

The command guides you through the following workflow:

  • Select a coding agent: Claude Code, OpenCode, or Codex.
  • Scan your codebase for LLM clients and agent frameworks.
  • Install the tracing SDK and configure export to the Inference platform.
  • Wire setup() into your app entrypoint so spans start before clients are constructed.
  • Add stable service and agent identity so traces group cleanly in the dashboard.
  • Review the generated changes before applying them.

Pick both instead of tracing if you also want to route requests through the Inference Gateway in the same pass.

Run your app

Run your application how you normally would. Traces stream to the Inference platform as your code executes.

View your trace

Open the dashboard and filter by your service name to see the captured trace tree.

Want the full canonical guide for this workflow? See Install with AI.

Use this path if you want to wire it up yourself. The example below uses OpenAI. For other providers and frameworks, see the Tracing integrations guide.

Install the SDK

Configure export

Set the Inference platform traces endpoint and token before your app starts.

export INFERENCE_OTLP_ENDPOINT="https://telemetry.inference.net"
# Get your API key from https://inference.net/dashboard/api-keys/
export INFERENCE_API_KEY="<your-token>"
export INFERENCE_SERVICE_NAME="checkout-agent"

Use a stable INFERENCE_SERVICE_NAME per deployed service. It makes traces easier to filter and compare across environments.

Initialize tracing early

Call setup() before constructing clients from instrumented SDKs.

If the process is short-lived, always call shutdown() before exit so batched spans are flushed.

Longer-lived processes flush differently. A long-lived server (HTTP, Slack bot, queue worker) memoizes setup() and calls shutdown() on SIGTERM, not per request. A serverless or edge function instead flushes per invocation with tracing.provider.forceFlush(). See Flushing and process lifecycle in the quickstart.

View your trace

Open the dashboard and navigate to the Agents or Traces tab. You'll see an LLM span with input messages, output messages, model name, invocation parameters, finish reason, and token counts.

Group calls under an agent

A single LLM call is captured automatically. To get an AGENT row with stable agent.id, agent.name, and session.id for dashboard grouping, wrap your code in agentSpan. Use manualSpan inside it for non-LLM steps like tools, retrieval, and validation. The example below reuses the tracing and client from the step above.

The trace now shows an AGENT row for "Hello Agent" with the LLM call and the validate_reply CHAIN row nested under it. For the full surface (framework integrations, multi-agent setups, identity propagation), see the tracing integrations guide.

Need a different provider or framework? See Tracing integrations for OpenAI, Anthropic, LangChain, LangGraph, the Vercel AI SDK, Vercel Eve, OpenAI Agents, LiveKit, Claude Agent SDK, Pi Agent, PI AI, Pydantic AI, and more.

That's it. Spans are streaming to the Inference platform and your trace is ready to inspect.

What gets captured

Span dataExamples
Inputs and outputsinput.value, output.value
Messagesuser, system, assistant, tool, and tool-result messages
Tool callstool names, IDs, JSON arguments, and tool results
Model metadatamodel name, provider/system, invocation parameters
Usageprompt, completion, total, and prompt-cache token counts
Agent structureagent spans, framework spans, tool spans, graph/node spans
Errorsexception status and error details on failed spans

Next steps

On this page