Traces

Inference Tracing captures the full execution of your AI apps and agents: LLM calls, tool calls, framework steps, agent runs, and any custom spans you wrap. Spans are emitted as OpenInference-shaped OpenTelemetry, exported over OTLP/HTTP, and shown in the Inference platform as trace trees, agent sessions, and per-span IO.

Use tracing for agent improvement. Once spans are flowing, point Halo, our open-source agent-loop optimizer, at your traces and get back concrete suggestions for fixing prompts, tools, and harness logic. See Analyze your traces for the dashboard and Halo flow.

How Tracing Fits Together

Tracing has three layers. You add them in order, and most apps stop at the second.

1. Automatic SDK spans. Pass an SDK to setup({ modules }) and every call through it becomes its own span, with input messages, output, model name, finish reason, and token counts. You change nothing at the call site. For a one-shot script, or a batch of unrelated LLM calls, this is all you need.

2. An agent span around the run. When one request drives several model or tool calls (an agent run, a conversation turn, a workflow), wrap it in agentSpan. The agent span is the parent row the SDK spans nest under, so you see "Support Agent" with "openai chat" and "lookup_order" beneath it instead of three orphan rows. It carries agentId, agentName, role, and sessionId for grouping in the Agents dashboard, and holds the run's high-level input and output. Without it the SDK spans are still captured, just as a flat list with no agent or conversation to group them by.

3. Manual spans for your own steps. When a step inside the run is not an SDK call (a tool you execute yourself, a retrieval, a router, a validation pass), wrap it in manualSpan with the matching span kind. It nests under the agent span next to the auto-captured LLM spans, so the trace tree reflects everything the run did, not just the model calls.

Packages

Two packages, one shape. Install whichever matches your runtime.

Both packages share the same setup() entry point, the same env-var contract, and the same OpenInference attribute conventions. Pick whichever fits your stack and write the same code shape in either language.

Getting Started

The fastest path is the Inference CLI:

inf instrument --mode tracing

tracing mode installs the SDK and wires up setup(). both mode does the same plus gateway routing in one pass. The command launches your choice of coding agent (Claude Code, OpenCode, or Codex) to make the edits. See Install with AI for the full flow.

Prefer to wire it up by hand? Start with the path that matches what you want to instrument:

1. Quickstart installs the SDK, configures export, and captures your first provider span. Both Install with AI and Install manually paths included.
2. Provider and framework guides cover OpenAI, Anthropic, LangChain, LangGraph, LangSmith, OpenAI Agents, LiveKit Agents, Claude Agent SDK, Claude Code, Pydantic AI, ElevenLabs Agents, Vercel Eve, Pi Agent, PI AI, Cursor SDK, the Vercel AI SDK, and other supported surfaces.
3. Manual spans wrap custom agents, subprocesses, tools, retrieval, routing, and unsupported SDKs.
4. Agent identity sets stable agent.id values so the Agents dashboard groups executions correctly.

Supported Trace Integrations

• OpenAI: Trace Chat Completions, tool calls, structured outputs, and Responses API calls.
• Anthropic: Trace Messages API calls, tool use round trips, and prompt-cache usage.
• LangChain: Capture chains, agents, model calls, and tools through callback instrumentation.
• LangGraph: Capture graph and node spans while preserving parent-child relationships.
• LangSmith: Bridge LangSmith OpenTelemetry spans to the Inference platform.
• OpenAI Agents: Trace agent runs, tools, handoffs, and nested OpenAI model calls.
• LiveKit Agents: Trace LiveKit sessions, model nodes, and function tools through native OTel spans.
• ElevenLabs Agents: Trace conversation sessions, transcripts, and client tool calls.
• Vercel Eve: Trace Eve agent turns, model calls, sub-agent invocations, tools, and $eve.* workflow tags.
• Pi Agent: Trace current @earendil-works Pi agents, model turns, streams, tool calls, usage, and costs.
• PI AI: Trace legacy @mariozechner/pi-ai model turns, streams, tool calls, usage, costs, and agent identity.
• Cursor SDK: Trace Cursor agent runs, streamed messages, and tool calls.
• Vercel AI SDK: Trace generateText, streamText, ToolLoopAgent, tool calls, structured output, and usage.
• Claude Agent SDK: Trace Claude Agent SDK query loops and yielded messages.
• Claude Code SDK: Trace Claude Code CLI subprocess calls and SDK-style agent invocations.
• OpenCode: Record your own OpenCode coding sessions with LLM calls, tool calls, token usage, cost, and errors.
• Pydantic AI: Trace Pydantic AI agent runs and tool calls (Python only).
• Manual spans: Wrap custom agents, CLI calls, provider routing, and unsupported SDKs.
• Agent identity: Set stable agent IDs, display names, and roles for dashboard grouping.

What Gets Captured

All keys follow OpenInference conventions, so the same spans can be consumed by any OpenInference-aware viewer.

Configuration

Both SDKs read the same env vars and accept the same options on setup().

Env Vars And Identity

Precedence: explicit options on setup() win, then INFERENCE_* env vars, then SDK defaults.

Instrumentation Control

Two options control which SDKs Inference platform patches at setup time.

Three patterns, in order of how often they show up in real apps:

Auto-detect everything. Call setup() with no modules:. The auto-scan finds every supported SDK and patches it. Easiest path for scripts, demos, and small apps.

const tracing = await setup();

Explicit modules plus auto-detect. Pass the SDKs you use to modules:. Those entries are patched directly (no resolver gymnastics), and setup() also auto-detects anything else.

const tracing = await setup({ modules: { openai: OpenAI } });

Explicit only. Pass modules: and set autoInstrument: false. Only the SDKs you list get patched, even if a transitive dep happens to pull in another supported one. Cleanest pattern for production services that know exactly which providers they call.

const tracing = await setup({
  modules: { anthropic: Anthropic },
  autoInstrument: false,
});

Imports (TypeScript)

For almost all setups, @inference/tracing is the only import you need. setup() auto-patches every supported global SDK it can find (OpenAI, Anthropic, LangChain, LangGraph, ElevenLabs, LiveKit, legacy PI AI, and the rest). Current Pi Agent apps use per-instance Models collections, so pass the collection as modules.piAgent.

When you need a subpath

Four cases call for importing from a per-integration subpath:

1. You're using the Vercel AI SDK. The AI SDK has its own experimental_telemetry option, so there is nothing for setup() to patch. You import createAISdkTelemetrySettings and pass its result into each generateText / streamText call.
2. You're using Vercel Eve. Eve enables telemetry by loading agent/instrumentation.ts, so you export defineCatalystEveInstrumentation() from @inference/tracing/eve in that file.
3. You're adding tracing after creating a Pi Models collection. Call instrumentPiAgent(models, tracing) before the first model call.
4. You're installing instrumentation on a custom tracer provider instead of the one setup() registers. Rare in practice; most apps want the standard flow. The subpath helpers (installOpenAIInstrumentation, etc.) accept an explicit provider argument.

If neither applies, you can skip the table below.

Python uses one import root (inference_tracing) and gates each integration behind a pip extra (pip install 'inference-tracing[openai,anthropic]').

The setup() Return Value

setup() returns a CatalystTracing handle:

For long-lived and serverless processes, memoize the setup() promise and flush correctly for the process shape. See Flushing and process lifecycle.

How It Works

1. setup() configures an OpenTelemetry tracer provider with a Inference Tracing OTLP/HTTP exporter.
2. The SDK auto-patches any supported provider or framework SDK it finds. In TypeScript you pass them via modules. In Python it auto-detects installed packages.
3. Your app runs normally. Spans are batched and exported in the background.
4. Spans land in the Inference platform grouped by service, trace, span, and task metadata. View them in the Traces tab or the Agents tab.
5. Run analysis on them with Halo to understand your traces and gain insights you can pass into your favorite coding agent.

Inspect Traces from the CLI

After spans are flowing to the Inference platform, you can inspect the same trace data from the terminal:

# Browse recent traces
inf trace list --range 1h

# Open a trace tree or timeline
inf trace get <trace-id> --view tree
inf trace get <trace-id> --view timeline

# Search spans and inspect captured IO
inf span list --trace-id <trace-id> --kind LLM
inf span get <trace-id> <span-id> --view io

See inf trace and inf span for the full command reference.

Next Steps