Check out the newest way to compare different models for a task/agent harness: AutoEvals
Agent Frameworks

Traces

Claude Code SDK Traces

Trace Claude Code CLI and SDK-style invocations with OpenInference AGENT spans.

Use this guide when your application invokes Claude Code as a CLI process, such as claude -p, or when you have an existing Claude Code wrapper that shells out to the claude binary. The trace shape is an explicit AGENT span with the prompt as input and Claude Code output as output.

Anthropic renamed the Claude Code SDK to the Claude Agent SDK. If your app imports @anthropic-ai/claude-agent-sdk or claude_agent_sdk, use Claude Agent SDK traces. If your app invokes the claude binary directly, use the subprocess pattern below.

Install

TypeScript CLI Invocation

Resolve the claude binary once, then wrap each non-interactive invocation with agentSpan(). Set CLAUDE_BIN when Claude Code is installed outside PATH or when you want to pin a specific executable.

TypeScript
import { execFile, execSync } from "node:child_process";
import { promisify } from "node:util";
import { agentSpan, setup } from "@inference/tracing";

const execFileP = promisify(execFile);

function resolveClaudeBin(): string {
  if (process.env.CLAUDE_BIN != null && process.env.CLAUDE_BIN !== "") {
    return process.env.CLAUDE_BIN;
  }

  const pathSansNodeModules = (process.env.PATH ?? "")
    .split(":")
    .filter((pathEntry) => !pathEntry.includes("/node_modules/"))
    .join(":");

  return execSync("which claude", {
    env: { ...process.env, PATH: pathSansNodeModules },
    stdio: ["pipe", "pipe", "pipe"],
  })
    .toString()
    .trim();
}

const tracing = await setup({ serviceName: "claude-code-worker" });
const prompt = "Inspect this repository and list the top three TODO comments.";

await agentSpan(
  {
      agentId: "claude-code-repo-worker",
      agentName: "Claude Code",
      spanName: "claude-code.invocation",
      sessionId: "conversation-cli-hello",
      system: "anthropic",
  },
  async (span) => {
    span.setInput(prompt);

    const { stdout } = await execFileP(resolveClaudeBin(), ["-p", prompt], {
      encoding: "utf-8",
      timeout: 120_000,
    });

    span.setOutput(stdout.trim());
  },
);

await tracing.shutdown();

Python CLI Invocation

Use the same pattern in Python when a worker, job, or automation script shells out to Claude Code.

Python
import os
import shutil
import subprocess
import sys

from inference_tracing import agent_span, setup
from dotenv import load_dotenv


def resolve_claude_bin() -> str:
    override = os.environ.get("CLAUDE_BIN")
    if override:
        return override

    resolved = shutil.which("claude")
    if resolved is None:
        print("`claude` binary not found on PATH.", file=sys.stderr)
        sys.exit(1)
    return resolved


load_dotenv()

tracing = setup(service_name="claude-code-worker")
prompt = "Inspect this repository and list the top three TODO comments."

with agent_span(
    tracing.tracer,
    agent_id="claude-code-repo-worker",
    agent_name="Claude Code",
    span_name="claude-code.invocation",
    session_id="conversation-cli-hello",
    system="anthropic",
) as span:
    span.set_input(prompt)

    completed = subprocess.run(
        [resolve_claude_bin(), "-p", prompt],
        capture_output=True,
        text=True,
        timeout=120,
        check=True,
    )

    span.set_output(completed.stdout.strip())

tracing.shutdown()

Agent SDK Query Loop

If you are using Claude Code programmatically through the Agent SDK, trace the SDK's query() loop instead of tracing a raw subprocess.

What To Look For

  • An AGENT span named claude-code.invocation for direct CLI subprocess work
  • agent.id=claude-code-repo-worker for stable Agents dashboard grouping
  • agent.name=ClaudeCode and gen_ai.system=anthropic on the span
  • Prompt input and stdout output captured on the span
  • Error status and exception details if the subprocess exits non-zero
  • Agent SDK query spans when using the claude_agent_sdk package

On this page