Check out the newest way to compare different models for a task/agent harness: AutoEvals
Providers

Traces

OpenAI Traces

Trace OpenAI Chat Completions, tool calls, structured outputs, and Responses API calls.

Inference platform instruments the OpenAI SDK in both TypeScript and Python. Initialize tracing before constructing clients so the OpenAI prototypes are patched before application calls start.

What Is Captured

  • Chat Completions request and response messages
  • Responses API input and output text
  • Tool call names, IDs, and JSON arguments
  • Tool result messages passed back into later turns
  • JSON schema response format in invocation parameters
  • Model name, finish reason, usage, and token counts

Install

Basic Chat

OpenAI Inside An Agent

Tool Calling

The first model call records the assistant tool call. The second model call records the tool result message and the final answer.

The capture below is the model-side view of tool calling — the request/response the LLM saw. To also capture the caller-side view (the actual function that ran, its input, output, and duration), wrap the tool function in a TOOL span using Manual spans. For a full agent loop, see the Production Agent Example.

Structured Outputs

response_format is included in invocation parameters, so you can inspect which schema constrained the model call.

TypeScript
const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    {
      role: "user",
      content: "Extract the city, temperature, and unit from: 72F in Berlin.",
    },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "weather_report",
      strict: true,
      schema: {
        type: "object",
        additionalProperties: false,
        properties: {
          city: { type: "string" },
          temperature: { type: "number" },
          unit: { type: "string", enum: ["F", "C"] },
        },
        required: ["city", "temperature", "unit"],
      },
    },
  },
});

console.log(response.choices[0]?.message.content);

Responses API

Responses API calls are traced with the same OpenInference attribute families as Chat Completions.

TypeScript
const response = await client.responses.create({
  model: "gpt-4o-mini",
  input: "In one sentence, what is OpenTelemetry?",
});

console.log(response.output_text);

On this page