What Agent Traces Should Actually Contain

Agent traces should do more than prove that a request happened. They should make a run understandable. When a user receives a slow, expensive, or incorrect answer, an engineer should be able to open one trace and see how the workflow reached that outcome.

Start with a stable root

Every trace needs a root operation representing the user task or background job. It should include a trace ID, the workflow name, the initiating user or organization when appropriate, the environment, the start and end time, and the final outcome. All child operations should inherit that context so the system can connect a tool call back to the request that caused it.

Record the execution tree

Agent work is naturally hierarchical. A root workflow may contain a planning model call, which produces two tool calls, followed by a synthesis model call and a validation step. Each operation should record its parent, type, source, start and end time, and outcome.

Useful operation types include LLM calls, retrieval, MCP client calls, MCP server execution, external API calls, cache lookups, validation, retries, and approval steps. The taxonomy does not need to be complicated, but it needs to be consistent.

Capture decisions, not just payloads

Raw payloads can be helpful, but a trace also needs the decisions that shaped the workflow. Which model was selected and why? Which cache rule matched? Was the request a hit, miss, or bypass? Did a policy deny a tool? Did a retry use a fallback? These fields explain behavior that timing alone cannot.

Payload capture must respect privacy and security. Sensitive fields should be redacted, access to full traces should be controlled, and retention should match the application's data policy.

Make timing and cost comparable

Each operation should expose duration, token usage where relevant, provider cost, and cache savings. Aggregate values at the root make it easy to understand the cost of a user task. Child-level values show which operation is responsible for the outlier.

Latency distributions are more informative than a single average. Comparing cache hits and misses, p50 and p90 latency, and different model routes can reveal changes that would be invisible in aggregate logs.

End with a meaningful outcome

A trace should record whether the workflow succeeded, failed, was cancelled, returned a partial result, or required human approval. Errors should preserve their source and classification so teams can separate a provider failure from an invalid tool response or an application bug.

Design traces for action

The best trace is not the one with the most data. It is the one that lets an engineer answer a question and make a decision: remove repeated context, cache a stable tool result, adjust a timeout, revise a policy, or investigate a model regression. That is when observability becomes part of the development loop rather than a passive archive.