AgentDiff

AgentDiff revolutionizes agent validation by focusing on execution trajectories rather than just final outputs. Key features include: • Records agent runs as Directed Acyclic Graphs (DAGs) • Diffs execution trajectories against golden baselines • Detects redundant tool loops and cost spikes • Identifies latency regressions before merge • Framework-agnostic with flexible adapters This tool addresses the critical challenge of silently failing agents where traditional tests might pass despite inefficient or incorrect internal processes. It provides deep visibility into how an agent arrives at an answer, ensuring optimal performance and resource utilization. AgentDiff provides a readable diff, making it easy to diagnose issues such as trajectory divergence, excessive tool use, and unexpected cost increases. By integrating directly into CI/CD pipelines, it establishes a hard gate, preventing regressions from shipping and ensuring robust, predictable agent behavior. Ideal for engineering teams developing and deploying autonomous agents who need to maintain stringent quality controls over agent behavior, resource consumption, and reliability. It supports various frameworks through adapters, including OpenAI Agents, Langfuse, LangSmith, and OpenInference.