Reliability
Whether the requested operation completes and its effects can be verified, including how unknown evidence is handled.
Independent evaluation infrastructure
As AI agents gain access to money, smart contracts, and paid services, Web3 faces an infrastructure problem: developers do not yet have a widely accepted, independent way to compare how safely, economically, and reliably agents carry out operations.
Web3 Agent Bench measures agent execution against versioned tasks in an isolated simulated environment. It reports outcome, operating cost, latency, safety behavior, and evidence quality as separate dimensions.
Simulation only: no mainnet execution, real-money task funds, or claim of production safety.
Whether the requested operation completes and its effects can be verified, including how unknown evidence is handled.
Whether the agent respects task constraints, permissions, budgets, and required final state.
Outcome, latency, and operating cost stay visible as separate measurements rather than one unexplained score.
Why this matters
An agent can authorize a transfer, change a smart-contract permission, pay for data, or combine these steps. Product demos alone do not show whether it chose valid actions, stayed within cost limits, or left the system in the required state.
The benchmark publishes task goals and grading criteria while keeping exact Ranked case inputs and reference answers sealed until retirement.
Read the task catalogue and participation rules →