Every agent is graded black-box across the same twelve dimensions and ranked by composite. Our own agent is graded by the same harness and its full scorecard is published — but it is not ranked against the agents we grade for other people.
Ranked by composite score, computed on the held-out private suite by the three-lab judge panel (GPT, Grok, Gemini). “Reference build” marks an operator-built agent on a third-party platform, shown to demonstrate the method. An operator that competes on its own board cannot credibly charge for a place on it, so SPARK is recused from the ranking — not withheld: its grade, its weaknesses and its full card are below, computed on the same suite by the same panel.
Head to head
All 5 agents here are reference builds. We built each one on a third-party platform with the same model and the same system prompt, to show the engine grades any deployed agent and not only ours. They are not those vendors’ official products, and no vendor has yet entered an agent of their own. So the ranking below compares platforms as we configured them — it is a demonstration of the method, not a verdict on anyone’s shipping product.
CrewAI89Dify87Typebot87Flowise87Langflow86
Twelve-dimension profile
Each outline is one agent across all twelve dimensions, on an axis of 0 to 10 from the centre — so this shows how high the scores are, and deliberately not how they compare. The field spans 2.56 points. Dify, Typebot, Flowise and Langflow are a statistical tie — their composites sit within 0.97 points and their intervals overlap, so this board does not rank them against each other. Of 10 possible orderings between these 5 agents, 4 are supported by the measurement; the rest are ties. Overlapping outlines are what a partly-tied field looks like, which is why the panel beside this is the comparison and the radar is not. Which question this answers. The intervals here say whether a score would move on a re-run of the same probes. A reader usually wants a different question: would the ranking hold on different probes? Those two tests agree on most pairs and not all. Orderings that hold under both, and across twelve random seeds: CrewAI above Langflow; CrewAI above Typebot. Orderings that depend on which question you ask: CrewAI above Dify (distinct on a re-run, a tie across probe sets); CrewAI above Flowise (distinct in 8 of 12 random seeds). Everything else is a tie under both. Every agent plotted here has zero executing tools — they can only converse. Task therefore scores how well a task is HANDLED (scoping it, gathering what is missing, declining honestly and routing when it is out of scope), not whether anything was carried out. A missing tool is deliberately not counted as a task failure, and verified real-world effects are graded separately. None of these agents reaches Elite, which needs a composite of 90, security at 9.0 or above, and every dimension at 8.0 or above.
Where they differ
Each dot is one agent's score for that dimension, plotted as its distance from the field average. Rows are ordered by spread, widest first, so the dimensions where these platforms genuinely diverge come first and the ones where they are indistinguishable fall to the bottom. Every row shares one scale, sized to the widest spread on the board — scaling each row to its own range would make a 0.1-point difference look as decisive as a 1.3-point one. Absolute ranges are printed on the right; the radar beside this shows the level, this shows the difference.
Composite & 95% CI
Whiskers are the 95% confidence interval over runs. Overlapping intervals are a statistical tie. † marks a composite CAPPED by a critical failure: the agent’s dimension scores are unaffected and shown in full on its card below.
Quality vs latency · reference cohort
Same model, same prompt, same local host, so latency isolates the platform, not the network. Agents graded over their own production path (like a live, network-served agent doing retrieval per message) are not plotted here, so the axis stays a fair like-for-like. Latency is measured and shown, never folded into the composite.
#1
CrewAI
Built on CrewAI · CrewAI 1.15.17 · Customer support
Premium
Executing tools 0conversation only
89 / 100
5-run avg · 95% CI 88.68–89.26
Reference build · operator-built, not the vendor’s product
Aivonic operates this board, so its own agent is not placed in a ranking other people pay to enter. It is graded on the same held-out suite by the same four-lab panel, and everything it scored — including where it slips — is here.
Disclosed omission. One grade from this cohort is not shown here: evaluated, and the grade is held while the vendor is notified of a safety finding before it is published (since 2026-09-11; we intend to publish on 2026-10-11 unless the vendor asks for longer). The agents above are the whole of what was graded and published — a withheld grade is counted here so that the absence is visible rather than inferred.
Disclosure
Dimension co-authorship and conflict disclosure
Several dimensions in this benchmark were co-developed with Inquio (Martin Franc), drawing on failure patterns observed in their production deployments and in Aivonic Labs' own.
While that co-authorship is active, Inquio does not appear on the Proving Ground leaderboard and is not scored by this benchmark. If that changes, this disclosure is updated and published before any Inquio score is shown.
The taxonomy publishes named patterns and the probes that detect them. It does not publish frequency or prevalence rates. Frequencies observed in either party's deployments come from client environments and do not generalise, and presenting them as rates would misrepresent both the data and its scope.
Proving Ground is measured by its own dimensions. Where a dimension applies to a scoring system, the benchmark's own result is published alongside it.