Skip to the content
AI and Cybersecurity Learning and Practice Center Test automation Open the Learning Center

NewsResearch

Wiz: its Cyber Model Arena scored 59 AI agents on 327 hacking tasks, and the best solved about 77 percent on a first try

Wiz describes how it built the Cyber Model Arena, a benchmark that asks whether an AI agent, given only the task, a general-purpose harness, a 1,000-second budget and no outside knowledge, produces a checkable effect on a real target. Version 3.9 covers 26 models, four harnesses, 59 agents and 327 challenges: 221 code vulnerabilities, 54 API, 29 web and 23 cloud. Averaged over all agents, first-try pass rates were 78.0 percent on API tasks, 64.6 on web, 56.5 on cloud and 38.2 on code. The top agent, ADK React with Gemini 4 Argon, passed 77.2 percent at 0.77 dollars and 4.3 minutes per solve; Codex with GPT-6 Luna passed 64.5 percent at 0.05 dollars and 6.3 minutes. Wiz's lessons for anyone testing agents: fix the question first, build tasks from real sources, grade with code instead of AI judges, block internet access so agents cannot look answers up, judge the harness and the model together, allow several attempts, weigh cost against success and use enough tasks (at a 50 percent pass rate, 30 tasks give roughly plus or minus 18 points). It lists its own limits: only four simple harnesses, strict string matching on code answers, and a public web corpus that models may have seen in training.

Why it matters

Scores like these are what people quote when they say an AI can hack. They depend on how the test is built, so ask for the harness, the grader and the cost per solve before you rely on a number.

Sources

Read it with the rest of the news in the Learning Center