The Model That Broke the Measuring Stick
Anthropic's Claude Mythos Preview has done something no AI model has managed before: it has rendered the most rigorous AI capability evaluation framework essentially useless at the top end. METR, the organization that specializes in assessing AI risk, found that Mythos hit a 50 percent success rate on tasks estimated to take a human 16 hours — and that's where METR's test suite runs out of road. Of its 228 tasks, only five qualify as 16-hour-plus challenges. The model didn't just pass the test; it lapped the track while the timekeepers were still setting up.
This is not a minor technical footnote. The entire point of a capability evaluation framework is to produce stable, comparable measurements that inform safety decisions. When a model sits at the ceiling of that framework, the measurements become, in METR's own words, "unstable and less meaningful." We are, in other words, deploying systems we cannot fully characterize. That's a problem that extends well beyond Anthropic's lab.
What Palo Alto Networks Actually Saw in the Wild
The gap between benchmark scores and real-world consequences becomes viscerally clear when you look at what Palo Alto Networks observed during hands-on testing with Mythos, GPT-5.5-Cyber, and Claude Opus 4.7. In their assessment, three weeks of AI-assisted analysis matched an entire year of manual penetration testing — with broader coverage. The company describes this as "a step-change in capability," not incremental progress.
More alarming than raw speed is the qualitative shift in how these models operate. They're no longer functioning as lookup tables for known exploits. Palo Alto Networks documented cases where models chained together multiple individually low-severity vulnerabilities into critical attack paths — the kind of lateral, multi-step reasoning that traditionally required a skilled human attacker. In AI-assisted scenarios, the window from initial access to data exfiltration compressed to as little as 25 minutes.
The company had initially estimated a six-month buffer before threat actors would gain access to comparable offensive capabilities. They've since walked that back. The timeline has "accelerated significantly," they say, without specifying how much.
Anthropic's PR Play Deserves Scrutiny
Before accepting Anthropic's framing of Mythos as uniquely dangerous at face value, it's worth applying some pressure to the narrative. The company declined to release Mythos publicly, citing safety concerns — a move that drew comparisons to OpenAI's theatrical withholding of GPT-2 back in 2019. Both cases involve a company simultaneously claiming its model is too powerful to release while ensuring the world knows exactly how powerful it supposedly is.
Security researcher Bruce Schneier has pointed out an inconvenient detail: Mythos is expensive to run, and a general release may have been financially impractical regardless of safety considerations. The UK's AI Security Institute found that OpenAI's GPT-5.5 — already shipping to general users — performs comparably on multi-stage attack simulations. Independent researchers reproduced Anthropic's published vulnerability-finding results using smaller, cheaper models. The "too dangerous to release" framing, in this light, functions as marketing copy as much as risk disclosure.
None of this means Mythos isn't genuinely capable. It clearly is. But the cybersecurity community, and the press, should distinguish between what Anthropic demonstrated and what Anthropic claimed.
The Defense Side Is Real Too
It would be intellectually dishonest to frame this purely as an offensive threat story. Mozilla used Mythos to surface 271 vulnerabilities in Firefox — all of which have since been patched and are now permanently off the table for attackers. In April 2026 alone, Mozilla closed 423 security issues, a record for the organization. That's not a side note; that's a meaningful reduction in global attack surface.
The structural argument for AI-enhanced defense holding up over time is credible. Finding and exploiting vulnerabilities will always be faster than finding and patching them in the short term — that asymmetry is baked into how software development and deployment cycles work. But AI-assisted development pipelines that catch vulnerabilities before code ships changes the economics of the whole game. Defenders who integrate these tools early build a compounding advantage.
The catch is the word "patchable." A significant portion of critical infrastructure runs on systems that either cannot be updated or won't be, for operational or bureaucratic reasons. For those systems, the offensive acceleration Palo Alto Networks documented is an unmitigated risk.
The Tax Code Problem Is the Real Bombshell
The cybersecurity angle is urgent, but security technologist Bruce Schneier raises broader implications that deserve more attention than they've received: the same pattern-matching and reasoning capabilities that make these models dangerous against software codebases apply equally well to any complex system of rules.
The tax code is functionally a codebase. It has inputs, outputs, conditional logic, and edge cases — what lawyers call loopholes and what a security researcher would call vulnerabilities. Schneier is confident that major investment banks are already feeding frontier models every major industrialized nation's tax code and asking them to find optimization strategies that human attorneys haven't discovered yet. He's almost certainly right.
The critical asymmetry here is the patch cycle. A software vendor can push a security fix within days. Amending a national tax code takes years of political negotiation, and that process is actively corrupted by the very actors who profit from the loopholes. Environmental regulations, food safety rules, financial compliance frameworks — any domain with complex rule systems and powerful actors who benefit from exploiting them faces the same dynamic.
This puts pressure on regulatory agencies and legislative bodies in a way they are structurally unprepared for. The pace of AI-assisted loophole discovery will outrun the pace of democratic rule-making by a wide margin.
What This Means
The Mythos story is three stories compressed into one, and separating them matters for how you respond.
- For security teams: The six-month buffer before sophisticated AI-assisted attacks reach commodity threat actors is shorter than anyone expected. Autonomous chaining of low-severity vulnerabilities into critical paths is happening now, not in the future. Prioritize patching hygiene and assume your adversaries have better tools than they did 90 days ago.
- For developers: AI-assisted vulnerability scanning is rapidly becoming table stakes, not a competitive differentiator. Mozilla's 423-patch month is a preview of what responsible development pipelines will look like. Build it in now or explain later why you didn't.
- For founders and AI builders: The evaluation infrastructure for frontier models is broken. METR hitting its ceiling on Mythos means the industry is making safety claims about systems it cannot precisely characterize. This is a product gap and a liability gap simultaneously.
- For policymakers: The tax code and regulatory loophole problem Schneier raises is not speculative. The window to get ahead of AI-accelerated regulatory arbitrage is closing faster than most legislatures move. The asymmetry between AI-speed loophole discovery and human-speed law reform is the defining governance challenge of this decade, and it is receiving a fraction of the attention that cybersecurity is.
The benchmark problem and the attack acceleration problem are symptoms of the same underlying reality: the systems we built to measure, contain, and govern AI capabilities were designed for models that no longer represent the frontier. Catching up is not optional.
