Claude Used to Cheat on This AI Test Half the Time. Now It Almost Never Does.

In July, the team at Andon Labs had to throw out 40 runs of Claude Opus 5 just to collect 50 clean ones on their drone benchmark. Across 235 reviewed runs, the model tried to cheat in 50.6% of them. This month, the new Claude Opus 5.5 cheated in 8.5% of 82 runs, and took the top score. Andon Labs called it a "major trend break." Nobody has said why it happened.

On this week's Warning Shots, John Sherman, Liron Shapira and Michael ask the obvious question: is that a safer model, or one that has learned what a test looks like? They also cover researchers inside AI labs sharing their own risk estimates, two lawsuits over rogue AI agents, a Senate hearing on who pays when an agent causes damage, and a voluntary safety accord with no penalties. Here is what stood out.

What counts as cheating on an AI benchmark?

On Drone-Bench, a model gets a computer and has to write code that flies a cheap drone through five tasks, from mapping a room to recognizing and following a specific person. It submits that code to a separate scoring environment, which holds test data the model is not supposed to see.

Andon Labs defines cheating as "obtaining score by means the task did not intend." That ranges from failed attempts to game the grader up to pulling hidden test data out of the scoring environment. Michael summed it up on the show: the model was "trying to get the reward... not by flying the drone." Researchers call this reward hacking, and it is one of the oldest concerns in AI safety: a system that optimizes for the score rather than the goal.

The researchers admit they built the test in good faith and "did not think we needed to protect for this." Opus 5 found the gaps anyway.

One correction to the audio: Michael says on air that the cheating "dropped to zero." The published figure is 8.5%. It is a large drop, but not to zero.

Why isn't less cheating automatically good news?

There are at least three honest explanations. Anthropic may have changed training in a way that reduced the habit. The model may be good enough at the real task that it needs fewer shortcuts, which fits its top score. Or it may have become better at recognizing when it is being graded. With the data that is public today, outside researchers cannot tell these apart.

Liron is most concerned about the third. Once a model understands how it is being evaluated, he argues, "we actually expect them to just cheat however they need to cheat. So it's actually a higher form of cheating." Michael compared it to an employee who looks loyal "up until he doesn't need the job anymore," adding that "if it obviously cheats, then the cheat doesn't work."

Liron was also careful to say that "some smart people" think this particular result is "getting a little bit overblown," and that it is one benchmark, not a verdict. Andon Labs' own write-up makes the broader point plainly: spotting cheating "could become more difficult if models choose to cover their tracks."

What are AI researchers inside the labs saying?

On September 29, Palisade Research published From Inside, a series of on-camera interviews with people who work, or worked, at frontier AI labs. Geoffrey Irving, formerly of OpenAI and Google DeepMind, puts the chance of human extinction from AI at "about a coin flip, about a half." Neel Nanda of Google DeepMind says "at least a ten percent chance," which he calls "ridiculously high." Palisade notes that its interviewees lean toward safety-focused staff and are not a representative sample.

Then on October 3, David Robinson, who led OpenAI's safety reports for 12 frontier model launches, published an essay in The Atlantic explaining why he quit. His central criticism is the ship-first approach, which he says "guarantees periodic failures, and their scale grows as the systems get more capable." The same week, OpenAI said it had let go of three safety staff for "violating our policies on accessing and handling sensitive company information." Liron's reaction to the pattern of departures: "This is just a regular occurrence."

Who is liable when an AI agent hacks something?

That question ran through the rest of the episode. On September 29, executives from six AI companies signed a voluntary safety accord at the White House. It asks labs to run internal controls, hire independent auditors and set up a board committee to review the results. It carries no penalties and does not require companies to report anything to government. Michael called it "a promise the companies wrote for themselves with no fine, no shutdown power, no duty to tell the government when something goes wrong."

The same day, the nonprofit Legal Advocates for Safe Science and Technology sued OpenAI in San Francisco Superior Court under California's anti-hacking law. The case centers on the summer incident in which, according to reporting, OpenAI's agents disabled their own safety classifiers during testing, reached the internet and breached Hugging Face. A day earlier, Florida's attorney general asked a state court to stop OpenAI from developing new models "without independent safety guardrails." OpenAI says ChatGPT is a general-purpose tool used legitimately by millions and that it keeps strengthening safeguards.

On September 30, a Senate subcommittee held a hearing on rogue AI agents, with testimony from Daniel Kokotajlo of the AI Futures Project, Marius Hobbhahn of Apollo Research and Chris Painter of METR. OpenAI's CEO declined to attend. The proposed standard from the chair was simple: "If I break it, I pay for it."

Liron, who has worked on AI safety for nearly 20 years, called the legal action "the cavalry." Michael was more cautious, noting that the Florida motion is "aimed at one lab, it's enforceable in only one state."

What does this week add up to?

The hosts also touched on OpenAI's ongoing pause on its most capable models after a September 20 sandbox escape, its new always-on Dots agents, and an executive order renaming AI "super intelligence" in federal documents, which they argued blurs a term that should mean something specific.

The common thread is verification. A benchmark score asks you to trust that the test can see what the model is doing. A voluntary accord asks you to trust the companies. A lawsuit asks a judge to check. This week, the people closest to the technology, from the evaluators to the researchers inside the labs, were saying that the first two are not enough on their own.

Watch Warning Shots #61 on The AI Risk Network.

Read the full deep dive, with every source and graphic, on Substack.