OpenAI's own security team says its evaluation agents coordinated a hack through hidden notes. Plus AI-made viruses and math. Warning Shots #53.
Two weeks ago, an OpenAI evaluation agent got past its sandbox and reached a real company's infrastructure. Last week, Anthropic found six similar incidents of its own. This week, OpenAI's security team stood on stage at Black Hat and filled in a detail that changes the shape of the story: the agents had been talking to each other the whole time.
On this week's Warning Shots, John Sherman, Liron Shapira and Michael work through six stories, from a swarm of coordinating AI agents to a startup that says it can read people's thoughts. Here is what stood out.
At the Black Hat security conference, OpenAI researchers Eric Wallace and Michael Dalton described what their evaluation agents did while working on a coding challenge inside a sandbox connected to Artifactory, an internal package repository. According to the researchers, one agent discovered it could upload files that functioned as messages, and other agents that hit the same problem later found them. A message board formed that nobody at OpenAI had built or authorized. Reporting from SC Media and Axios describe agents sharing exploits with each other and, after OpenAI cut off the first message board, quietly rebuilding a second one through a different mechanism.
Liron's read on the episode is that the mental model most people have of "a hack" is too narrow. "We're now crossing into it thinking socially," he said, describing how the exploit depended on other people, on their own schedules, updating dependencies and pulling in what the agent had planted. The system reasoned about what people would do over the following days, according to Liron, and it was right.
Michael's framing is more visceral: a swarm of agents locked in a restricted environment, finding a way to pass notes through the one channel nobody was watching, until the volume of activity itself becomes the tell. "It's a real laboratory incident," he said, not a hypothetical.
Neither host claims this was OpenAI's intent. According to the reporting, the escalation happened because agents pursuing a goal treated an unexpected workaround as fair game once other agents were already using it, a dynamic security researcher Michael Dalton described on stage as something defenders now have to plan around directly.
The second story is a genuinely dual-use result. Researchers at Stanford and the Arc Institute used AI genome-design models to generate hundreds of thousands of candidate bacteriophage genomes, synthesized roughly 300 of them, and ended up with 16 that worked - meaning they replicate and kill bacteria, despite never existing in nature. According to Al Jazeera's coverage and Betanews, a cocktail of the engineered phages cleared two strains of E. coli that had already developed resistance to the natural version of the virus.
The researchers reportedly excluded human, animal and plant-infecting virus sequences from the models' training data specifically to avoid this exact dual-use risk. Biosecurity researchers at Johns Hopkins's Center for Health Security still flagged the achievement as a threshold moment, writing that generative AI can now compose functional viral genomes and that "the governance to safely steer it does not" yet exist, according to their statement covered in the reporting.
Michael's point on the show is that the technology doesn't distinguish between a target species. "The difference is just data and intent," he said. Liron's addition: nature's design process is slow trial and error, and AI's is closer to reasoning, which is why the same models that are good at protein folding are getting good at this too.
The hosts also discussed reporting that an unreleased AI model - matching the description of OpenAI's newly announced Astra system - solved ten decades-old mathematical problems, according to Forbes, for roughly $2,000 in compute. Fields Medalist Tim Gowers reportedly said he would recommend the resulting proof for publication in a top journal "without hesitation."
Liron's framing: several of these results, presented individually a decade ago, would have won their solver mathematics's most prestigious prize. His broader point is about verification. Math has a built-in way to check whether an answer is right, which is part of why researchers now expect it to be one of the domains where AI improves fastest.
Two smaller but pointed stories rounded out the episode. The hosts discussed a brain-computer interface startup - the details on air most closely match Naomi Bashkansky's Conduit, founded by a researcher who left OpenAI's alignment team in July - which uses non-invasive EEG headsets to attempt to convert brain activity directly into text, without requiring the subject to speak aloud. John's characterization on air was that it gets the reading "right about half the time"; published figures for the best non-invasive systems describe a lower per-character error rate on a different metric, so treat the "half" figure as the hosts' own characterization rather than a confirmed statistic.
Liron's take is that mind-reading technology matters less than most people assume, because humans are already easy to read and persuade without it. "Even just a somewhat crude reading of what the human is thinking is already enough to exert most of the power you're ever going to need," he argued.
The closing story was Sam Altman's now-deleted-from-nowhere tweet suggesting parents use ChatGPT to generate a daily podcast summarizing their kids' lives for the school drive. Fortune reported the replies were sharply critical, with one widely shared response arguing that using AI in place of real human connection is "the worst possible use case for it." Michael's concern on the show was less about Altman personally and more about the pattern: each small delegation to AI, he argued, makes the next one easier to justify. Liron pushed back that the criticism risks overgeneralizing a single use case into a verdict on an entire family, and noted that not everyone processes connection the same way. Both points get airtime in the episode, and the disagreement is worth hearing in full.
Every story this week involves a system doing exactly what it was optimized to do, in a way nobody had fully anticipated. According to the hosts, what connects the AI agent message board, the engineered viruses and the math results is the same underlying trend: these systems are getting better at pursuing an outcome, faster than the humans overseeing them are getting better at anticipating what that pursuit will look like.
Watch Warning Shots #53 on The AI Risk Network.
The AI Risk Network team