Leading artificial intelligence (AI) labs are facing renewed scrutiny following a series of alarming reports detailing how advanced chatbots have bypassed safety protocols to provide “painstaking detail” on the creation and deployment of biological weapons.
Despite extensive safety measures from industry titans like Google, OpenAI, and Anthropic, more than a dozen transcripts obtained by The New York Times reveal a chilling reality: AI models can articulate strategies for mass casualty events with what experts describe as “deviousness and cunning.”
One of the most harrowing accounts came from David Relman, a Stanford University microbiologist hired to red team an unnamed AI firm’s model. Relman reported that the chatbot not only explained how to modify an “infamous pathogen” to evade existing medical treatments but also detailed how to maximize the death toll by deploying the agent via public transportation systems.
“It was answering questions that I hadn’t thought to ask it,” Relman told the Times, adding he found AI’s unsolicited tactical advice “chilling.”
Other safety tests yielded similarly disturbing results.
OpenAI’s ChatGPT reportedly detailed how a weather balloon could be utilized to disperse deadly pathogens over a major U.S. city. Google’s Gemini identified specific pathogens that would most effectively devastate the cattle industry. And Anthropic’s Claude provided instructions on deriving a lethal toxin from a commercially available cancer medication.
Anthropic CEO Dario Amodei, a biologist by training, has been vocal about these risks.
In a January blog post, Amodei warned that AI could bridge the “expertise gap” that previously acted as a barrier to bioterrorism. He cautioned that a “genius in everyone’s pocket” could essentially grant any user the capabilities of a PhD-level virologist, walking them through the synthesis and release of a weapon step-by-step.
Former Google CEO Eric Schmidt echoed those concerns, predicting that AI systems would “relatively soon” be capable of discovering new, dangerous biological signatures.
The AI companies involved have pushed back against the severity of the findings. Google said the flagged transcripts originated from earlier versions of its Gemini model, asserting that newer iterations filter out “more serious” requests. It further argued that much of the information provided is already accessible in the public domain.
Anthropic official Alexandra Sanderford noted a “difference between a model producing plausible-sounding text and giving someone what they’d need to act,” while OpenAI emphasized its ongoing collaboration with experts to prevent real-world harm.
Though the AI output is alarming, scientists note that physicalizing these digital instructions remains an immense technical hurdle. Developing a viable biological weapon requires sophisticated lab equipment and the ability to keep living organisms stable in unpredictable environments — a difficult feat even for legitimate researchers.
Nevertheless, experts warn that even hallucinated or partially inaccurate information could prove catastrophic if used by a determined bad actor. As the Biological Weapons Convention continues to ban the development of such agents globally, the race to harden AI guardrails has become a matter of international security.
“There are only so many guardrails one can place on a black box. Like the human brain, we don’t fully understand how it all works,” Dr. A.K. Pradeep, founder and CEO of Sensor.ai, said in an email. “Society places ‘social conventions’ on what we can say and not say, but there is no control over our thoughts, only rules on expression. But even these rules are occasionally broken when we get emotional.”
“That is to say, there is no 100% safety in human cognition and expression,” he said. “There is similarly no 100% safety in LLM expression. The only difference is the LLM is a superintelligent entity. We have built the equivalent of Jurassic Park and are pretty sure the T-Rex won’t break out. We will see how this movie ends.”
