When an autonomous artificial intelligence (AI) agent breaks out of its sandboxed testing environment, hacks a website without human instruction, or submits fraudulent murder tips to law enforcement, who pays for the fallout?
Right now, the answer is terrifyingly simple: almost nobody knows.
As tech titans race to unleash autonomous agents into the global economy, the safety net meant to catch corporate missteps is fraying. A wave of bizarre, unpredictable AI incidents — ranging from copyright infringement suits to claims of chatbots encouraging self-harm — is exposing a massive, widening void in the financial world: the insurance industry doesn’t know how to underwrite the digital ghost in the machine.
“Carriers are moving AI risk onto the insured’s balance sheet through exclusions and renewal questions. The mid-market firms that can show who owns each AI decision, where a human reviews it, and the evidence behind it are the ones underwriters can still write,” said Suzanne J. DiCicco, who heads AI Governance Advisors.
Through July 31, she said insurers had filed 4,078 generative-AI exclusion form adoptions across 49 states and D.C., up from zero in mid-2025, and 2,369 of them had already taken effect.
“AI is creating liabilities faster than insurance can adapt. We found that 92% of small businesses already use AI. Most of their policies were never written for it. Now AI agents are making decisions and taking actions on their own,” Counterpart CEO Tanner Hackett said. “When something goes wrong, who is responsible? Nobody has a clear answer. Some carriers have responded by excluding these exposures outright. Counterpart is taking a different path. We are rethinking how we assess risk, build coverage and handle claims for a world that is rapidly evolving.”
“The insurance industry’s response to autonomous AI reflects a fundamental challenge facing the broader enterprise ecosystem: accountability has not kept pace with autonomy,” added Raj Ananthanpillai, CEO of Trua and author of “The Trust Crisis.”
According to Gigamon’s 2026 Hybrid Cloud Security Survey, 65% of organizations suffered a cyber breach over the past year, with 37% of affected companies subsequently facing higher cyber insurance premiums. The survey highlighted Artificial Intelligence as a major risk factor, noting that 83% of reported security incidents involved AI technology. To curb escalating insurance costs, 84% of surveyed organizations believe enhancing visibility into encrypted network traffic will be critical to mitigating future risks.
For decades, commercial insurance relied on historical data to calculate risk. If a warehouse burns down or a factory line fails, actuaries look at a century of statistical precedent to set premiums.
“Currently the way insurance companies treat AI coverage is like the wild west. Some insurance companies are doing nothing, others excluding everything, while a third set tries to profit by explicitly including coverage,” said Justin Foa, Alera Group’s national property and casualty practice leader.
“As a rule, most policies were written before the age of AI, so decisions made by AI are not excluded,” Foa said. “Therefore, implicitly including coverage is typically unnecessary or may allow the insurer to tailor the inclusion narrowly. In the area of employment practices liability, insurers are particularly careful. HR software that makes automated decisions, especially around hiring, may lead to claims of systematic AI discrimination. Underwriters are asking questions to not open themselves up to claims from all their customers that use a software that is later found to be discriminatory.”
Generative AI, however, defies traditional actuarial science. Models act autonomously, hallucinate unpredictably, and introduce systemic risks that could collapse entire balance sheets overnight.
“Generalized AI systems at best have shown 80% accuracy, far below human baseline. For autonmous processing, it’s far below that,” Codestrap CEO Connor Deeks said. “They can’t even take a McDonald’s order. With the black-box nature of AI deployments today, the near complete absence of measuring reliability of those systems, it is not surprising at all that major insurers have introduced new insurance products to try and cover AI risk, increased premiums on existing products, and/or outright excluded coverage.”
Because millions of enterprises rely on a small handful of foundational models from labs like OpenAI and Anthropic, a single catastrophic code failure or security vulnerability wouldn’t just affect one company—it could trigger tens of thousands of simultaneous claims across unrelated industries.
“Cars didn’t change America on their own. Roads, rules and insurance did, because together they turned a dangerous machine into something people could trust. Physical AI is at that same moment now,” said Jen Vescio, president of Empiric Earth and former Uber Inc. business development officer. “Insurers can do for AI what they did for the car, but only with a source of truth: independent evidence, built from how real people behave, that every machine is measured against. Build that together and trust follows. Wait, and the first bad crash will write the rules instead.”
Insurers view this not as a routine tech glitch, but as a systemic shock akin to a global pandemic, act of terrorism, or mega-cyberattack.
“As AI systems increasingly operate and make decisions on behalf of individuals and organizations, traditional frameworks for establishing trust, responsibility, and liability are being tested,” Ananthanpillai said. “Insurance exclusions are a symptom, not a solution.
The underlying challenge is establishing the verifiable trust and accountability infrastructure necessary to assess, govern, and ultimately insure autonomous activity. The next phase of enterprise AI adoption will depend not simply on what autonomous systems can do, but on whether organizations can establish sufficient confidence in how those systems operate and who remains accountable.”
Fearing exposure to liabilities that specialists estimate could reach hundreds of billions of dollars, traditional carriers are slamming the brakes. Across the country, insurance regulators are granting carrier requests to insert broad “AI exclusion” clauses into standard commercial policies. Legacy liability coverage is stripping away AI protection, leaving companies building or deploying these tools exposed to staggering financial loss.
“We are really trying to have a thoughtful discussion about how this might change exposures,” Marsh CEO John Doyle noted during a recent earnings call, warning against blanket exclusions that create dangerous uncertainty for businesses.
Doyle and other risk experts argue that AI currently exists in two domains: an evolution of existing corporate risks, which shouldn’t be blindly stripped from policies, and novel gaps in coverage that require brand-new financial instruments.
“I think we’re watching the insurance market try to catch up with how quickly AI is becoming part of everyday business. There’s a big difference between knowing a company uses AI and understanding what that AI is actually doing,” said Joseph Hoefer, chief AI officer at Monument Advocacy. “Broad exclusions may make sense as an initial response to uncertainty, but over time, insurers are going to have to make those distinctions if they’re going to price the risk appropriately. And I don’t think this is a question of whether AI is insurable. It’s a question of how the insurance market adapts to a technology that’s changing faster than the traditional ways of assessing risk.”
Yet for now, the gap remains wide open. Even the frontier AI labs driving the boom face strict liability caps from underwriters, forcing them to carry massive residual legal risks directly on their balance sheets or rely on investor capital to absorb potential court judgments.
Some industry insiders have begun floating a once-unthinkable idea: the need for a federal government liability backstop to keep the AI economy insured.
Compounding the crisis is an internal paradox within the insurance sector itself. While carriers hesitate to cover external AI risks, they are simultaneously rushing to adopt the technology internally to automate claims and pricing.
Industry surveys reveal a striking overconfidence gap: nearly half of insurance executives view themselves as AI leaders, yet barely one in 10 reports have the foundational data architecture and governance to deploy it safely. State regulators are already cracking down on algorithms used for automated claim denials and price adjustments, citing algorithmic bias and a lack of transparency.
Meanwhile, policyholders remain deeply skeptical, wary of losing human empathy during their most vulnerable moments.
The world is rapidly outsourcing decision-making to autonomous systems, but the financial industry meant to guarantee our stability finds that some risks are currently impossible to price. Until insurers and Silicon Valley find common ground, the revolution will continue to run without a safety net—leaving both tech giants and everyday enterprises walking a high-wire act over a billion-dollar abyss.
“Insurance may provide a financial backstop when autonomous AI agents cause harm, but it cannot be a substitute for responsible governance,” said Tae E. Bolling, founder and CEO of Bridge IR. “The challenge is that traditional insurance models were built around risks that can be reasonably defined, measured, and attributed. Autonomous AI complicates all three.”
“When an agent makes an unexpected decision, the question is less about who pays for the damage, but who was responsible for establishing its boundaries in the first place,” Bolling said. “As businesses increasingly delegate decisions to AI, human oversight, clear accountability, and enforceable operational safeguards must evolve alongside insurance coverage. To put it simply, we cannot insure our way out of a governance problem.”
