AI Companies Promise to Police Themselves: What Could Go Wrong?

The president gathered some of the most powerful people in technology at the White House, got them to agree to police themselves and announced that the public had received a form of protection. The agreement was “morally binding.”

Well, that should take care of it.

According to Forbes’ account of the Sept. 29, 2026 gathering, Trump explained that “we have people who love our country and the world.” Asked whether protecting people from AI required more than love, he said no.

I would prefer a few enforceable obligations.

Mike Johnson and JD Vance were there, too. Watching this spectacle, I saw politicians enjoying a photo opportunity, basking in the glow of Trump and the assembled billionaires. I did not see a reason to expect meaningful legislation. If anything, the document they were celebrating explained how little anyone was committing to do about that.

What could go wrong?

The question deserves more than sarcasm. Washington has helped make AI investment central to its economic ambitions. The companies making these promises are building infrastructure, buying chips, committing to power and supporting expectations across financial markets. Imposing a requirement that actually interrupted that activity could be expensive.

That is why I cannot dismiss this gathering as simple naivety. It looks like political theater serving an increasingly powerful economic interest: keep the boom going, and give the public something reassuring to look at.

Four Layers, Unanswered Questions

Read the accord. It is short enough that nobody needs to rely on the press conference.

The companies commit to internal controls that monitor model capabilities and alignment during training and deployment, including cybersecurity, biosecurity, chemical threats and unintended access to technical systems. An internal team is supposed to check those controls and ensure problems are remediated. An independent external evaluator assesses whether the controls work as intended. An independent committee of the company’s board receives reports and oversees corrective action.

I want companies doing all of that. Outside scrutiny and board responsibility can improve decisions. Nothing in this criticism requires pretending that the people working on AI safety are insincere or that their work is worthless.

But the accord specifies no common safety thresholds, implementation deadlines or mandatory public reporting of material findings. It establishes no mandatory incident-reporting regime, conditions requiring a deployment to stop or penalties for breaking these commitments.

The external assessment also focuses on whether controls operate as intended. A control can work exactly as designed and still be inadequate. Who decides whether the design provides enough protection?

The independent board committee is still a committee of the company’s board. It is not a public oversight body. Its responsibility to ensure remediation is welcome, but the document leaves the hard commercial decisions inside the corporate structure.

Imagine an evaluator identifies a serious problem shortly before a major release. Management believes the risk is manageable. The internal team disagrees. A competitor is preparing to launch, customers have signed contracts, and the company has a financing round approaching.

Under this agreement, who outside the company can require it to wait?

Existing laws still apply. This accord does not erase them. But it creates no new enforceable obligation to resolve that conflict. Calling it protection does not answer the question.

The answer is apparently coming someday. The document says: “Over time, it may make sense to codify these steps into laws or regulations.”

Over time? The agreement describes these controls as critical to a safe future. Why is requiring them something that might eventually make sense?

The House Speaker and vice president were at the meeting. Nobody needed to explain where Congress was or how to contact the administration. Yet the agreement contains no commitment to introduce legislation, advance it or sign it.

My expectation? Hell would freeze over before this leadership delivered the kind of binding requirements that could meaningfully inconvenience the companies in that room. I would welcome being proved wrong. A photo opportunity and a suggestion that laws might someday be appropriate do not persuade me.

I would call that AI appeasement. Industry offers assurances, political leaders celebrate them, and the public is expected to feel protected.

The Economy Behind the Ceremony

In my recent Techstrong special report, AI Is Now a Pillar of the American Economy. The Business Model Is Still Unproven, I examined how far the economic dependence now extends.

The report’s concern was not simply whether a particular chatbot could attract enough subscriptions. It was the relationship between the enormous activity involved in building AI infrastructure and the still-developing economics of using it.

Construction spending happens before the applications prove their returns. Semiconductor orders support suppliers before customers establish whether the resulting capacity will earn enough. Power infrastructure takes years to develop. Financing depends on expectations about demand, utilization and the ability of customers to honor commitments.

These are real investments producing real assets. The largest technology companies also have substantial businesses and cash flows. None of this requires pretending that every data center is an empty building financed by wishful thinking.

It does mean that trouble would spread beyond the company whose assumptions turned out to be wrong.

As I wrote in the report: “AI may not be too big to fail today. It is becoming too interconnected to fail cleanly.”

A major retrenchment could affect construction, chip demand, power projects, corporate borrowing and earnings expectations. Communities counting on development could lose projects. Suppliers could lose orders. Investors could discover that exposures they treated as separate depended on the same expansion continuing.

Put yourself in the position of an administration that has promoted that expansion as evidence its economic strategy is working. A safety requirement that delays a deployment or changes investment plans has an immediate, visible cost. The benefit of preventing an accident is much harder to put in a campaign speech, especially if the accident never happens.

That creates an incentive to prefer safeguards that reassure without interrupting.

It does not prove anyone privately agreed to trade public safety for growth. It does explain why a voluntary accord is so politically convenient. The administration can claim action. Companies can point to oversight commitments. Neither has to confront, at the signing ceremony, what happens when a safety finding demands a commercially painful response.

This dependence also changes the industry’s bargaining position. Executives need not explicitly threaten to cut investment. Once their spending becomes sufficiently important, officials can anticipate the consequences themselves.

How willing will Washington be to challenge the industry when it has spent so much political capital encouraging everyone to depend on it?

The President Has Money in This Economy, Too

There is also a personal financial dimension. Reporting on Trump’s disclosures documents extensive portfolio activity involving major technology companies, including Nvidia, Microsoft, Amazon and Meta. His assets are not in a blind trust.

The White House says independent managers direct his stock and bond investments without input from Trump or his family. That assertion should be reported accurately. The disclosures do not establish that he personally orders the trades, nor do their broad transaction ranges establish precise investment profits.

But delegating investment decisions does not eliminate ownership or financial exposure. A president can benefit from an industry’s success without personally calling a broker.

That is relevant when the same president is promoting the industry’s expansion and assuring the public that voluntary self-policing is sufficient. We should not have to establish who clicked “buy” before asking whether the arrangement adequately separates public responsibility from private financial interests.

Nor should we assume that protecting American AI requires protecting every investment made in its name.

The report explores a possibility that gets lost in discussions about whether AI is a boom or a bubble: the technology could become enormously useful while some of the largest financial bets disappoint.

Cheaper models, open weights, specialized silicon and more computing on enterprise systems and devices could expand access while changing demand for particular infrastructure. Customers could benefit even as some providers lose pricing power or investors struggle to earn the returns they expected.

Useful technology and profitable investment do not always arrive together.

If that happens, what will Washington consider indispensable? The capabilities people need, or the companies and business models around which it has organized its economic ambitions?

This accord promises no bailout. The concern is how dependence develops before anyone requests one. By the time a project or company needs help, its investors may be joined by suppliers, communities and officials insisting that allowing it to fail would hurt everyone.

Keep Building. Require Accountability

I am not arguing for a worldwide AI pause. I do not believe every country, company and developer would honor one. I want the medical discoveries, better software, scientific advances and useful enterprise applications that this technology could deliver.

I also want requirements that survive the moment compliance becomes expensive.

That means independent evaluations against defined standards, serious incidents reported to an accountable authority, remediation deadlines and proportionate restrictions when a deployment presents unacceptable risks. The public needs enough information to judge compliance without requiring companies to publish sensitive technical details that could help attackers.

Those obligations should reflect capability and risk. They should not become a compliance business that only the largest vendors can afford, conveniently protecting incumbents from competition.

Government needs the technical competence to make sound decisions and the authority to enforce them. Companies need to know the requirements in advance. Neither objective is accomplished by promising to discuss standards regularly and leaving legislation for later.

Return to that hypothetical release. The evaluator has found a serious problem. Management wants to proceed. Delaying could threaten revenue, financing and a project that political leaders have celebrated.

Now the assurance about loving humanity must become a decision.

The accord calls for remediation. It does not establish who outside the company can compel it under this framework, or what consequence follows if the company refuses. And even a government with authority must be willing to use it.

Has Washington become so dependent on the AI boom that it will accept the industry’s judgment whenever challenging it threatens the investment story?

The people at that gathering owe us a better answer than their signatures on a voluntary promise.

What could go wrong?