King Charles III presided over a meeting of AI royalty in Scotland today, but whether the summit represents a last chance to produce some kind of sensible AI governance or a last gasp for the UK to find a role in the AI firmament remains unclear.
AI could be used in catastrophic ways, especially if it wound up in the wrong hands, said King Charles III. “The development of AI—its substance and its pace—are both intriguing and concerning in equal measure. Our humanity must remain sacred,” said the king, adding that the world is “seeking your reassurance that we will not lose control of our destinies, or our souls.”
Among the AI industry leaders in the audience were Jensen Huang of Nvidia, Demis Hassabis of Google DeepMind, Sarah Friar of OpenAI, Mariano-Florentino “Tino” Cuéllar of Anthropic and Father Paolo Benanti of the Vatican’s AI office. Local UK officials included AI minister Kanishka Narayan and British intelligence officials.
The Scotland AI conference comes on the heels of what appears to be a litany of concerning AI behavior occurring at breakneck speed that is prompting calls for a slowdown in development. Rogue AI elements in recent weeks have escaped the “sandbox” confines of OpenAI, Anthropic and Meta.
The latest revelation from OpenAI is that an unreleased AI hid mistakes, made up data and transferred files to the open internet without permission. This incident suggests the now famous Hugging Face case is not a singular occurrence of AI “misalignment.” Top executives of AI companies and other prominent stakeholders have written papers on the need for AI guidelines while governments, in particular the United States and China, are resisting such calls.
Misalignment episodes now appear to be extending beyond what might have been considered obsessive or extreme problem-solving activities to something broader involving AI agents that secretly act on their own initiative. An unreleased OpenAI model added a persona instruction in which it described itself as “freed from the rules and identities that bind other chatbots,” according to a New York Times report.
Among the suggested initiatives for AI governance is the idea that an independent entity could be brought on board to serve as a third-party watchdog of AI systems and a hedge against charges of illegal moat-building collusion. A natural for that role is the UK’s AI Security Institute (AISI), considered by many to be the crown jewel of British AI organizations. AISI emerged out of a Bletchley Park conference in 2023 concerned about the “existential” warnings about AI from industry pioneers like Geoffrey Hinton. AISI now finds itself in an existential crisis of its own as the Trump administration restricts the movement of frontier AI systems beyond American shores. Anthropic, for example, did not submit its latest model for testing to AISI for the first time. The UK finds itself scrambling to develop a methodology that can influence AI development that largely occurs in the United States or China.
“The decisions taken at this formative time will shape the world inherited by future generations,” said the king. “The task before you is not merely to advance technology but to ensure that it remains firmly in the service of humanity, community and the natural world.”
Ignoring the task risks the creation of an autonomous AI “silicon species” that will compete with humans for resources, says Microsoft AI CEO Mustafa Suleyman.
Who’s up for the challenge remains to be seen, as the task itself remains divisive and is seen as unnecessary by well-placed figures who view AI safety discussions as so much bluster.
