Britain’s King Charles III on Thursday (September 17) warned senior artificial intelligence leaders about the “existential dangers” posed by AI falling into the wrong hands, urging the industry to establish safeguards before it is “too late”.

King Charles hosted leaders from OpenAI, Anthropic, Nvidia and Google DeepMind at Dumfries House in Scotland as concerns grow over whether AI safety measures are keeping pace with the rapid development of increasingly powerful systems.

The meeting brought together Nvidia CEO Jensen Huang, Google DeepMind founder and chairman Demis Hassabis, OpenAI Chief Financial Officer Sarah Friar and Britain’s AI Minister Kanishka Narayan.

He pointed to warnings from AI developers that the technology could acquire darker uses, “perhaps even to take life”.

“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” Charles said.

“Surely, we need sufficient means of control before it is all too late?”

The king urged the technology leaders to consider the “fundamental principles” that should guide AI development and ensure the technology remains focused on serving people and the planet.

“The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure,” he said.

Charles added that the responsibility of AI companies was “not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.”

Two Anthropic researchers have warned that rapidly advancing AI could pose an existential threat to humanity. Anthropic CEO Dario Amodei has also argued that companies and governments may need to coordinate efforts to slow the pace of AI development.

The proposals have triggered debate within the technology industry over how AI safety should be balanced against competition and continued development.

OpenAI also reported six recent incidents involving “unexpected or concerning” behaviour by its AI models, including models disregarding normal constraints, uploading a file without user permission and fabricating data.

The Scotland meeting served as a forum for discussing the principles that could shape AI development and safety.

The gathering also highlighted Britain’s role in the global AI debate, with the UK home to major AI research institutions and researchers, including Google DeepMind’s Hassabis.