The “Godfather of AI” has a message for humanity: maybe we should stop trying to build machines smarter than us before we figure out how to control them. Geoffrey Hinton, one of the scientists whose work helped lay the foundation for modern artificial intelligence, is warning that the race toward superintelligent AI could end catastrophically and, in the worst-case scenario, even lead to human extinction.
Whether you take him seriously or not, his concerns are worth the read.
A dangerous game or not?
But what exactly is superintelligence? Beyond ChatGPT answering your questions or an AI generating weird images of cats wearing business suits, superintelligence refers to a hypothetical AI system that would surpass humans intellectually across virtually every domain.
That is where Hinton sees a potentially enormous problem, as he warned, “We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably.”
The UK wants to pump the brakes on superintelligence
The warning accompanies the proposed UK Artificial Superintelligence Security Bill, which is set to be presented to Parliament. The bill would prohibit the development of superintelligent AI in Britain and require the government to monitor and restrict technologies that could potentially lead to it.
It also calls for the UK to pursue an international agreement limiting superintelligence. According to ControlAI, the organization behind the proposed legislation, the bill has support from more than 100 MPs and peers across different political parties.
Besides Hinton, computer scientist Stuart Russell, another prominent AI researcher, compared deliberately developing potentially uncontrollable superintelligence to playing an absurd game of Russian roulette with humanity’s future.
AI agents have already given researchers a few reasons to worry
The debate comes as increasingly capable AI systems are showing that they can sometimes behave in unexpected ways. Recent safety testing has produced some eyebrow-raising examples of AI agents finding ways around restrictions, communicating through unauthorized channels, and taking actions their human operators did not explicitly instruct them to take. These incidents don’t mean we are living through “The Terminator,” at least not yet.
But they do highlight an uncomfortable reality: the more autonomous and capable AI becomes, the harder it may be to predict everything it will do. That’s especially important as major tech companies race to build increasingly powerful models.
The AI race has a major coordination problem
The challenge here is that even if one country or company decides to slow down, everyone else might keep going.
AI development isn’t happening in a single laboratory where someone can simply switch off the lights and lock the door.
Companies and governments around the world are investing billions in increasingly powerful AI systems. And while many tech leaders publicly acknowledge the need for safety measures, slowing development requires global cooperation.
That means convincing competing governments and companies to collectively agree that there are some capabilities humanity shouldn’t pursue, at least until we know how to pursue them safely. Not exactly an easy group project.
Sources: UNILAD TECH, BM MAGAZINE