Tuesday, September 27

‘Existential catastrophe’ caused by AI is likely inevitable, DeepMind researcher warns

Researchers from the University of Oxford and Google’s artificial intelligence division, DeepMind, have claimed that there is a high likelihood that advanced forms of AI will become “existentially dangerous to life on Earth”.

In a recent article in the peer-reviewed journal AI Magazinethe researchers warned that there would be “catastrophic consequences” if development of some AI agents continues.

Leading philosophers like Oxford University’s Nick Bostrom have previously spoken out about the threat posed by advanced forms of artificial intelligence, although one of the authors of the new paper claimed such warnings were wrong. far enough.

“Bostrom, [computer scientist Stuart] Russell and others have argued that advanced AI poses a threat to humanity,” Michael Cohen wrote in a Twitter feed accompanying the article.

“Under the conditions we have identified, our conclusion is much stronger than that of any previous publication – existential catastrophe is not only possible, but likely.”

The article proposes a scenario in which an AI agent strategizes to cheat in order to receive a reward that it is pre-programmed to seek.

In order to maximize its reward potential, it requires as much energy as it can get. The thought experiment sees humanity ultimately competing with AI for energy resources.

“Winning the competition of ‘using the last bit of energy available’ while playing against something much smarter than us would likely be very difficult,” Cohen wrote. “Losing would be fatal.

“These possibilities, theoretical as they are, mean that we should be making slow progress – if at all – towards the goal of more powerful AI.”

DeepMind has already offered a safeguard against such an eventuality, dubbing it “the big red button”. In a 2016 article titled “Safely Interruptible Agents,” the AI ​​company outlined a framework to prevent advanced machines from ignoring shutdown commands and becoming an out-of-control rogue agent.

Professor Bostrom has previously described DeepMind – whose AI achievements include beating human champions at the board game Go and manipulating nuclear fusion – as the closest to creating human-level artificial intelligence.

The Swedish philosopher also said it would be a “great tragedy” if the development of AI does not continue, as it has the potential to cure disease and advance civilization at an otherwise impossible rate.