Historically, there are zero examples of machines killing millions of humans, so, by induction, it cannot happen in the future.

Calculators didn’t take over the world therefore, there is no reason to worry about superhuman AI. Here are just a few of the dozens that I have read in articles or heard at conferences:Įlectronic calculators are superhuman at arithmetic. Some well-known AI researchers have resorted to arguments that hardly merit refutation. In what most would consider a classic example of British understatement, The Economist magazine’s review of Bostrom’s book ended with: “The implications of introducing a second intelligent species onto Earth are far-reaching enough to deserve hard thinking.” Nick Bostrom’s 2014 book Superintelligence : Paths, Dangers, Strategies presented a detailed case for taking the risk seriously. Already, we see examples such as social-media algorithms that learn to optimize click-through by manipulating human preferences, with disastrous consequences for democratic systems. We are unable to specify the objective completely and correctly, nor can we anticipate or prevent the harms that machines pursuing an incorrect objective will create when operating on a global scale with superhuman capabilities.

The reason is that the “standard model” of AI requires machines to pursue a fixed objective specified by humans. If it succeeds in its current form, however, that could well be catastrophic for the human race. Photo: Jamie Chung/Trunk Archive Hand: Justin MetzĮditor’s note: This article is based on a chapter of the author’s newly released book, Human Compatible: Artificial Intelligence and the Problem of Control, published by Viking, an imprint of Penguin Publishing Group, a division of Penguin Random House.ĪI research is making great strides toward its long-term goal of human-level or superhuman intelligent machines.

Superintelligence paths dangers