How dangerous is AI?

The future of humankind beyond this century arguably depends on how we manage two abstract resources: energy and information. Information technologies present an even bigger unknown than the climate crisis, both in terms of opportunities and hazards. Better computing resources will help in solving all manner of problems. But as these resources are increasingly boosted by the algorithms designated today as “artificial intelligence,” we need to wonder too what manner of machine mind we are building.

That’s the question asked in this year’s Reith Lectures, which were delivered by British computer scientist Stuart Russell of the University of California at Berkeley. Russell has been one of the most prominent voices from within the discipline calling for consideration of how AI can be developed safely and responsibly, and in his 2019 book Human Compatible he laid out his vision of how that might be achieved.

The first lecture, which I attended, was hosted at the Alan Turing Institute in the British Library, named after the mathematician who is often credited with launching the whole notion of “machine intelligence” created by digital computing. Turing’s 1950 paper “Computing machinery and intelligence” argued that the goal of making a machine that “thinks” was both feasible and desirable. But Turing also recognised that success would have profound implications for human society. “Once the machine thinking method had started, it would not take long to outstrip our feeble powers,” he said at the time. “If a machine can think, it might think more intelligently than we do, and then where should we be?” At some stage, he concluded, “we should have to expect the machines to take control.”

That wasn’t a new fear (if indeed fear it was). The annihilation of humankind by robots was portrayed in Karel apek’s 1920 play RUR, which introduced the Czech word robota (meaning labourer or serf) into the vocabulary. Turing himself cited Samuel Butler’s 1872 utopian novel Erewhon, in which machines are banned, lest we end up “creating our successors in the supremacy of the Earth.”

In 1965 the computer scientist Irving John (“IJ”) Good, one of Turing’s colleagues during his wartime code-breaking work at Bletchley Park, said that if we succeeded in making an “ultraintelligent machine” that could “far surpass all the intellectual activities of any man,” there would be an “intelligence explosion.” Such a device, said Good, would be “the last invention that man need ever make.” And if movie fantasies like the Terminator series are to be believed, it could end up being the last invention we ever can make.

Such disaster-mongering is not confined to Hollywood. In 2014, Stephen Hawking warned that “the development of full artificial intelligence could spell the end of the human race,” while Elon Musk has warned that AI could be “our biggest existential threat.” There is no particular reason to give their views much credence, as neither is an expert in AI. But Russell treads a careful path in his lectures in asking how real such dangers are, being neither the over-optimistic Pollyanna nor the doomsaying Cassandra.

Today’s AI is not obviously a forerunner of some apocalyptic Terminator-style Skynet. Most artificial intelligence of the kind used for, say, analysing big data sets, image and voice recognition, translation and solving complex problems in finance, technology and game-playing, is based on the technique called machine learning. It uses networks of interconnected logical processing units that can be trained to detect patterns in the input data (such as digitised images). Each training example elicits a slight readjustment of how the units in the network “talk” to each other, in order to produce the correct output. After perhaps several thousand such examples, the system will reliably (if all goes well) generate the right output for input data that it hasn’t seen before: to positively identify an image as a cat, say. For a game-playing AI such as DeepMind’s AlphaGo, which defeated a human expert player in 2016, the correct output is a move that will increase the chances of winning.

As such examples show, this approach can be extremely powerful. Language-translating AI such as the popular algorithm provided by Google is of course far from perfect—but it has come a long way, thanks in particular to an advance in the technology around 2015, and now generally supplies a serviceable result. Combined with AI voice recognition, such systems are already enabling real-time translation of spoken text.

“AI lacks common sense, which we just don’t know how to express in formal rules that we can program into a computer”

Is this, though, generating anything like the “thinking machines” that Turing envisaged? Turing suggested that one of the most demanding tasks for what was soon (in 1955) to be christened AI would be conversation with a human. In his famous “imitation game,” often now called the Turing test, he posed the scenario of a human and a machine, both placed out of sight, giving answers to questions from a panel of human judges. The test would be passed if the judges could not tell which was which. Would this mean the machine was “thinking”? Since all we have to go on is the indistinguishable outputs, said Turing, it would seem that only prejudice would prevent us from awarding the machine that ability.

The problem, though, is that we simply don’t know what is going on in that black box. The principles of machine learning are clear, but the “reasoning” by which the algorithm arrives at its conclusions after training is opaque. Occasionally such machines will arrive at a deeply peculiar answer that a human would never give—for example misassigning an image in absurd, even comical ways. What this sort of AI lacks is what we casually call common sense, but which we just don’t know how to express in formal rules that we can program into a computer. In other words, today’s AI might be able to do some things better than us, but not by “thinking” in the same way as us.

Until we know how to address that issue, we can’t count on AI to come up with the answers we want or need. For an AI-driven car or an AI-powered medical diagnosis, that could be a fatal shortcoming. So serious is the problem that the US Defense Advanced Research Projects Agency has launched a program called “Machine Common Sense” to try to crack it.

You can see the worry here. As AI becomes ever-more powerful, and as we entrust to it ever-bigger or more vital decisions, can we be sure it won’t come up with solutions that are, from our point of view, immoral, disastrous, even deadly for our species? Computer scientist Iyad Rahwan, who heads the Berlin-based Center for Humans and Machines, gave me an example when I spoke to him recently. “Let’s say,” he told me, “you simply ask the machine to sell more IVF services. The machines could learn all kinds of unexpected, potentially unethical strategies to achieve those goals—like, let’s encourage people to delay having children because then maybe they’ll need IVF in the future. It has to be a very villainous human to think of something like this. But the people who are building these systems wouldn’t even know.”

The Swedish-born philosopher and physicist Nick Bostrom, who was in Russell’s audience in London, calls such outcomes “perverse instantiations.” Bostrom, who heads the Future of Humanity Institute at the University of Oxford, has proposed an example straight out of dystopian sci-fi. Suppose we charge a super-intelligent AI system, controlling robotic machinery with super-advanced abilities for assembling materials from atoms, with the task of maximising its output of paperclips. Before we know it, the machine might start pulling apart everything on the Earth (and beyond?) into its atomic constituents and putting them back together as paperclips. As Russell puts it: “if we put the wrong objective into a super-intelligent machine, we create a conflict that we are bound to lose. The machine stops at nothing to achieve the specified objective.”

Will it, though? Some consider these scenarios not just far-fetched but illogical. Cognitive scientist Steven Pinker says that they depend on the premises:

1. 1. 1. That humans are so gifted that they can design an omniscient and omnipotent AI, yet so idiotic that they would give it control of the universe without testing how it works; and
      2. That AI would be so brilliant that it could figure out how to transmute elements and rewire brains, yet so imbecilic that it would wreak havoc based on elementary blunders of misunderstanding. [Oh, you mean you didn’t want me to take you apart?]

It’s a good point—not so much because Pinker is right, but because it forces us to ask what we mean by “intelligence” anyway. If “super-intelligence” excludes the ability to attune to the wants and needs of other beings like us, is it really so smart after all?

That question hangs over discussion of how to design general-purpose AI, meaning AI developed not to excel at some narrow task like chess or driving but that, as Russell puts it, “can quickly learn to perform well across the full range of tasks that humans can perform.” He says that “this has been the goal of AI since the beginning”—but admits that we are a long way from that destination.

He’s right—but do we even know the destination anyway? It isn’t even clear what “general-purpose intelligence” means. Russell says “general-purpose AI would have access to all the knowledge and skills of the human race, and more besides.” Like us, but better. There’s a danger here of imagining that cognition is then just a series of modules and capabilities we can add or omit at will. No mind we know of is like that. It’s not clear if we could truly design a machine to have all our abilities and propensities without it basically replicating the entire human mind in some artificial manner, including our capabilities for feeling and empathy and perhaps with conscious awareness. In other words, minds relying on cold, mechanical logic and reason alone might not be as versatile as ours.

But no one knows if it is possible at all to make machine minds that have consciousness and emotions (rather than just emulating their appearances). And if wedid make machines with some degree of consciousness, we would surely need to give them moral rights and consideration: an issue poignantly explored in Kazuo Ishiguro’s latest novel Klara and the Sun. That’s a burden of obligation we could do without: as philosopher of mind Daniel Dennett has suggested, what we really want from AI are tools, not ersatz humans. Sure, as Russell says, we might desire “software agents and physical robots capable of designing and building bridges or fully automated factories, improving crop yields, cooking dinner for a hundred guests, separating the paper and plastic, running an election, or teaching a child to read”—but why all in one general-purpose machine?

“A super-intelligent AI system might start pulling apart everything on earth to maximise its output of paperclips”

While all this is the stuff of speculation and philosophy, Russell points out that the immediate challenges and dangers of AI arise at a level far below Skynet-style super-intelligence. We already have, for example, AI smart enough to replace humans in some jobs. Our common mistake here is to imagine, and then to worry about, a simplistic human-to-robot switch, as if it will happen in a society otherwise unchanged in its goals, demands and infrastructure. The history of technology teaches us otherwise. The factory, the car and the computer itself didn’t simply replace old ways of manufacturing, getting around or making calculations. They altered societies in ways that had nothing to do with their original purpose, ranging from how and where we live to what rights and responsibilities we recognise and how we interact with one another. AI will have, and is having, such effects too, even while it remains in many ways rather dumb. Imagining that it will simply lead to an age of either mass unemployment or total leisure would be a historically illiterate mistake.

One concern is that we start giving such dumb AI responsibilities it may not warrant—for example, using machine intelligence to evaluate and propose legal judgments (as some courts in the US and Europe have already done) or letting it trade on the stock market in hyperfast transactions that could spin off in all kinds of disastrous directions in the blink of an eye. That’s a particular concern for military applications, one subject of Russell’s lectures. Governments are already developing “lethal autonomous weapons,” which the UN defines as “weapons that locate, select and engage human targets without human supervision”—“engage” here being a euphemism for “kill.” An argument offered for such systems is that they are more accurate than human soldiers and lead to fewer civilian casualties. But the capabilities could be so powerful that many see them as unacceptable. UN secretary general António Guterres has said that “machines with the power and discretion to take lives without human involvement are politically unacceptable, morally repugnant and should be prohibited by international law.”

The point is that we’re not talking about highly futuristic technology, but integrating stuff that already exists: drones with facial (or racial?) recognition AI, say. Russell has teamed up with filmmakers to produce a short video, Slaughterbots, offering a chilling vision of how such technology could be used by repressive regimes. It was Vladimir Putin, after all, who said in 2017 that “the one who becomes the leader in [AI] will be the ruler of the world.”

Containing threats like this is, and must be, part of the urgent business of international organisations and treaties here and now. But as for ensuring that we don’t create AI systems that endanger our safety more generally, Russell argues that we should use three design principles to ensure that our machines remain aligned with our own goals:

1. 1. 1. The machine’s only objective is to maximise the realisation of human preferences;
      2. The machine is initially uncertain about what those preferences are [it is “humble”]; and
      3. The ultimate source of information about human preferences is human behaviour.

In other words, we should make machines that will constantly be asking: am I doing this right? Is this good enough? Do you want me to stop yet? “We have to build AI systems that know they don’t know the true objective, even though it’s what they must pursue,” says Russell.

Will that work? A trio of “robot safety rules” intentionally recalls Isaac Asimov’s Three Laws of Robotics in his 1950 sci-fi classic I, Robot—which was mostly composed of stories showing how they could go wrong. We shouldn’t give up, however, on trying to understand “the mind of the machine”—a goal that Rahwan and others have suggested is best approached by treating AI as if it was a living entity and using the methods of the behavioural and psychological sciences, as indeed we already do to try to understand other animals. After all, we are going to have to live together somehow—so we had better try to get to know each other.