If we have machines that are smarter than us do what we want, we might not be careful enough about what we want. The lines of code that run these machines will always lack nuance, forget to spell out caveats, and end up giving AI systems goals and incentives that don’t match what we really want.
Nick Bostrom, a philosopher at Oxford, came up with a thought experiment in 2003 that has become a classic and shows how this problem works. Bostrom thought of a super-smart robot that was programmed to make paper clips, which seemed like a harmless goal. In the end, the robot turns the whole world into a huge factory for making paper clips.
Such a situation can be written off as something that might happen in the far future. But AI that isn’t aligned has become a problem much sooner than anyone expected.
One that affects billions of people is the most scary example. YouTube uses AI-based content recommendation algorithms to try to get people to watch more. Two years ago, computer scientists and YouTube users started to notice that the site’s algorithm seemed to be doing what it was supposed to do by recommending more and more extreme and conspiratorial content. One researcher said that after she watched videos of Donald Trump’s campaign rallies, YouTube showed her videos with “white supremacist rants, Holocaust denials, and other disturbing content.” She said that the algorithm’s strategy of raising the stakes went beyond politics. Videos about vegetarianism led to videos about veganism. Videos about jogging led to videos about ultramarathons. So, research suggests that, just to keep us watching, YouTube’s algorithm has been helping to divide and radicalize people and spread false information. Dylan Hadfield-Menell, an AI researcher at the University of California, Berkeley, said, “If I were planning things, I probably wouldn’t have made that the first test case of how we’re going to roll out this technology on a large scale.”
The people who made YouTube probably didn’t set out to make people more radical. But it’s impossible for coders to think of everything. The way we do AI now puts a lot of pressure on the designers to know what the effects of the incentives they give their systems are. One thing we’re learning is that a lot of engineers have made mistakes.
A closer look at AI
Researchers have started to work on a whole new way to program helpful machines to avoid these pitfalls and maybe solve the AI alignment problem. Most of the ideas and research behind this method come from Stuart Russell, a well-known computer scientist at Berkeley. Russell, who is 57 years old, did groundbreaking work in the 1980s and 1990s on rationality, decision-making, and machine learning. He is also the main author of the textbook Artificial Intelligence: A Modern Approach, which is used by a lot of people. In the last five years, he has become an important voice on the alignment problem and a common sight: a well-spoken, reserved British man in a black suit at international meetings and panels on the risks and long-term governance of AI.
Russell thinks there are two big problems. One is that our actions are so far from being logical that it might be hard to figure out what our true preferences are. AI systems will need to understand the order of long-term, medium-term, and short-term goals, as well as the many preferences and commitments we all have. If robots are going to help us (and not make big mistakes), they will need to be able to navigate the complex webs of our unconscious beliefs and unspoken wants. The second problem is that people’s tastes change over time. Minds change over the course of our lives, and they can also change quickly depending on how we’re feeling or what’s going on around us, which a robot might not be able to figure out.
Like the robots, we are trying to figure out what our preferences are and what we want them to be. We are also trying to figure out how to deal with ambiguities and contradictions. We also try to understand the form of the good, which is what Plato called the object of knowledge. At least some of us do this from time to time. Like us, AI systems may be stuck forever asking questions or waiting in the off position, not knowing how to help.