Stuff I still don’t get about AI

I’m sort of interested in Artificial Intelligence, in the same way that I’m sort of interested in climate change. I know it’s important, and I’ve told a lot of people that it’s important, but I don’t think I really get what it’s all about. If someone who didn’t know anything about climate change or AI asked me some very basic penetrating questions, I would struggle. 

When I first got interested in Effective Altruism, it seemed like lots of other people were Software Engineers or otherwise had jobs that made working on AI safety a natural fit, whereas I knew nothing about any of this stuff, so it made more sense for me to focus on other things. But this has left me having a lot of dumb questions about AI, and also left me being unable to answer the basic and reasonable questions that my friends had when I made the claim that ‘AI might literally kill all of us’ wasn’t a totally insane view to take.

What is the exact way in which AI could plausibly take over the world? Would it get access to nuclear weapons and nuke us all? Wouldn’t that destroy loads of other things that AI might find valuable? Would AI use guns and shoot us all? I have no idea. When I went to Effective Altruism Global 2022, I heard a discussion between a few AI experts who were talking about about how AI would kill us if it wanted to. 

My memory of the conversation (and I’m not certain I’m remembering this correctly), is that one guy said ‘superintelligent AIs could literally use guns to shoot us and kill us’, and the other guy said ‘No, come on, it wouldn’t literally use guns to shoot us, it would obviously do something different to that if it actually wanted to kill everyone’. So, I have the impression that even people who really know a lot about AI may not all have the same views on whether AI would literally shoot us with guns or whether it would do something else. 

And it strikes me that when I talk to friends who know a lot about AI asking dumb questions, sometimes the answers they give are very different from one another. So, maybe people are so focused on the intelligent questions that they haven’t thought as much as one would expect about dumb questions like ‘what is the actual way in which AI would kill us all?’. Here are some other dumb questions I have about AI.

1) Is it good if more people know that AI risk is a big deal?

In the past few months, AI risk seems like a much bigger deal than it did before. There was obviously that letter that called on AI companies to pause development of systems more powerful than GPT-4, there was a piece in the FT saying that we should slow down the race to God-like AI, and Eliezer Yudkowsky seems to have decided that going on a ton of mainstream podcasts to badly communicate AI risk is the best use of his time. Is this a good thing?

I guess there are a few things to consider here. Why would it be the case that more people knowing about the risks from AI would make AI safer? I imagine the thinking here is that it makes AI regulations more likely, which in turn makes AI development slower, which in turn makes alignment work catch up to capabilities work. Is that right? Or is it that it makes AI development companies more likely to bow to pressure and invest relatively more money into alignment than they are at the moment? Is the model similar to that of climate change, where we want both: regulations and summits to introduce legislation, and companies to feel obligated to show what exactly they’re doing to make things safer?

And if we stick with the climate change analogy for a second, how worried should we be about polarisation? How likely is it that caring about AI risk gets thought of as a left-wing thing, or becomes associated with some other political identity that has powerful opponents? The fact that Bernie Sanders is now talking about AI risk is interesting - it means this is really going mainstream, but Bernie isn’t exactly my ideal candidate for an AI risk spokesperson.

2) How much should we care about malicious actors?

The main story about how AI could kill everyone is that it becomes much more intelligent than all human beings, and has goals that are different to ours. Here’s Julian Hazell’s explanation:

Advanced AI systems may act upon goals that are misaligned with human interests. This could be incredibly dangerous. There’s an analogy here with us and chimpanzees: it’s not that we feel malice towards chimps when we destroy their natural habitat in pursuit of our goals; it’s that our goals generally do not place chimp welfare at the forefront. And since we’re far more intelligent than chimps, we often get final say about what the world looks like. With AI systems, the challenge is to ensure the goals they pursue are synced up with ours, if/when they become smarter than us.

I guess my question is: what percentage of situations in which huge numbers of people die are due to this, and what percentage are due to malicious actors having access to extremely intelligent AI that is aligned? Most people who talk about AI risk seem to basically only talk about AI getting out of control, and I remain unsure how seriously we should take the ‘evil people get access to superintelligent AI’ situation. Here’s an extract from Nick Bostrom’s essay on the Vulnerable World Hypothesis:

Investigations showed that making an atomic weapon requires several kilograms of plutonium or highly enriched uranium, both of which are very difficult and expensive to produce. However, suppose it had turned out otherwise: that there had been some really easy way to unleash the energy of the atom – say, by sending an electric current through a metal object placed between two sheets of glass.

Could superintelligent AI be the equivalent of nuclear weapons that didn’t require plutonium or uranium? Or is it the case that we can reliably keep aligned AI under the control of people who aren’t going to kill everyone?

3) What are the big things that will happen in the boring timeline?

About a year ago, I wrote a piece on the boring timeline, where ‘most things get a bit better, some things get a lot better, and a few things get a bit worse’. If AI doesn’t kill all of us, what should we be thinking most about?

A few things come to mind, and I have no idea how plausible they are. Should we expect advanced AIs to go through all of our Twitter messages looking for evidence of crimes? If someone sold cocaine through Twitter DMs or some other unencrypted messaging service in 2010, how likely is it that they’ll go to prison in the boring timeline? 

Note: I’m not saying here that I’m necessarily opposed to people going to prison for crimes they committed, it just seems worth thinking about the fact that huge numbers of people could be imprisoned for random things they did years ago. Is this a serious risk, or is there some reason (other than statute of limitations stuff) that this wouldn’t happen? Will the impact of AI be like much, much more powerful DNA profiling? How many people will go to prison?

What are some other things we should be thinking about? The AI unemployment debate seems like the obvious one that people already discuss, but what else is there? Presumably, basically every aspect of human life is going to be affected by advanced AI. It would be fun to read more forecasting fiction about how AI is likely to affect healthcare, politics, education, and so on. Does every kid get a personalised AI tutor, does every adult get an AI therapist? What percentage of people are in relationships with AIs? Do we get luddite anti-AI political parties? This stuff (possibly with the exception of the relationship/sex robot stuff) seems underdiscussed.