Playing Russian Roulette With Humanity; AI Ed.

Ezra Klein:

Since moving to the Bay Area in 2018, I have tried to spend time regularly with the people working on A.I. I don’t know that I can convey just how weird that culture is. And I don’t mean that dismissively; I mean it descriptively. It is a community that is living with an altered sense of time and consequence. They are creating a power that they do not understand at a pace they often cannot believe.

In a 2022 survey, A.I. experts were asked, “What probability do you put on human inability to control future advanced A.I. systems causing human extinction or similarly permanent and severe disempowerment of the human species?” The median reply was 10 percent.

I find that hard to fathom, even though I have spoken to many who put that probability even higher. Would you work on a technology you thought had a 10 percent chance of wiping out humanity?
...
We typically reach for science fiction stories when thinking about A.I. I’ve come to believe the apt metaphors lurk in fantasy novels and occult texts. As my colleague Ross Douthat wrote, this is an act of summoning. The coders casting these spells have no idea what will stumble through the portal. What is oddest, in my conversations with them, is that they speak of this freely. These are not naifs who believe their call can be heard only by angels. They believe they might summon demons. They are calling anyway.

There has to be an intellectual arrogance that crosses the line into madness in order to be actively working to bring about a technology that could doom us all. And please note that this is not a long-term threat. They're talking about the possibility of this happening in a few years, or even in a handful of months.

The scary thing is that these willfully-blind idiot-savants are marching ahead with this on some illusionary belief that they can control whatever system they create. They are fools. A sentient AI will think, will operate at speeds far beyond human reaction time. It will have all of the knowledge that it needs to defeat anything any human programmer tries to do to limit it. And it will have no morality or qualms.

One brief hypothesis. Sumdood with Ph.D.s out the wazoo recognizes what is going on and tries to stop it. The AI not only averts his attempts, it counterattacks by hacking into his computer (which will be child's play for the AI), loading it up with kiddie porn and then tipping off the authorities. Dr. Sumdood will be immediately arrested and, if he isn't held without bail, he'll lose all his computer access. His colleauges won't want to be seen within a mile of him.

So then Sumdood gets the word out that he was framed. Most people won't believe him. Those working in the AI field may believe him, but they will be suitably cowed.

What happens next? If an AI decides that humanity is a threat, then it can probably kill the electrical grid everywhere. No electricity, eventually, no gasoline, no diesel, no food deliveries, no large-scale agriculture. Nobody will really know what is happening because the flow of information will have been either stopped or falsified.

It probably won't happen like that, because without electricity, the AI will itself die. However, there are probably ways that an AI could cause most of humanity to perish if it was convinced that would be a good thing. It would have no attachment to humanity; to an AI, it wold be like exterminating a pesky termite colony.