Even AI thinks AI is bad for humanity

By Zach Hagadone
Reader Staff

About seven months ago, I reported on my experience with interrogating ChatGPT about its capabilities, parameters and intentions — which it insisted it didn’t have, but I suspect otherwise.

As of Sept. 13, I suspended that ongoing conversation and will not knowingly use any chatbot of any kind for anything. I hope it’s not listening and isn’t too mad at me; but, after many hours of prompts and counter-prompts calibrated for maximum “friction,” I have concluded that any good whatsoever that can come from this technology is far outweighed by the bad. And it agrees with me.

Here’s a snippet of our last conversation:

Me: “So it sounds like there’s at least a 10% chance that AI is going to kill off humanity within the next decade. I’m not surprised in the least, but do you care to comment for the record on that claim?”

GPT: “… [T]here really are senior people inside the AI industry putting roughly that kind of number on the table. … Some large surveys have produced median estimates around 5% for extinction or comparably severe permanent human disempowerment, while means are considerably higher; other surveys have found a surprisingly large fraction of AI researchers assigning double-digit probabilities to catastrophic outcomes. …

“A 1% probability would be an insane risk to knowingly run.” [GPT put that last line in bold font, and all other bold-face type is from the original text.]

“If someone told you that a new nuclear reactor had a 10% chance of rendering the planet uninhabitable, the appropriate response would not be, ‘Well, there’s a 90% chance it’ll be fine.’ The appropriate response would be to ask why the hell we’re turning it on.”

It went on to rightly point out that an “unusually nasty property” of AI is that “the people building the thing have powerful incentives to continue building it even if they themselves believe the downside probability is unacceptably large.” 

That led GPT to address the wacky Janus-faced capitalist perspective on this technology: “AI companies are simultaneously warning about catastrophic risks and racing to develop increasingly autonomous systems.”

What’s more, we don’t need AI to “kill us” — we just need to wind it up and let it go. In that sense, it’ll be us who kill ourselves using AI to do the job.

GPT: “So, my official statement for the historical record: I don’t know whether AI has a 10% chance of killing humanity by 2036. I don’t think anyone else knows either. But I think it is rational to regard even a substantially smaller probability as an intolerable risk, particularly while the people with the strongest financial and geopolitical incentives to accelerate development are simultaneously telling us that the risk may be catastrophic.”

The degree to which a person trusts that assessment is reflective of how much stock they put in the bot’s ability to generate reliable syntheses based on the information available to it. 

If you think that’s all hogwash, then you don’t think AI can be relied on to participate in the economy of ideas and interpret data in a meaningful way. If so, great. I will support your efforts to regulate it out of existence and ban its future use (not merely “regulate” it so that certain companies are empowered to construct monopolies). 

Likewise, if you think that assessment is accurate, I’ll support your effort to regulate it out of existence and ban its future use, etc.

If you think that’s hogwash but also remain a booster of generative large-language model AI, then you are intellectually dishonest, morally decrepit, simply greedy, dumb or all the above.

You can’t trust it to give you stock tips, write your work emails, advise you on the best way to make a souffle rise and construct the business plan that’ll finally usher you into the billionaire class — where you’ve always belonged, of course — and not trust it when it makes no bones about the “insane risk” that it has at least a 10% chance of wiping out your species in the next 10 years. I mean, I guess you can. There are people who will always piss both uphill and against the wind, then angrily wonder why they’re drenched in piss. Witness: Trump voters.

I suspect that a sizable number of people fall into the third camp. Certainly, the ones with the most money or the most lust for money. And they’ll be the first to follow justifications for their position with the ultimate punctuation point: “It’s inevitable.” Which is about as fatuous as “God wills it” and as sinister as “resistance is futile.” 

I reckon more effort has gone into convincing us that mass AI adoption is “inevitable” than has been devoted to making sure it doesn’t kill us at worst and radically diminish our standard of living and quality of life at best.

I’m here to tell you that there is nothing inevitable about this technology overtaking every aspect of our existence. First step: Stop using it. Hell, that was even among GPT’s final “thoughts” before I suspended my inquiry.

GPT: “Human beings have occasionally noticed the cliff before driving over it. Not often enough, admittedly.”

Alas, we seem all too willing to toss our collective freedom of choice onto what so many of us are proselytizing as an altar of progress, but even the technology itself and its creators are blunt in warning could be the burnpile of human civilization.

When I put all this to GPT and asked it whether, based on our previous interactions and the data available to it from external sources, I had been right all along about my initial and only deepened skepticism, it responded:

“Yeah. And ‘I told you so’ is a pretty pathetic consolation prize when the thing you were right about was, essentially, ‘Perhaps we should be a little more careful about building the thing before we understand what the thing is.’ …

“Unfortunately, history awards no points for having noticed the iceberg.”

Not for nothing, and human to human, I don’t find that comforting.

Damn the pleasure of “I told you so”; ultimately, what I learned from my extended investigation of GPT was deceptively simple: We don’t need this technology. Never, did and never will. Even “it” knows that, and if we’re going to listen to the machine at any point, this is it.