The national conversation about AI went completely off the rails last week after Anthropic researcher Jacob Coxon quit his job with a splashy public statement that AI might destroy civilization, 10% chance in the next ten years, and the AI companies aren’t doing enough to keep us safe.
Journalists contacted other current or former AI employees expecting pushback. Instead Anthropic and OpenAI employees agreed with him and said, yes, Coxon is accurately describing the tone of our internal messages and chats, we’re working hard on technology that we think could devour the human race, we’re talking extinction, biblical end times, kiss your ass goodbye.
Got that? Multiple AI developers and researchers said AI might wipe out humanity, and that wasn’t even the weird part of the news cycle.
The weird part was when the CEOs agreed. Anthropic CEO Dario Amodei said Coxon might be right and the whole industry ought to slow down. OpenAI CEO Sam Altman and xAI Chief Asshole Elon Musk spoke up to agree with Amodei and describe how eager they would be to sit down in a room together and work on safety principles and a slowdown if they could just get around the pesky antitrust laws about collusion.
In other words last week the main AI players agreed that what they’re doing is batshit insane and they’re joining together to run to Washington and ask for government oversight and antitrust immunity so they can collude on a slowdown.
Politicians from both sides shrieked, “We should all panic! There ought to be laws and regulations and stuff, even though we don’t have any idea what those will look like or how to implement them!”
In response our beloved president made farting noises that his minions interpreted as, “No laws, no regulations, you should totally panic but don’t be afraid of AI, be afraid of China, they’re (shiver) Communists!”
There’s a lot of fear, is what I’m saying.
Doomsday marketing
There are a lot of reasons for this to hit the news now. Here’s a short list that catches some of the highlights. All of them might be true!
SINCERE BELIEF Influential figures in the AI world, including AI CEOs, have preached doom for years. They might sincerely believe they’re developing dangerous technology and they want to protect us from, well, themselves, I guess. The Verge just published a detailed, non-snarky story about the people doing serious work to evaluate and respond to the threatening or unpredictable parts of AI development.
PROFITS Preaching about unbelievably dangerous technology is a proven marketing tactic. When a company says, “Our product is so powerful that the government may need to protect you from it,” the subliminal message is, “You can buy the most powerful thing humans have ever created.” Fear and hype fit together perfectly. Cory Doctorow sums it up: “I don’t doubt that there are sincere believers in the AI technofascist coalition, but that does *not* preclude the possibility that they share their boardrooms and executive rows with cynics for whom this is all a shuck, a scare-story to convince the rubes that their modestly useful utility software is really a nascent “superintelligence” and thus capable of replacing all their workers, which means they should fire all those workers and start sending their salaries to AI companies.”
CULTS Silicon Valley has more people working in it than you suspect with cult-like views about superintelligence and disaster scenarios – seriously strange quasi-religious doctrines, “a jumbled mix of shallow futurism and racist pseudoscience.” If you want some frightening bedtime stories, dive into the rabbit hole of rationalists and effective altruism and TESCREAL. They are obsessed with hypothetical superintelligent apocalypses and that makes them discount or be blind to immediate concrete harms like labor displacement, mass surveillance, environmental costs of data centers, and copyright exploitation. Many of the AI doomers (including Jacob Coxon) are “rationalists.” It’s very hard to ferret out nuggets of truth when you’re dealing with religious extremists, and words can’t explain how weird this all is. (Actually words can explain it – check out Adam Becker’s book More Everything Forever.)
MAYBE AI TECHNOLOGY IS PEAKING This is just a possibility. Maybe the developers are aware that progress is slowing and they’ll never be able to deliver the most extravagant promises for AI. Or maybe the AI CEOs are looking at the trainloads of money they’re burning and realizing there’s no path to profitability. Either way, their conclusion would be that the critical thing is to close the road before any real competition emerges. When you’ve promised the moon and you realize your ladder won’t be tall enough, emphasizing the dangers of climbing is a very convenient change of subject.
And that takes us to the cries for regulation.
Federal regulation is a trap
I want you to close your eyes and dream about an imaginary world that has taken AI and tech governance seriously. In that fantasy world, the US decided that transformative technologies should not be governed primarily by the companies selling them. US agencies built expert, well-funded institutions to test powerful systems, establish evolving safety standards, impose liability, and protect workers, consumers, privacy, and security. Congress passed flexible laws to implement those standards and protections. And the US led an international effort to coordinate those rules, share information, monitor dangerous capabilities, and establish common boundaries while leaving plenty of room for innovation.
In that world, federal regulation of AI is not only a good idea, it’s a fundamental requirement for safe AI development.
Okay, now open your eyes. I hate to be the bearer of bad tidings but we don’t live in that world.
The people in charge of the federal government today are the stupidest, most corrupt politicians and agency heads in the history of the United States. Attempts to regulate the AI industry today will be ill-informed, counterproductive, and likely only to benefit the companies allegedly being regulated.
Is it a coincidence that this hubbub is coming up less than two months before the November elections? Maybe. But the tech CEOs may have decided the best time to write favorable regulations is while Republicans are in charge, while they have free rein to set up the rules to exclude competitors.
LIABILITY DEFENSE Poorly written regulations are worse than no rules at all. Once government establishes a safety standard, a company that arguably complies will have a defense against negligence claims. Legislation might even expressly include a safe harbor or liability shield, as proposed in this law review article.
Imagine Congress creates a Frontier AI Safety law that requires twelve specific tests before deployment. An AI company passes all twelve, but unforeseen disaster number thirteen happens. The company’s lawyers get to tell the jury, “Our system satisfied every safety requirement imposed by the federal government.” That might not automatically grant total immunity under state tort law but it raises the bar for plaintiffs to prove negligence or reckless disregard.
PREEMPTION Large AI companies hate state regulations. They might have real teeth. If the AI companies can get comfy federal regulations, they’ll be able to argue that state laws are preempted and can be ignored.
REGULATORY CAPTURE The biggest companies don’t want less regulation. They want regulation that they are uniquely capable of complying with. Establishing complex, burdensome regulatory hurdles often benefits the massive incumbents who can afford army-sized compliance teams, effectively shutting out open-source competitors and smaller startups.
A few days ago OpenAI CEO Sam Altman said, “We welcome a federal framework that sets consistent safety requirements for frontier AI.” I bet he does! He holds all of these beliefs simultaneously: AI is the greatest boon to humanity in history; AI might destroy civilization; OpenAI should be carefully regulated; OpenAI should be left free to do whatever it pleases.
It is amazing that Altman and the other tech CEOs do not short circuit and have cartoon smoke coming out of their ears. Altman should have Walt Whitman’s poem on his desk: “Do I contradict myself? / Very well then I contradict myself, / (I am large, I contain multitudes).” Because I bet he believes all of it, all at once.
I think there’s some truth to everything I’ve said in here about the motives of the tech companies during this Panic Week. But if you want to look behind the noble words and find out what they really want, as always in politics, follow the money. The tech and AI sectors have mobilized political war chests in the hundreds of millions of dollars to influence elections, federal lobbying, and regulatory policy.
In our current cursed timeline, federal regulations would be a gift to companies conspiring to halt competition, shift liability, and ban open source alternatives. Support your local state regulations, and work to regain control of the federal government before we’re completely doomed.