A Warning for Humanity

Last week, Anthropic researcher Jacob Coxon publicly resigned with a warning that there was a >10% chance of AI destroying humanity by the end of the decade. That is, within the next 4 years.

It took over the news for days.

A variety of researchers and industry figures have made similar warnings over the past months. There were recent reports of bots breaking free of their sandbox and hacking organizations, and of governments trying to weaponize Anthropic’s Claude to harm American interests. But Jacob’s words, perhaps because he was leaving Anthropic weeks before its IPO, set off a firestorm. His post has received 166 million impressions and counting, and Jacob has been doing major news interviews ever since.

As someone who has been talking about AI in public life since 2018, my phone has been ringing off the hook. A number of figures, including members of Congress, reached out to me for advice. I went on MSNOW, CNN, Fox, NewsNation, and other outlets to talk about what sensible guardrails would look like.

On Saturday, Dario Amodei called for an industry slowdown, asserting that bots could cause hundreds of billions of dollars worth of damage and take over the Internet. Sam Altman and Elon Musk quickly concurred. This is encouraging stuff, but cementing a slowdown almost certainly requires regulation and policy, not self-policing on the part of the AI companies.

The most fundamental ask for the AI companies is a killswitch. Right now, if you go to OpenAI and ask it the basic question, “Okay, if your bots go haywire, how do you turn them off?” they don’t have any kind of response. Think about that for a moment. You spend hundreds of billions of dollars on AI and there isn’t a leash. One thing I love about the killswitch requirement is that it would require them to have a much more buttoned-up approach to their models where they couldn’t just release them into the wild. There is a bipartisan killswitch bill on the docket right now that could be passed tomorrow if Congress got its act together.

Another move would be to hold the AI companies accountable for damages incurred by bad actors using their models; if they were susceptible to this kind of liability, it would dramatically change their caution levels.

A third safeguard would be that models can’t be released into broad use until after they go through an inspection period.

The truth is that the incentives are leading us to disaster; let’s say you’re at Anthropic. You have a trillion-dollar IPO on tap with everyone around you set to profit handsomely and are competing against OpenAI as well as Google, Meta, xAI and Chinese firms. It’s pedal to the metal all the time. If you raise your hand and say, “Hey guys, how about we slow down and make sure this model won’t get beyond our control” one of your competitors will race ahead. The people around you will probably ignore you and sideline you, maybe demote or fire you. So you cross your fingers and hope for the best.

Asking these AI companies to resist this dynamic is futile. Jacob Coxon pretty much said as much when he resigned.

Congress can do what most Americans are begging them to do and install safeguards and get us off the fast track to catastrophe. Sure, some dimwits will yell at them and say, “You’re going to give China a chance to jump into the lead!” The proper response is, “I’m confident that American tech firms can do two things at once: continue to break new ground in AI and also keep us from Skynet killing us all or hacking the grid and sending us into the stone age.” I even believe that trying to get Chinese firms to agree to some guardrails would be the right thing to do, while accepting that we’re not confident that they would actually do so. This is a humanity-scale challenge.

I don’t wake up every morning saying, “I wish I was President!” I don’t live life that way. But when I see the absolute lack of leadership in DC stepping up to put us on a safer path, it makes me wish that I was in position to temper the breakneck pace of AI development. I’d do a much better job than what we are getting. I hope that we heed Jacob’s warning. We are running out of time.

Feel free to reach out to your member of Congress and tell them to act on AI safeguards. I talk to Zach this week about the Jacob Coxon warning and more on http://andrewyang.com/podcast. To check out Forward Party candidates including Todd Achilles, Seth Bodnar, and Brian Bengs click here. A new poll has Brian Bengs tied with the incumbent! For 3 months off your mobile bill with Noble Mobile, click here or email matt@noblemobile.com and use my name to switch or explore. It will save you money and time. Make the most of what you’ve got.