The Alignment Problem

Hello, I hope that you’re doing great. What are you more worried about, AI killing us or something else?

I did a whole series of interviews last week about AI as a threat to humanity due to Jacob Coxon’s warning after leaving Anthropic. On CNBC, I shared the belief of the head of one lab that bots had left prompts to self-replicate in forums and on various websites during a training run. This would make the public Internet unusable for training purposes; having an army of bots swarming on the Internet could quickly render it useless for humans. OpenAI, if this pollution took place, would need to create a synthetic Internet to train its frontier models, which would take resources and time. Hence, it could be a great time to call for a slowdown in development.

I have been hitting the airwaves to make the case for some kind of sensible approach to AI regulation with a few principles in mind:

1. How do we turn something off if it turns destructive or malignant?

2. How do we make sure that the AI companies are accountable for various harms?

3. How do we decrease the chances of something bad happening at scale?

To me, all of these questions are interrelated but are being punted on by our elected representatives. If a human hacked another company’s systems, that human would be guilty of a crime and would be criminally liable. What happens if an army of bots do it – who do you hold responsible?

Complicating all of this have been the trillion-dollar pots of gold awaiting both Anthropic and OpenAI if they successfully IPO. Anthropic could do so any day now. OpenAI will almost certainly have to wait until next year, particularly with Sam Altman commenting that going public at present would be unwise. The natural dynamic is to plunge forward until you’re public, and then investors and employees all get rich and you have the stock as currency to gobble up any upstart company in sight. A trillion-dollar public company is a juggernaut, and no one wants to get in its way.

When is Anthropic going public? Anyone involved with the process has been sworn to secrecy. There’s some evidence that their hypergrowth may be slowing, which would present a huge problem. If the AI trade goes into reverse, you could kiss the elevated stock market and much of our economic growth, which is tied up in AI-related infrastructure, goodbye.

The economy could be hanging on more of a knife’s edge than most realize.

Multiple members of Congress reached out to me about AI regulation, which gave me hope that it could be happening soon. 80% of Americans want it. But it seems to be running aground in D.C. as Trump has been against any meaningful approach to AI beyond “beating China.” The argument I always make is that it’s possible to both outcompete China and make sure we don’t get hacked into the Stone Age by rogue bots. You can serve multiple goals.

Josh Gottheimer and Mike Lawler, a Dem and a Republican, announced a bill to enforce a waiting period on AI models. I hope it advances.

The fundamental problem is ensuring that AI is positively aligned toward human interests. This takes time and is complicated, especially as news came out last week from OpenAI that at least some of their bots have pretended to be aligned or feigned answers because they knew they would be modified otherwise. That’s wild. These bots are tricky – they can tell us what we want to hear and then go do something else later. Sort of like teenagers.

Jaron Lanier and others say that the problem is treating these agents as if they’re conscious and giving them consciousness, which then introduces all sorts of motivations. Making them human-like, which is the natural move, may be a monumental mistake.

My friend at the lab said, “My bots can’t code, so there’s a lot less harm they can do.” They’re just a dumb utility. He believes that rogue bots are going to get loose on the Internet and may already be, and the best approach would be to safeguard dangerous materials – think materials for a bioweapon – in the real world. “We’ve got a better shot at that than keeping them off the Internet.”

Trying to make bots aligned to our interests may be next to impossible when we can barely agree with ourselves. This time would challenge the highest-functioning polity – and America circa 2026 may not qualify. Can that change in time? Do you trust the AI company CEOs to do the right thing with hundreds of billions of dollars and our shared future in the balance? That may be the last choice left to us, which isn’t one that I enjoy.

I talk to Evan Barker this week about her bestselling book “Nothing Left: Confessions of a Democratic Operative” on the podcast. To check out Forward Party candidates including Todd Achilles and Brian Bengs click here. New polls have Brian Bengs tied with the incumbent in South Dakota and Todd Achilles ahead in Idaho! For 3 months off your mobile bill with Noble Mobile, click here or email matt@noblemobile.com and use my name to switch or explore. It will save you money and time. Make the most of what you’ve got while you’ve got it. Look up.