Radio Open Source in Clips: AI Apocalypse?
Follow into
Save into
Follow into
Source: https://radioopensource.org/ai-apocalypse/
AI Apocalypse? On Radio Open Source in Clips, Christopher Lydon talks with Ben Hurlbut, professor in the School of Life Sciences at Arizona State University. Lydon introduces the subject as the life-and-death hazard of artificial intelligence. The phrase, he says, sounds a little panicky, but when the masters of the craft began arguing this summer about how soon, in the next ten years, their science might wipe out human life forever, attention had to be paid. Hurlbut, Lydon notes, has been a wary eye on science in general, a teacher and guide for issues that are simply difficult to talk about. Five years earlier, Hurlbut led the program through the CRISPR conundrum around editing the human genome. Now Lydon asks how we got to killer AI, lethal AI, and how we might get past it.
Hurlbut begins with a careful correction. He does not claim to have good answers, and he says he is alarmed by the level of alarm. As a historian and social scientist, he wants to ask why the panic has taken this shape. The right focus, he argues, is not to chase after AI. It is to ask about the regimes of creativity, the formations of freedom and constraint, that are producing these particular worries about risk. Listen to Hurlbut's central framing, and to Lydon's response:
or not. I don't have the expertise to make a definitive judgment about that question. But whether those concerns are as existential as they've been made out to be, the worry is to me, the way in which those concerns are being articulated and by whom. In effect, the creators are also the whistleblowers, and the creators are engaged in a kind of frenzied project of hyper-rapid creation about which they are declaring there needs to be constraint, even stoppage. I mean, what's given us that formation? That seems to me to be the fundamental question, because if this is a problem of democratic governance, we got to recover the democratic ground upon which we engage in the project of governance. And that's so welcome. I'm thrilled that we're not talking about specific death counts or whatnot, and that you're steering it toward the question of American political culture.
AI Apocalypse?, from 2:26
That is the argument Hurlbut returns to throughout the episode: the creators of AI are also its whistleblowers and its prophets. After Lydon welcomes the turn away from specific death counts, Hurlbut places the problem inside American constitutional history. The country was invented by eighteenth-century gentlemen working from the English Enlightenment, with convictions about rights and the pursuit of happiness. Now, Lydon says, it is run by tech nerds. Hurlbut says the deeper issue is that a scientific and technological society has not asked foundational questions about what it means to govern the forms of knowledge and technology that order its life. He calls AI a symptom of a larger pathology, not the disease itself.
Hurlbut adds that the problem is not only concentrated power but delegation and deference. AI is a technology of delegation. The dream is to build digital extensions of human work, little digital slaves. The trouble is that handing tasks to the AI is only a later expression of an earlier delegation: the public has handed the AI builders authority to imagine and declare what progress is. The recent papal encyclical, Hurlbut says, tries to put the human back at the center, in a position of humility. Silicon Valley, by contrast, speaks in a different spiritual register.
Lydon observes that the Pope sounds like a humanist while the technologists sound like spiritualists. Hurlbut answers that AI discourse is full of apocalypse, prophecy, and salvation, but it is a spiritualism without humility. He points to an interview with Jensen Huang, the CEO of Nvidia, who is ordinarily a nuts-and-bolts figure. Listen to Hurlbut quote Huang and then say what he hears in that language:
is about the valorization of the self, really, whether they recognize it or not. So I was listening to Ezra Klein's interview with Jensen Huang, the CEO of NVIDIA, who is about as nuts and bolts sort of down to earth character in this domain as there is, disinclined to engage in this kind of eschatological discourse. But he said when Ezra Klein asked him about the future, he said there are these moments of transformation. There was the rise of electricity and suddenly we could do anything because we had power. And then there was the rise of the Internet and suddenly we could search for anything. And with AI, the transformation it will bring about, and this is a quote, is that it will allow us, quote, to know everything and do anything. It will give us omnipotence. and omniscience. There is an idolatry.
AI Apocalypse?, from 10:35
Hurlbut calls this an idolatry, a religion divorced from the humility of human life. Lydon pushes on the difference between spiritual death and literal extinction. Hurlbut says the two should be thought together. The public, he says, is being offered only two options: progress of a sort never before imagined, or we all die. When Bill Gates warns that AI could cause a billion deaths, Hurlbut asks not just what the mechanics would be, but who the prophets are, and what sort of purification their prophecy calls for. He hears an incipient spirituality expressing itself in prognosticated death statistics.
From there, the conversation turns to the absence of adults in the room. Lydon quotes a European Union adviser who says there are no adults in the room, and that this is what makes the whole thing so terrifying. Hurlbut agrees, and sharpens the point: the governing language is safety, not flourishing. You want your children to be safe, but not merely safe. The fact that safety dominates the discourse shows an impoverished, even juvenile moral imagination. Lydon reads a description of Dario Amodei, the CEO of Anthropic, as an intellectual and quasi-religious leader. Hurlbut's question for Amodei is whether he feels in control of what he is in control of, or whether he feels imprisoned by his own agency.
Hurlbut sees a bizarre intertwining of hyper-agency and inevitability. The AI builders speak as if they are building a freight train they did not make and cannot stop. Yet the technology requires enormous investments of energy, capital, and talent. Lydon brings up Jill Lepore's book on the artificial state, and the idea that AI leaders are living out science fiction they misread as children, including fantasies of escaping their own humanity. Hurlbut asks whether those stories are predictions or self-fulfilling prophecies. He says the makers of AI are reaching for a position of playing God, with power over annihilation or total transformation. For Hurlbut, the missing question is simply what is it for? To what problem is AI the solution? Why should we want it?
That question, he says, has not been asked democratically. The most material manifestation of AI right now is the data center, and the grassroots reaction against data centers is a kind of democratic voice saying, thank you but no. Yet anyone with a pension or a 401k is implicated in the speculative bet. Pull back into the space of collective democratic imagination, and the conversation has not happened. The thing that threatens to kill us is a thing the public did not ask for but was told was good for it. Hurlbut says this is an old bad habit: the makers of a promised future declare what that future must be and dismiss dissent as regressive. He recalls the 2023 congressional hearings in which the industry said, in effect, we are building dangerous things, please regulate us, and the conversation ended with, maybe you should regulate yourselves.
Lydon connects this to the internet. He says the larger purpose and the downside of the internet were never really reckoned with. Hurlbut answers that the internet was regulated, but by removing constraints. Section 230 gave platforms no liability, emancipating them from responsibility so they could move fast. That produced the pools of capital now invested in AI. The utopian promise of frictionless information was a fragile determinism: it could unfold only if no law or worried person got in the way. It became a self-fulfilling prophecy.
Lydon asks how the tech culture sees rights and responsibilities differently. Hurlbut points to effective altruism and long-termism. To be a good person in this frame is to maximize future good, which turns the moral actor into an oracle. Present consequences become incidental; what matters is the future the actor claims to produce single-handedly. Hurlbut contrasts this with an older long-termism, the Judeo-Christian horizon of eternity, in which the present life has meaning in relation to divine justice. By comparison, the new long-termism is an impoverished version of the afterlife.
Then Hurlbut turns to Carl Schmitt, not out of admiration but because Schmitt is the theorist of emergency. Schmitt's insight, Hurlbut says, is that in circumstances of emergency, authority flows to whoever decides in the exception. The sovereign is the one who stands up, takes control, and receives deference. The AI discourse, now at fever pitch, is a discourse of emergency. Lydon connects this to the political present: a White House that exalts in ruling by decree and emergency. Hurlbut agrees with Jill Lepore that there is a family resemblance between the technological and political orders. But he departs from Lepore by asking listeners to look at their own responsibility, their own inclination to sit on their hands and defer to expertise.
So when Lydon asks for an encouraging precedent, Hurlbut offers a deliberately complicated one. The Asilomar conference on recombinant DNA in 1975 is often remembered as a model of responsible self-governance. Paul Berg called a voluntary moratorium, and roughly a hundred and twenty leading researchers gathered on the California coast to debate constraints. Hurlbut says the meeting produced real constraints, but it also narrowed what could be asked. Listen to Hurlbut describe what Asilomar actually produced:
In the end, they set a series of constraints. They asserted the technical expertise of that group as the only basis for placing limits. In other words, only the group that knew the most could govern the research appropriately. And therefore, scientific self-regulation was the best way and the only way to effectively govern research in this domain. What did that mean, scientific self-regulation? The imposition by the scientific community on itself of voluntarily adopted constraints in the name of the safety of research. So what Asilomar produced was a set of guidelines that was handed to the NIH and the NIH implemented as guidelines.
AI Apocalypse?, from 47:19
So Hurlbut does not present Asilomar as a clean model. It is an ambiguous inheritance. The scientists acknowledged the danger and then claimed the right to be the ones who worried, constrained, and assured the rest of the world. Lydon says that is why the era is also interesting for a different reason: in Cambridge, the populist city councillor Al Vellucci demanded that the DNA game be stopped before creepy crawlies came out of the sewers. Hurlbut hears in that pushback a democratic voice that the AI conversation still needs.
The conversation ends with Lydon thanking Ben Hurlbut for changing the subject, or revealing the subject, around AI. The open question is not whether AI is dangerous, but whether a democratic public will recover the authority to ask what it is for, who decides, and why it should exist at all.