Existential and/or Normal Technology

My summary and commentary on a recent debate on the future of artificial intelligence

One panel of Detroit Industry, North Wall, by Diego Rivera. Credit: Getty Images

---

This piece is part of a broader effort to synthesize key debates in AI policy. My goal is to make these arguments legible to non-specialists while contributing to a more grounded public conversation about what lies ahead.

Will artificial intelligence upend global power in the next few years or merely reshape industries over a generation? This question, fraught with economic and political implications, was answered in two recent papers that outline radically different predictions. On Tuesday, May 6th, Americans for Responsible Innovation (ARI) pitted the two visions against one another in a debate featuring an author from each respective paper and moderated by ARI’s McKenna Fitzgerald. What is at stake, depending on which narrative convinces you, is either the future of human civilization or the diffusion of a major technology comparable to electricity or the internal combustion engine. 

Eli Lifland, a professional forecaster and co-author of AI 2027, represented one side of the debate. In their paper, Lifland and other expert forecasters predict that in two years, superintelligent AI will be capable enough to replace the vast majority of economically productive human tasks, spurring an AI arms race between the US and China. What happens next depends on humanity’s ability to align AI models with human values; if successful, they predict a rapidly advancing society coexisting with systems far more intelligent and capable than humans; if unsuccessful, humanity faces replacement by AI.

Sayash Kapoor, a computer science PhD student at Princeton University and co-author of AI as Normal Technology, represented the other side. Kapoor, along with Princeton professor of computer science Arvind Narayanan, argue that AI’s impact will be seen on the timescale of decades, not months, and should be viewed (at least for now) in the light of previous technological advances, not millenarianism. They reject the framework of superintelligence and the concern of AIs having a role to play in their own future, and ultimately conclude that reducing uncertainty and building resilience to harms from AI systems are the most important current policy goals. 

Roadmap

1. Analysis

2. Agreements and Disagreements

3. Commentary

4. Conclusion

Analysis

“AI 2027” follows in the footsteps of philosopher Nick Bostrom’s 2014 book Superintelligence: Paths, Dangers, Strategies, and Leopold Aschenbrenner’s 2024 paper “Situational Awareness,” in anticipating an “intelligence explosion.” This is a kind of escape velocity or point of no return once AI models become smart enough to become their own AI engineers, leading to rapid improvements in algorithms.

Since the authors see recent progress of LLMs as on an inevitable trend line leading to such a model sometime in 2027, the question that is most important to answer for them is how to align these consequent superintelligent models with human values. This follows the alignment literature that has dominated discussions of AI safety. The fear is that as AI becomes more capable, it might pursue goals that are misaligned, either because its objectives were poorly specified or because it develops unintended instrumental strategies to achieve its tasks, like deception or power-seeking. This misalignment could pose serious risks, especially with advanced systems that make autonomous decisions at scale. Researchers aim to design AI that is robustly beneficial, meaning it does what we want even in unforeseen situations.

“AI 2027” includes an interactive panel which tracks relevant statistics throughout their timeline. By March 2027, for example, they predict that the most capable AI model will reach the ability of a superhuman coder that works at 30 times a human’s speed. Credit: AI 2027, Kokotajlo et al., 2025. 

In contrast, Kapoor and Narayanan’s paper, subtitled “An alternative to the vision of AI as a potential superintelligence,” rejects the superintelligence and alignment frameworks. While they accept that AI’s recent progress is transformative, they do not think that extrapolating from this recent progress to runaway rogue superintelligence is as realistic, or as productive, as the “AI 2027” authors do. Following George Washington University assistant professor of political science Jeffrey Ding’s 2024 book Technology and the Rise of Great Powers: How Diffusion Shapes Economic Competition, they view the question of diffusion of AI throughout the economy as central to AI’s effect—a task which they see as having far more regulatory, practical and economic roadblocks than AI 2027’s account. 

“AI as Normal Technology” argues that the feedback loop between innovation and diffusion will drive AI’s impact, with plenty of speed limits along the way. Credit: “AI as Normal Technology,” Narayanan and Kapoor, 2025. 

Importantly, Narayanan and Kapoor do not necessarily reject the possibility of future superintelligent models; what they reject is “fast takeoff” scenarios that view these models as arriving in the next few years. Because of this, they “do not see it as necessary or useful to envision a world further ahead than we have attempted to.” (AI as Normal Technology, p. 2)

Agreements and Disagreements 

While both Lifland and Kapoor agreed that artificial general intelligence won’t come overnight, their timelines for its appearance vary greatly. In ARI’s debate, Kapoor echoed the terminology of Wharton professor of management Ethan Mollick and others noting AI’s “jagged frontier,” namely that AI excels at some work but is very limited at others. This complements Part I of “AI as Normal Technology,” where the authors explain reasons why they believe AI diffusion will be bottlenecked, why AI progress won’t immediately translate to economic gains, and inherent speed limits to AI development. What matters will be restructuring current industries around the new technology to leverage its full benefits, a process they expect will take decades, like in the case of electricity and the internet. 

Lifland conceded that Kapoor’s prediction is plausible, though still holds that it will be much more likely that AI’s rapid progress in recent years will continue apace, and an AI remote worker working far faster than a human being will be available in the near future. As AI 2027 readily admits, their predictions before the AI supercoder-induced intelligence explosion for the next few years are “heavily informed by extrapolating straight lines on compute scale ups, algorithmic improvements, and benchmark performance” (AI 2027, Appendic C: Why Our Uncertainty Increases Substantially Beyond 2026). In the AI 2027 scenario, the United States and China realize the economic and political importance of superintelligence and established Special Economic Zones (SEZs) free of some of the roadblocks “AI as Normal Technology” argues will slow down AI diffusion. The profit incentive, and the AI arms race between the US and China, will speed up potential roadblocks to diffusion in the real world.

“Figure 5. Two views of the causal chain from increases in AI capability to loss of control.” Credit: “AI as Normal Technology,” Narayanan and Kapoor, 2025.

Figure 5 in “AI as Normal Technology” neatly captures the heart of the disagreement. The superintelligence view sees more capable models as essentially synonymous or inevitably leading to more powerful models. Therefore, they focus on how to decrease the chances that we would lose control of them. Conversely, the normalist view Narayanan and Kapoor hope to promulgate does not take capability → power as a given. In their telling, the real locus of safety is in institutions, incentives, and regulatory design: the things that determine who wields AI systems, how they’re deployed, and under what constraints. Rather than asking whether GPT-6 might go rogue, they’re asking whether the people and systems using it are doing so safely, slowly, and with proper guardrails in place, and they see these as solvable problems.

There is still plenty of room for agreement between the two views. Both Lifland and Kapoor agreed that policymakers should 1) increase the transparency of model capability and 2) increase state capacity to be prepared for disruptions like AI-caused labor shocks.

Unsurprisingly, there was significant debate between the two co-authors on the future of work. Kapoor anticipates a redefinition, rather than mere replacement, of many jobs and finds an important role for human beings to monitor and verify AI models. Humans, in the normal technology lens, will still have a role to play to monitor and verify the outputs of AI models. On the other hand, Lifland sees such a role for humans as merely a brief transitional period, before the AIs themselves become more capable at monitoring AI models and replace the humans. 

Commentary

I am conflicted between the two pieces. On the one hand, knee deep in the super intelligence-and-alignment-informed discussions on AI safety, I have gotten the same feeling that Narayanan and Kapoor seem to get—haven’t we gone through this before? Prognostications of coming revolutionary change from technological innovations are the norm, not the exception, throughout history, and so far none of them have proved true (at least at the speed and scale that such technological transformations actually had). Indeed, forecasts about the future of artificial intelligence stretch back at least to computing pioneers in the 1950s and 60s who predicted machines that can do “anything a man can do” or have “the general intelligence of the average human being” in the proceeding decade or two. When those predictions proved unrealistic, the field entered its first “AI winter,” part of the cycle of breakthrough, optimistic predictions, over-investment, and subsequent crash that has characterized AI’s progress from the beginning.

I also agree that problems of labor disruption, exacerbation of inequality and algorithmic discrimination are sidelined by the superintelligence story; even if AI progress stalls where it is now those problems are very much real. There are reasons to doubt that AI progress will continue apace. Ding’s book on diffusion is convincing as to the importance of diffusion as the guiding principle to a technology’s impact. Finally, the superintelligence and alignment discourse sometimes feels a bit too much like science fiction, detached from the realities of the world and assuming that a superintelligent AI trained on human data would have reason to destroy its creator.

On the other hand, as AI 2027 co-author Scott Alexander points out, the default position we should have is that AI progress will continue apace, and we need specific reasons to argue that it will slow down. While Narayanan and Kapoor do some of this, I find some of their perceived bottlenecks to be hurdles that will be jumped over as progress continues and businesses and countries gamble on its potential economic and national security importance. I also think it’s prudent to start doing the game theory on how this technology might play out, and it is prudent to consider that how models might deceive their users - something they are already doing.

While Narayan and Kapoor’s work is surely seminal in its articulation of the normal technology view, I think many policymakers in Washington have already been thinking in similar ways. Outside the Californian cyberpunk colony of San Francisco, AI is simply another technology that could help foster American innovation. Last year, Senator Schumer’s AI Insight Forums led to little regulation but a lot of buzzwords about technology and innovation, punting regulation to the states. And, as seen in the recent Senate hearing on AI, there is little indication that policymakers are focusing on the catastrophic or existential threats that AI might bring.

At the same time, policymakers and the administration are keenly aware of a topic that was somewhat absent from “AI as Normal Technology” but prominent in “AI 2027,” namely that of an AI arms race with China. The framework of diffusion takes domestic policy to implement AI in the economy as central, but it may have blindspots when it comes to the international dynamics of AI competitionand espionage on AI labs. Of course, Narayanan and Kapoor do not see AI progressing nearly as fast as Lifland et al., so they do not see it as having the same relevance to national security debates.

Lifland and Kapoor’s debate exemplifies the fork-in-the-road moment for AI policy: whether we are in the midst of a Cold War-esque security dilemma or a marathon to diffuse the technology safety determines what is the best use of our limited resources. If AI 2027’s predictions are taken seriously, then policymakers should prioritize investing in alignment and mechanistic interpretabality research to be able to read models’ minds, preparing society for rapid replacement in both white and blue collar jobs, and increasing scrutiny and transparency of frontier labs. If AI is more akin to previous technological breakthroughs, policymakers should view AI through the lens of existing sectors of regulation, while also looking to increase steady-state governance capacity to track AI progress, craft labor market cushions, and increase the feedback loop between innovation and diffusion.

In my view, I think both perspectives need to be kept in mind. I don’t see obvious reasons for AI progress to slow down anytime soon, but perhaps it will take years for institutions to adapt and fully leverage the technology. I think that alignment discourse might be overstating the potential of an AI model to want to destroy humanity, but that the disruptive effect of powerful AI models in the hands of malicious users, or the possibility that some models gain interests that are not in line with their creators and wreak havoc, is very real and should be addressed.

Conclusion

When past technological upheavals threatened to outrun society, leaders reached for new playbooks. Pope Leo XIII, confronting the Industrial Revolution, responded with Rerum Novarum (1891), also known as “Rights and Duties of Capital and Labor,” a statement that reframed Catholic social teaching for industrial capitalism.

Last week, after being chosen to serve as the 267th leader of the Catholic Church, then-Cardinal Robert Prevost considered what his papal name would be. He thought about his own pontificate’s place in history and the role he could play in guiding the Church’s 1.4 billion adherents through the next few decades.

By adopting the name Leo XIV, he signaled his conviction that coming technological changes may reshape society as profoundly as the Industrial Revolution once did:

“In our own day, the Church offers to everyone the treasury of her social teaching in response to another industrial revolution and to developments in the field of artificial intelligence that pose new challenges for the defense of human dignity, justice and labour,” Pope Leo XIV said in one of his first papal addresses to the College of Cardinals. Indeed, he laid out AI as one of the primary challenges facing humanity today.

Just as Leo XIV chose his name to signal that the Church will meet the AI age head-on, the rest of us need to decide which playbook we’re willing to follow. AI 2027 warns of a two-year sprint toward super-intelligence; AI as Normal Technology sketches a slower diffusion that still transforms the nature of work. One of those stories—or a blend of them—will shape the budgets we pass, the guardrails we build, and the jobs we train for.

I encourage you to read both papers (linked here and here) and come to your own conclusions. It’s well worth an hour or two of your time to see what expert forecasters and computer scientists think might be ahead of us.