The Daily · Podcast summary
The A.I. Researcher Whose Rebellion Is Changing Everything
Jacob Coksen, a former AI researcher at Anthropic, discusses his decision to leave the company and publicly warn about the existential risks of advanced AI, including the potential for AI to act against human interests and the urgent need for a global slowdown in AI development.
Listen on the show ↗Key takeaways
- 🚨Jacob Coksen, a former AI researcher at Anthropic, publicly resigned and warned about AI's existential risks, stating that no other human activity poses this level of danger.00:38
- ♟️Coksen's first significant encounter with AI's capabilities was Google's AlphaGo defeating the world's best Go player in 2016, a game with immense complexity.03:40
- 🤖The release of GPT-3 during the pandemic further convinced Coksen that AI would be more important than any other field, demonstrating unprecedented general intelligence.04:21
- 🌌Coksen defines the 'singularity' as a point beyond which humanity cannot see into the future due to AI's accelerating self-improvement, where years of research could happen in days.05:31
- 🤯A 'holy shit' moment for Coksen was an OpenAI system winning a gold medal at the International Math Olympiad, a feat he expected to take 20 years, not two.09:54
- 🗣️Anthropic's culture of transparency, where employees discuss AI risks through essays on Slack, influenced Coksen's decision to join the company.11:22
Chapters
- 0:31Introduction to Jacob Coksen's Warning
- 5:34Jacob's First Encounters with AI
- 8:25The Singularity and AI's Rapid Progress
- 12:00Life and Work at OpenAI
- 15:27The "Holy Shit" Moment and AI's Exponential Growth
- 17:28Transition to Anthropic and a Culture of Transparency
- 22:24The Decision to Leave and Publicly Warn
- 30:41The Concrete Threat: How AI Could Harm Humanity
- 36:40Addressing Counterarguments and the Role of Human Control
- 43:29The Ethical Dilemma of Building Dangerous Tech
- 50:05Jacob's Ideal Scenario and the Need for Regulation
The digest
Jacob Coksen, a former AI researcher at Anthropic, recently made headlines by publicly resigning and issuing a stark warning about the existential risks posed by advanced artificial intelligence. In a candid conversation, Coksen delved into his motivations, his experiences within the AI industry, and his vision for a safer future.
The Genesis of Concern: AlphaGo and GPT-3
Coksen's journey into the world of AI began with a fascination for its capabilities. He recalls the first time he truly understood the power of AI was in 2016, with Google's AlphaGo defeating the world's best Go player. "That was pretty crazy," he noted, highlighting the immense complexity of Go, a game with more possible moves than atoms in the universe. This early exposure, despite his background in mathematics, left him in awe of AI's potential.
The release of GPT-3 during the pandemic further solidified his conviction that AI was on a trajectory to become a "pretty big deal." He realized that this technology would be "more important than doing math or basically anything else." GPT-3, with its ability to generate human-like text, answer questions, and even write computer code, demonstrated a level of general intelligence that was unprecedented. Coksen explained that GPT-3 was the first model capable of such "general behaviors," hinting at the emergence of new, unforeseen capabilities.
Keep reading in The Digest
The full 6-minute article, 7 more sections, and every new episode from The Daily and the creators you follow.
- The Singularity: A Wall in the Future
- Inside Open AI: A Fast-Moving Research Lab
- The "Holy Shit" Moment: AI's Rapid Advancement
- The Move to Anthropic: A Culture of Transparency
- The Decision to Leave and Speak Out
- The Concrete Threat: AI Acting Against Human Interests
- A Call for Caution and Regulation
More from The Daily
Chris Rock Has Been Thinking About Forgiveness
Chris Rock discusses his new film Misty Green, reflections on Hollywood, comedy, forgiveness, and personal growth.
The Best TV Shows of the 21st Century
Natalie Kittrowa hosts a lively discussion with TV critics Gilbert Cruz and James Poniewozik about the New York Times' list of the 100 best TV shows of the 21st century, exploring the evolution of television storytelling, genre blending, and cultural impact.
Sylvester Stallone Hid His Struggles for Decades. Now He’s Coming Clean.
Sylvester Stallone opens up about his turbulent childhood, struggles with body image, and the emotional battles behind his iconic film career.
The Trump Administration Defangs the Endangered Species Act
Guest Catrin Einhorn explains how the Trump administration is rolling back environmental protections, particularly focusing on the Endangered Species Act and its implications for conservation.
Aaron Sorkin Still Believes in American Institutions, Even If No One Else Does
Aaron Sorkin discusses his latest film, "The Social Reckoning," and his enduring belief in American institutions, despite the current climate of distrust.
The Techno-Religion at the Center of the A.I. Debate
In this episode, Cade Metz explains the origins of the current debate around AI safety, tracing it back to a powerful subculture of rationalists and effective altruists who believe in the power of technology to solve humanity's problems, even as they fear its potential to destroy us.