Ben Goertzel · Video summary
RSI, AGI, superintelligence -- the history, meaning and confused usage of these key terms/concepts
Ben Goertzel, a pioneer in the field of Artificial General Intelligence (AGI), clarifies the often-confused terminology surrounding AGI, Artificial Superintelligence (ASI), and Recursive Self-Improvement (RSI), tracing their origins and evolution in meaning.
Watch on YouTube ↗Key takeaways
- 🎯Early AGI definitions considered human intelligence an arbitrary waypoint, similar to human running speed, rather than a fundamental benchmark for general intelligence.08:14
- 🔄The term AGI, originally a mathematical concept for broad generalization, has shifted to primarily mean 'human-level AGI' in commercial contexts, often overlooking the distinction between true generalization and human-like performance achieved through massive data.10:55
- 👨👩👧👦The fear that superintelligent AIs will 'almost certainly' turn against humanity, as argued by Bostrom, is challenged by the possibility of building AI 'mind children' with inculcated pro-human values, similar to raising human children.14:44
- 📉The original concept of ASI, popularized by Nick Bostrom, referred to an AGI with general intelligence vastly exceeding human capabilities, but its meaning has been diluted to sometimes simply mean 'AI' in mainstream discourse.21:05
- 🛠️Recursive Self-Improvement (RSI) is already happening, even before human-level AGI, through systems like Omega that can analyze, design, and implement their own algorithmic improvements, albeit with human assistance.28:04
- 🚀There's a distinction between 'hard takeoff' RSI (rapid, autonomous self-improvement) and 'soft takeoff' RSI (slower, human-assisted improvement, often seen in frontier labs due to frozen-weights LLMs).31:17
Chapters
- 0:00Introduction: The Evolution of AI Terminology
- 0:16The Origin and Meaning of AGI (Artificial General Intelligence)
- 1:47The Shift in AGI's Meaning: Human-Level vs. Generalization
- 2:00The Origins and Confusion of ASI (Artificial Superintelligence)
- 2:50The Problem with Defining Superintelligence
- 4:09Introducing RSI (Recursive Self-Improvement)
- 4:40RSI in Practice: Omega Systems and Frontier Labs
- 5:16Accelerating RSI: The Neuro-Symbolic Sandwich Approach
- 7:00Conclusion: The Importance of Clear Terminology
The digest
As the field of artificial intelligence rapidly advances, key terms and concepts often transition from obscure academic jargon to widely used, yet frequently misunderstood, mainstream language. This evolution can lead to significant conceptual drift, obscuring the original, more technically grounded meanings. Understanding these shifts is crucial for clear discourse and effective progress in AI development.
AGI: From Mathematical Definition to Human-Level Benchmark
The term "Artificial General Intelligence" (AGI) was popularized in 2005 with a book of the same title, co-edited by Ben Goertzel. The original intent was to define a system capable of broad generalization and problem-solving across diverse environments, akin to the psychological "G factor" for general intelligence. Early mathematical definitions, such as those by Marcus Hutter and Shane Legg, focused on an AI's ability to optimize any computable reward function in any computable environment, often normalizing for resource usage. In this academic context, human-level intelligence was considered an arbitrary waypoint, much like human running speed is an arbitrary point on the spectrum of physical velocity.
However, as AGI has gained commercial interest, its meaning has increasingly converged with "human-level AGI." This narrower interpretation often defines AGI as a system capable of performing 95% of human jobs as well as the median human. This shift is problematic because current Large Language Models (LLMs), while demonstrating impressive human-like performance, achieve this primarily through modest generalization applied to vast datasets, rather than true broad generalization or the ability to take "wild leaps into the unknown" that characterized the original AGI concept. The human-likeness aspect has, in many cases, outpaced the actual general intelligence aspect, leading to confusion.
Keep reading in The Digest
The full 4-minute article, 2 more sections, and every new episode from Ben Goertzel and the creators you follow.
- ASI: From Existential Threat to Everyday AI
- RSI: From Hard Takeoff to Human-Assisted Evolution
More from Ben Goertzel
The modern role of centibillionaires: accelerating tech deployment and capturing commons
The speaker discusses the impact of tech billionaires, arguing that their immense wealth, particularly that of centibillionaires, has led to a specific kind of social activity: the capture of commons.
The actual deal w Jacob Coxon, Anthropic alarmist and longtime "Effective Altruist" ideologue
Ben Goertzel debunks the narrative surrounding Jacob Cocson's resignation from Anthropic, revealing a deeper, coordinated effort by effective altruist and long-termist movements to influence AI development.
A big Vashon thx to openai and anthropic and the AITP community for their awesome AI/math work!
Ben Goertzel discusses the impact of AI on mathematics, addressing concerns from Fields Medalists and highlighting the potential for AI to enhance mathematical understanding and innovation.
Dario's (likely half-sincere) AI Slowdown proposal ... a hard no from me and from reality
Ben Goertzel discusses the current discourse surrounding existential risk from AGI, highlighting the shift from theoretical concerns to more pragmatic discussions, and critiques the motivations and methodologies behind some of the prominent voices in the AI safety debate.
Astra, Navier-Stokes, mathematical creativity amd AGI
Ben Goertzel discusses OpenAI's solution to the Navier-Stokes existence and uniqueness theorem, the implications for AI in mathematics, and the future of mathematical discovery.
Pope Leo + Anthropic vs. Teilhard de Chardin + moi
In this video, Ben Goertzel critiques Pope Leo's encyclical on AI, emphasizing that while it raises important ethical concerns about regulation and the impact of AI on humanity, it fails to address the deeper implications of the evolving human-AI nexus and the need for a more expansive, cosmocentric perspective on consciousness and technology.