a16z Podcast · Podcast summary
The Infrastructure Behind the Machine Age
Ben Horowitz, Raghu Raghuram, and Martin Casado discuss the launch of a16z's Machine Age Fund, dedicated to the infrastructure powering the next era of AI.
Listen on the show ↗Key takeaways
- 💡The current AI boom is unique because the bottleneck isn't the models themselves, but the underlying physical infrastructure like chips, memory, networking, power, cooling, and data centers.01:03
- 📈Demand for AI is "basically infinite," leading to unprecedented pressure on every part of the supply chain, with hyperscalers placing massive purchase orders that will reach $700 billion this year.05:30
- 💰Unlike previous tech booms, where infrastructure was built speculatively, every GPU being created now is already pre-sold, indicating that demand is outpacing supply at an unprecedented rate.06:51
- 🌍The demand for AI is expanding on two fronts: the unit of work (more tokens consumed per task) and the number of people who will benefit (knowledge workers, not just developers).07:09
- 🏗️The existing infrastructure was not designed for AI, necessitating not just more capacity, but entirely different kinds of infrastructure, as every component is reaching its physical limits.13:11
- ⚙️It now makes economic sense to build Application-Specific Integrated Circuits (ASICs) customized for individual AI models, as the cost of training a frontier model ($3-5 billion) justifies the investment in bespoke hardware for efficiency gains.27:41
Chapters
- 0:00Introduction to the Machine Age Fund
- 3:28The New Infrastructure for AI
- 6:40Unprecedented Demand and Supply Shortages
- 11:42The Expanding Universe of AI Use Cases
- 21:48Re-architecting for the Machine Age
- 47:01The Challenges of Building a New Infrastructure
- 1:03:49The Machine Age: A New Paradigm
- 1:14:15Investment Opportunities in the Machine Age
- 1:16:46The Founders of the Machine Age
- 1:28:12Conclusion: America Wins the Infrastructure Game
The digest
Ben Horowitz, Raghu Raghuram, and Martin Casado of a16z discuss the launch of the Machine Age Fund, a new initiative dedicated to building the foundational infrastructure for the next era of AI. They argue that the biggest bottleneck in AI isn't the models themselves, but everything underneath them: chips, memory, networking, power, cooling, and data centers. This new fund aims to address these critical infrastructure challenges.
A New Era of Technology Demands New Infrastructure
Ben Horowitz opens by stating that we are in a new technological era, one that demands an entirely new infrastructure. He emphasizes that this isn't just about servers, storage, and networks, but extends "all the way down to the mines of copper." This new infrastructure is experiencing unprecedented demand, requiring not only new chips and system software but also innovative approaches to power and even the replacement of traditional copper wiring. Horowitz notes that this is a "very exciting time" for hardware aspects, necessitating a fresh approach.
Raghu Raghuram echoes this sentiment, highlighting that in traditional computing, infrastructure discussions revolved around servers, storage, and networks. However, with AI, the focus shifts to the physical limits of hardware. He points out that the capabilities of AI models have steadily increased over the last three years, to the point where the model itself is no longer the bottleneck. Instead, the limitations lie "south of the model," in the underlying infrastructure.
Keep reading in The Digest
The full 6-minute article, 5 more sections, and every new episode from a16z Podcast and the creators you follow.
- Unprecedented Demand and Supply Shortages
- The Expanding Universe of AI Use Cases
- Re-architecting for the Machine Age
- The Challenges of Building a New Infrastructure
- The Machine Age: A New Paradigm
More from a16z Podcast
Beyond the God Model | Alex Atallah & Amjad Masad
Alex Atallah of OpenRouter and Amjad Masad of Replit explore the future of AI through specialized models, neurodiversity, and enterprise independence.
AI Can Write Code. Why Isn’t Software Better?
Diogo Almeida, founder of TypeSafe AI, reimagines AI's role in software by introducing JEV, a new primitive that embeds intelligence within software to enable true automation beyond faster coding.
Building a Team at AI Speed | Harvey’s Maggie Landers
Maggie Landers, VP of Talent at Harvey, shares how the company hires and scales culture at AI speed while transforming the legal industry.
AI Safety Language Is Destroying the Debate | Steven Sinofsky
Steven Sinofsky, former Microsoft Windows President and A16Z board partner, argues that the current language around AI safety is making the technology harder to understand and hindering productive debate.
Nas, Grandmaster Caz, Steve Stoute & Ben Horowitz on Paying Hip-Hop’s Pioneers Their Due
Nas, Grandmaster Caz, Steve Stoute, and Ben Horowitz discuss the Payday Full Foundation's mission to honor hip-hop pioneers by providing both financial support and recognition, highlighting the industry's historical oversight and the profound impact this initiative has on artists' lives and the culture at large.
What Makes a Consumer AI Product Stick? | Josh Elman
A16Z partner Josh Elman joins new economies to discuss what it takes to build a lasting consumer product in the AI era. Josh explains why the best products start by doing one thing exceptionally well, become a habit, and then earn the right to do more.