Invest Like the Best · Podcast summary
Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]
Neil Movva, founder of Sale Research, discusses his company's mission to make AI inference 10x cheaper, focusing on long-running AI agents and the underlying hardware and software optimizations required to achieve this goal.
![Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]](https://megaphone.imgix.net/podcasts/d56fad4c-876e-11f1-9151-5f1834daf707/image/de9954428adab72e672734106004bdcb.jpg?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress)
Listen on the show ↗Key takeaways
- 💰Sale Research aims to make AI inference 10x cheaper, believing this will unlock new product categories and enable a future of abundant, proactive intelligence.03:09
- 🚀The rise of open-source AI models and the shift towards long-running, background AI agents are key tailwinds for Sale Research's strategy.05:31
- 📉The biggest inefficiencies in AI today are in attention mechanisms and memory usage (KV cache), where data is often uncompressed and underutilized, rather than in raw compute scaling.06:03
- 🧠The future of AI involves proactive, intelligent agents that operate in the background, developing an encyclopedic view of a user's life and proactively surfacing useful actions, enabled by incredibly cheap intelligence.11:11
- ⚙️Sale Research optimizes across the entire stack—software, hardware, and power—to achieve ultra-low-cost inference, starting with squeezing maximum performance from GPUs at the kernel level.15:12
- ⚖️A fundamental trade-off exists between throughput and latency on GPUs; Sale Research prioritizes throughput for long-running background agents, contrasting with the current latency-focused chatbot paradigm.20:37
Chapters
- 3:39Introduction to Sale Research
- 5:09The Vision for Abundant Intelligence
- 8:51The Opportunity: Open Source and Long-Running Agents
- 17:24The Future: Proactive Intelligence and Unbounded Markets
- 25:12The Technical Stack: Software, Hardware, and Power
- 25:31Software: Squeezing Performance from GPUs
- 40:19Hardware: Beyond Bleeding Edge
- 1:23:47Data Centers: Distributed and Resilient
- 1:34:53The Scavenger's Advantage
- 1:46:47Culture and Hiring
- 1:49:50Open Source vs. Closed Source
- 1:53:31The Future of AI
- 1:55:31Divergent Views and Closing Thoughts
The digest
Neil Movva, founder of Sale Research, joins Patrick O'Shaughnessy on "Invest Like the Best" to discuss his ambitious mission: making AI inference 10 times cheaper. Movva believes that by drastically reducing the cost of AI tokens, Sale Research can unlock new product categories and enable a future of proactive, intelligent agents. This conversation delves deep into the technical and strategic decisions behind building a "token factory" designed for a specific kind of future where AI agents run in the background for extended periods.
The Vision: Abundant Intelligence and the Token Factory
Movva begins by outlining Sale Research's core offering: an API that allows anyone to use large language models (LLMs) for any task at an unbeatable price. He emphasizes that the company is building a "token factory" – an inference company designed for a future where AI agents run for hours, days, or even weeks at a time. In this world, latency matters less, and cost matters much more. Movva states, "The theme of our company is abundance. We want to deliver this new commodity of intelligence to as many people as possible at a cost that is sustainable for almost every industry." He believes that making something 10 times cheaper creates a new product category, and Sale Research aims to do this for AI tokens.
Patrick clarifies that Sale Research is serving a specific kind of inference, aiming to be the absolute cheapest provider and an enabler of a certain type of intelligence use. Movva agrees, highlighting that their "North Star" is to have the lowest cost per token in the industry, by a mile. He anticipates a shift from focusing on individual tokens to "outcomes," where agents perform a unit of work (e.g., taking many shots on goal) and the number of tokens used to achieve that outcome becomes a dependent variable.
Keep reading in The Digest
The full 6-minute article, 7 more sections, and every new episode from Invest Like the Best and the creators you follow.
- The Opportunity: Open Source and Long-Running Agents
- The Technical Stack: Software, Hardware, and Power
- Software: Squeezing Performance from GPUs
- Hardware: Beyond Bleeding Edge
- Data Centers: Distributed and Resilient
- The Future: Proactive Intelligence and Unbounded Markets
- Conclusion: The Scavenger's Advantage
More from Invest Like the Best
Michael Moritz - Lessons From 40 Years of Investing and Writing - [Invest Like the Best, EP.491]
Michael Moritz, legendary venture capitalist and author, shares lessons from his 40-year career in investing and writing, drawing insights from his family history and observations of human nature.
Walter Russell Mead - How America Keeps Winning - [Invest Like the Best, EP.490]
Walter Russell Mead discusses the enduring power of the US, the unique method of Donald Trump, and the future of global power dynamics in an era of rapid technological change.
Sarah Guo - What the 250 People Building AI Believe - [Invest Like the Best, EP.489]
Sarah Guo, founder of Conviction, discusses the current state of AI, the future of robotics, and the investment philosophy that guides her firm.
Ben Thompson on Big Tech, China, and the AI Boom Running Out of Money - [Invest Like the Best, EP.487]
Ben Thompson, author of Stratechery, discusses the implications of AI on geopolitics, the economy, and the future of technology, including the potential for an AI boom to run out of money.
Eric Vishria - A Decade of Lessons Investing in Software & Hardware - [Invest Like the Best, EP.486]
Eric Vishria of Benchmark Capital shares his decade of lessons investing in software and hardware, including the shifting competitive frontier, the rise of AI, and the future of compute.
Gavin Baker - AI Market Jitters - [Invest Like the Best, EP.485]
Gavin Baker of Baker Technology Partners returns to discuss the AI market's recent jitters, the accelerating pace of innovation, and the surprising resilience of the underlying fundamentals despite a challenging month.