a16z Podcast · Podcast summary
Why Medical AI Needs a Referee | Protege's Engy Ziedan
Protege co-founder and Chief Scientific Officer Engy Ziedan discusses why healthcare AI needs a much better way to measure performance, the gap between benchmarks and real-world clinical tasks, and what happens when models evolve faster than the healthcare system can evaluate them.
Listen on the show ↗Key takeaways
- 🤔Hundreds of millions of people ask ChatGPT health questions, but there's no one making sure the answers are safe and correct.00:00
- 🚫AI models are limited by the training data available, and current evaluations don't look beyond catastrophic failures.00:11
- 💰In today's AI market, there's a need for accurate pricing and understanding what's valuable or valueless, as AI doesn't have a marginal cost of zero.00:30
- 🏥A medical AI model can ace thousands of test questions but still fail at real-world jobs, highlighting the gap between benchmarks and clinical tasks.00:53
- 🩺The importance of evaluations in healthcare AI stems from the high stakes and the need to vet AI just like an employee.07:57
- ⚖️Healthcare has an information asymmetry where providers know more than patients, and AI agents can exacerbate this without proper oversight.08:26
Chapters
- 0:00The Problem with Current AI Evaluation
- 0:30The Value of AI in Healthcare: Beyond Hedonics
- 0:53The Asymmetry of Information and High Stakes
- 15:56Beyond Catastrophic Failures: Misalignment and Subtle Bias
- 22:07The Need for an Independent Referee
- 25:19The Limitations of Benchmarks and the Importance of Real-World Context
- 39:03Protege's Approach: Continuous Testing and Impartiality
The digest
In a recent A16Z podcast, Daisy Wolf and Eva Steinman sat down with Protege co-founder and Chief Scientific Officer Engy Ziedan to discuss the critical need for a robust evaluation system in healthcare AI. Ziedan, a healthcare economist and assistant professor at Indiana University, highlighted the inherent challenges in assessing AI models, particularly in a field where decisions can have life-or-death consequences.
The Problem with Current AI Evaluation
Ziedan began by pointing out a fundamental flaw in how AI models are currently evaluated. Hundreds of millions of people ask ChatGPT health-related questions daily, and doctors use AI tools in their practice. However, there's a significant lack of oversight to ensure the answers provided are safe and correct. Ziedan emphasized that models are "inhibited in their usefulness by the training data available for them." She argued that no one is truly looking beyond catastrophic failures and misalignment to ensure the broader safety and efficacy of these AI systems.
Keep reading in The Digest
The full 4-minute article, 6 more sections, and every new episode from a16z Podcast and the creators you follow.
- The Value of AI in Healthcare: Beyond Hedonics
- The Asymmetry of Information and High Stakes
- Beyond Catastrophic Failures: Misalignment and Subtle Bias
- The Need for an Independent Referee
- The Limitations of Benchmarks and the Importance of Real-World Context
- Protege's Approach: Continuous Testing and Impartiality
More from a16z Podcast
Beyond the God Model | Alex Atallah & Amjad Masad
Alex Atallah of OpenRouter and Amjad Masad of Replit explore the future of AI through specialized models, neurodiversity, and enterprise independence.
AI Can Write Code. Why Isn’t Software Better?
Diogo Almeida, founder of TypeSafe AI, reimagines AI's role in software by introducing JEV, a new primitive that embeds intelligence within software to enable true automation beyond faster coding.
Building a Team at AI Speed | Harvey’s Maggie Landers
Maggie Landers, VP of Talent at Harvey, shares how the company hires and scales culture at AI speed while transforming the legal industry.
AI Safety Language Is Destroying the Debate | Steven Sinofsky
Steven Sinofsky, former Microsoft Windows President and A16Z board partner, argues that the current language around AI safety is making the technology harder to understand and hindering productive debate.
Nas, Grandmaster Caz, Steve Stoute & Ben Horowitz on Paying Hip-Hop’s Pioneers Their Due
Nas, Grandmaster Caz, Steve Stoute, and Ben Horowitz discuss the Payday Full Foundation's mission to honor hip-hop pioneers by providing both financial support and recognition, highlighting the industry's historical oversight and the profound impact this initiative has on artists' lives and the culture at large.
What Makes a Consumer AI Product Stick? | Josh Elman
A16Z partner Josh Elman joins new economies to discuss what it takes to build a lasting consumer product in the AI era. Josh explains why the best products start by doing one thing exceptionally well, become a habit, and then earn the right to do more.