Y Combinator · Video summary

What If We Stopped Using GPUs? | YC Paper Club

YC Paper Club explores the future of computing beyond GPUs, delving into alternative architectures like optical and neuromorphic computing, and discussing novel optimization methods inspired by the human brain.

Y Combinator · October 2, 2026 · 81-min episode · 1-min read

What If We Stopped Using GPUs? | YC Paper Club

Key takeaways

📉The efficiency gains of GPUs in terms of gigaflops per joule have plateaued in the last two years, signaling a need for alternative computing methods.03:20
🧠The human brain operates on only 20 watts, demonstrating an incredible level of computational efficiency that current AI hardware cannot match.04:03
💡Optical computing leverages photons' low loss and high bandwidth, offering a potential for instantaneous computation with minimal energy consumption.07:07
✨SPSA, a zero-order optimization method, can optimize non-differentiable loss landscapes and does not rely on gradients, making it suitable for architectures not designed for backpropagation.09:52
🧩The 'soma sharded optimization mixture of assemblies' (SOMA) proposes training small, specialized expert models on sharded datasets, leveraging the benefits of zero-order optimizers and distributed compute.11:13
⚡A major challenge for optical computing is the energy cost of converting digital electronic data to analog optical signals and back.20:26

Chapters

0:00Introduction to Alternative Compute
0:01Discussion and Q&A on Optical Computing
0:06Ilker's Presentation: Optical Computing for Image Generation
0:10The Evolution of Deep Learning and GPUs
0:23Optical Diffusion Models for Image Generation
0:27Challenges in Optical Computing
0:27Alternative Optimizers: Beyond Backpropagation
0:30The Brain as an Efficient Computing Model
0:38Optical Computing: The Promise of Light
0:42Advantages of Photons for Computation
0:46Introducing the Speakers
0:46Experimental Results and Scaling Laws
0:49Limitations of Current GPU Architectures
0:55SOMA: Sharded Optimization Mixture of Assemblies
0:59Alok's Introduction to Neuromorphic Computing

The digest

For decades, GPUs have been the workhorse of modern computing, especially in the realm of AI. However, as the demands of increasingly complex models like transformers push the limits of current hardware, and the efficiency gains of GPUs begin to plateau, the question arises: what if we stopped using GPUs? This question, posed at a YC Paper Club event, sparked a fascinating discussion on alternative compute paradigms, drawing inspiration from the efficiency of the human brain and the potential of light-based computation.

The GPU Era and Its Limitations

The journey of deep learning, from AlexNet in 2012 to the rise of transformers, has been closely intertwined with the evolution of GPUs. Initially, GPUs were optimized for compute efficiency, particularly for convolutional neural networks (CNNs) which were not memory or bandwidth-bound. However, the advent of transformers, with their attention mechanisms, dramatically shifted the focus towards memory capacity and bandwidth. This led to a rapid increase in memory demands, with attention scaling quadratically with input size. While GPUs have adapted, the rate of improvement in gigaflops per joule has significantly slowed in recent years, indicating a potential ceiling for current architectures. This stagnation highlights the urgent need for new approaches to computation that can overcome these limitations.