OpenAI · Video summary
Stop Overpaying for Intelligence | DevDay 2026
OpenAI's AI deployment engineering team shares strategies for optimizing AI costs, focusing on shifting from a cost-per-token to a cost-per-task mindset and leveraging various technical levers.
Watch on YouTube ↗Key takeaways
- 💰The primary metric for optimizing AI costs should be 'cost per task', not 'cost per token', as cheaper models may require more tokens or human intervention.01:51
- 🎯Define the task and expected accuracy crisply, then use a Pareto curve to find the model and configuration that achieve the desired accuracy at the best cost.05:25
- 🚀Don't underestimate Luna; with high reasoning effort, it can often outperform larger, older models for 80-90% of workflows, a concept known as 'Luna maxing'.08:04
- 🛠️Programmatic tool calling can reduce input tokens and model calls by offloading data processing to code, sending only a packaged report to the LLM for judgment.11:52
- 🧠Adjusting reasoning effort is important; start with lower effort for routing tasks and evaluate, paying for higher effort only for difficult, differentiated tasks.13:22
- ⏳For non-urgent tasks, use batch or flex API processing to get 50% lower token pricing with a 24-hour processing window.14:09
Chapters
- 0:13Introduction to AI Cost Optimization
- 0:41Right-Sizing a Model for a Particular Task
- 0:58The Number One Mistake: Focusing on Cost Per Token
- 3:22Real-World Examples of Cost Per Task
- 8:56Understanding the Pareto Curve
- 10:32Choosing the Right Model Family
- 12:24The Power of Luna Maxing
- 14:51Levers Beyond the Model Itself
- 15:43Lever 1: Prompt Caching
- 17:05Lever 2: Programmatic Tool Calling
- 21:32Lever 3: Reasoning Effort
- 22:28Lever 4: Batch or Flex API Processing
- 27:21Optimizing Prompt Cache Hit Rate
- 30:44Advanced Prompt Caching Techniques
- 35:13Key Takeaways and Recommendations
The digest
Optimizing the cost of AI deployments is crucial for businesses and developers alike. While the initial instinct might be to focus on the list price of input and output tokens, a more effective approach involves a fundamental shift in how we measure and manage AI expenses.
The Right Metric: Cost Per Task
The most common mistake developers make is focusing solely on the cost per token. A model that appears cheap on a per-token basis may end up being more expensive overall if it consumes more tokens to complete a task or, worse, fails to complete the task, requiring human intervention. The true metric to optimize for is cost per task.
Consider a customer support chatbot. If a cheaper model fails to resolve a customer's query, a human agent must step in. The actual cost of that task then includes not only the tokens used by the chatbot but also the time and resources spent by the human. This principle applies across various applications. For instance, Perplexity reported that Astra 6.0 produced 9% more accurate results at half the cost compared to previous models, despite potentially having a higher per-token price. Similarly, Notion saw better accuracy and half the cost per task by migrating to newer models like 5.6 Sol, thanks to improved token efficiency and task completion rates.
Keep reading in The Digest
The full 4-minute article, 8 more sections, and every new episode from OpenAI and the creators you follow.
- Right-Sizing Your Model for the Task
- Leveraging Luna Maxing
- Levers Beyond the Model Itself
- 1. Prompt Caching
- 2. Programmatic Tool Calling
- 3. Reasoning Effort
- 4. Batch or Flex API Processing
- Key Takeaways for Developers
More from OpenAI
ClawLabs: Building Open Source Together | DevDay 2026
ClawLabs demonstrates how open-source development is evolving with AI agents, showcasing collaborative workflows and the new OpenClaw Enterprise platform for secure, scalable agent deployments.
From Single Player to Multiplayer with Codex | DevDay 2026
Jason from OpenAI's developer experience team discusses the evolution from individual AI use to collaborative, multiplayer AI systems, highlighting new tools and strategies for team-wide productivity.
Builders Unscripted: Ep. 4 - Pietro Schirano
In this episode of Builders Unscripted, Pietro Schirano discusses his multifaceted approach to creativity and innovation, highlighting how advancements in AI—particularly with models like GPT 5.5—enable him to explore new possibilities in design, music, and technology. He emphasizes the democratization of tools that allow anyone to become a builder, fostering a collaborative environment between humans and AI.
OpenAI's CFO Presents the Future of Finance with University of California’s Chief Investment Officer
In this video, OpenAI's CFO Sarah Frier discusses the transformative impact of AI on the finance industry, emphasizing the importance of curiosity, adaptability, and kindness for future professionals. She highlights how AI tools are revolutionizing finance tasks, enabling teams to work more efficiently while fostering a culture of innovation and learning.
Episode 15 - Inside the Model Spec
The model spec is OpenAI's public document outlining desired AI behavior, balancing user empowerment with societal protection through a hierarchical chain of command and iterative refinement.
Building AI for better healthcare — the OpenAI Podcast Ep. 14
OpenAI's Karan Singhal and Dr. Nate Gross discuss AI's transformative potential in healthcare, focusing on patient empowerment, clinician support, and system efficiency.