Running Bigger AI Models, More Efficiently | SambaNova
TL;DR AI has moved from experimentation into production, and that shift exposes three hard constraints: models keep getting bigger, cost climbs steeply at…
- TL;DR AI has moved from experimentation into production, and that shift exposes three hard constraints: models keep getting bigger, cost climbs steeply at scale, and power becomes a physical ceiling.
- Inference in the agentic era behaves nothing like the single-model throughput problem of the past.
- SambaNova's focus is premium inference, defined on the panel by speed and size: Running the largest models much more efficiently, at full precision, rather than trading accuracy for speed.
TL;DR AI has moved from experimentation into production, and that shift exposes three hard constraints: models keep getting bigger, cost climbs steeply at scale, and power becomes a physical ceiling. On a recent Fortune panel, SambaNova CEO Rodrigo Liang and Adaption Labs CEO Sara Hooker agreed the industry's central problem is now efficiency, though they approach it from different angles: Liang from the infrastructure side, Hooker from the model-architecture side. Large models are not going away. For the most demanding workloads, they are unavoidable, so the real question is how to run them efficiently rather than whether to run them at all. Inference in the agentic era behaves nothing like the single-model throughput problem of the past. In Hooker's words, “inference is a different beast.” Constant data movement, not raw compute, is the bottleneck. SambaNova's focus is premium inference, defined on the panel by speed and size: Running the largest models much more efficiently, at full precision, rather than trading accuracy for speed. Are Large Models Here to Stay? The question hung over the whole conversation: Are today's largest foundation models where AI is heading, or will we look back on them as a detour? A recent Fortune panel titled “From the AI We Have to the AI We Need” put it directly to two people building very different answers, SambaNova co-founder and CEO Rodrigo Liang and Adaption Labs co-founder and CEO Sara Hooker, moderated by Fortune AI editor Jeremy Kahn.
Sources
Related stories
Inside-Out AI: Rebuilding Airbnb Behind the Scenes and Across the Guest Experience
Prior to joining Airbnb as CTO in January, Ahmad Al-Dahle was head of generative AI at Meta and led the launch of its open source Llama models over 2023-2025.
How to Build a Model Router in the Harness
How to Build a Model Router in the Harness How to Build a Model Router in the Harness Many tasks don't need frontier intelligence. In our experiments, model routing cut median cost per coding task by 64% compared with the baseline, with no noticeable change in quality.
New tool lets users repair AI-generated 3D models, then fabricate them just the way they want
“What you see is what you get” is a guiding principle for many software engineers — create programs where the content you’re editing looks the same as the…
What Is Jev? A Guide to TypeSafe AI’s System One Model
What Is Jev? A Guide to TypeSafe AI’s System One Model Agents run in a loop: an LLM decides what to do, a tool executes, a model evaluates the results, and then continues in that loop until the task is complete.