How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering? - Apple Machine Learning Research research area Methods and Algorithms content type paper published October 2026 How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
- Apple Machine Learning Research research area Methods and Algorithms content type paper published October 2026 How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
- Authors Kirill Brilliantov†‡, Alejandro Hernández-Cano†‡**, Emmanuel Abbé† Recent autonomous machine learning engineering (MLE) agents have made significant progress on public leaderboards.
- We conclude that the effort spent elaborating hand-crafted harnesses around strong models yields poor returns for current MLE benchmarks.
- Apple Machine Learning Research research area Methods and Algorithms content type paper published October 2026 How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering? Authors Kirill Brilliantov†‡, Alejandro Hernández-Cano†‡**, Emmanuel Abbé† Recent autonomous machine learning engineering (MLE) agents have made significant progress on public leaderboards. Often motivated by progress stagnation over long-horizon cycles and limited Large Language Model (LLM) primitives, modern MLE agents are deployed on top of increasingly elaborate machinery: multi-agent orchestrators, dedicated retrieval subagents, and more. While such harnesses expand, the use of more primitive but improved coding agents—where LLMs have direct access to the execution environment through read, write, and bash primitives—has received little attention in the field. In this paper we find that, under an equal time budget and the same frontier LLM backbone, open-source state-of-the-art harnesses provide no advantages over a single session of a minimal-harness coding agent baseline, pointing to the backbone as the primary driver for performance. Via a series of large-scale systematic ablation studies, we argue that the machinery layers become redundant in the coding agent setting. We conclude that the effort spent elaborating hand-crafted harnesses around strong models yields poor returns for current MLE benchmarks. SCLATE: A Substrate for Continual-Learning Agent Training and Evaluation September 30, 2026 research area Methods and Algorithms , research area Tools, Platforms, Frameworks Continual-learning agents are systems of models, harnesses, and memory operating over long multi-session horizons. Evaluating and training them requires interleaving tasks with agent-side events such as session stop and start, crons, and memory consolidation. Yet existing benchmarks and training frameworks schedule only the benchmark’s own events, leaving each benchmark and agent pair to build a custom scheduling loop. We present SCLATE, an… AgentBuilder: Exploring Scaffolds for Prototyping User Experiences of Interface Agents January 9, 2026 research area Human-Computer Interaction Interface agents powered by generative AI models (referred to as “agents”) can automate actions based on user commands. An important aspect of developing agents is their user experience (i.e., agent experience). There is a growing need to provide scaffolds for a broader set of individuals beyond AI engineers to prototype agent experiences, since they can contribute valuable perspectives to designing agent experiences. In this work, we explore the… Discover opportunities in Machine Learning. Our research in machine learning breaks new ground every day.
Sources
Related stories
DeskForge Corpus Improves Computer-Use Agent Accuracy
A new corpus of annotated desktop observations, DeskForge, enables computer-use agents to achieve higher accuracy in complex desktop scenes.
SCLATE: A Substrate for Continual-Learning Agent Training and Evaluation
SCLATE: A Substrate for Continual-Learning Agent Training and Evaluation - Apple Machine Learning Research research area Methods and Algorithms , research area Tools, Platforms, Frameworks content type paper published September 2026 SCLATE: A Substrate for Continual-Learning Agent Training and Evaluation Authors Youngmok Jung, Sirajul Salekin, Henry Tran, Javier Movellan, Zhao Huang, Manjot Bilkhu Continual-learning agents are systems of models, harnesses,...
Salesforce Signs Definitive Agreement to Acquire Listen Labs
Salesforce, the #1 AI CRM, today announced it has signed a definitive agreement to acquire Listen Labs, an AI-powered customer research and human simulation…
Automating coherent long-form video generation
Automating coherent long-form video generation Automating coherent long-form video generation Yale Song and Yiwen Song, Research Scientists, Google We introduce a unified multi-agent framework that autonomously generates temporally consistent, long-form video narratives, overcoming the identity drift and cascading failures of current linear AI pipelines.