SCLATE: A Substrate for Continual-Learning Agent Training and Evaluation
- Apple Machine Learning Research research area Methods and Algorithms , research area Tools, Platforms, Frameworks content type paper published September 2026 SCLATE: A Substrate for Continual-Learning Agent Training and Evaluation Authors Youngmok Jung, Sirajul Salekin, Henry.
- Evaluating and training them requires interleaving tasks with agent-side events such as session stop and start, crons, and memory consolidation.
- We then post-train Qwen3.5-4B through unmodified harnesses and memory systems.
- The model learns to use both, reading 6.8× fewer file lines with a 16.7-point higher SWE-bench Verified pass rate, and writing richer memory records, while its held-out MetaClaw accuracy rises by up to 11.8 points.
- Apple Machine Learning Research research area Methods and Algorithms , research area Tools, Platforms, Frameworks
content type paper published September 2026
SCLATE: A Substrate for Continual-Learning Agent Training and Evaluation
Authors Youngmok Jung, Sirajul Salekin, Henry Tran, Javier Movellan, Zhao Huang, Manjot Bilkhu
Continual-learning agents are systems of models, harnesses, and memory operating over long multi-session horizons. Evaluating and training them requires interleaving tasks with agent-side events such as session stop and start, crons, and memory consolidation. Yet existing benchmarks and training frameworks schedule only the benchmark’s own events, leaving each benchmark and agent pair to build a custom scheduling loop. We present SCLATE, an execution substrate where benchmarks and unmodified agents each add their events to one open event scheduler through an adapter. A hybrid simulated clock runs these events on a shared timeline, flowing in real time while the agent works and skipping idle gaps, which compresses a month-long scenario into hours.
SCLATE also serves as a rollout engine that runs any agent’s harness and memory unmodified, recording the tokens and log probabilities of every model call through an in-container proxy. We port seven benchmarks to SCLATE and compare ten unmodified harness and memory configurations head to head on ten models. The comparison shows that an added memory system does not reliably beat the harness’s native memory and that models differ widely in how they use the same harness and memory. We then post-train Qwen3.5-4B through unmodified harnesses and memory systems. The model learns to use both, reading 6.8× fewer file lines with a 16.7-point higher SWE-bench Verified pass rate, and writing richer memory records, while its held-out MetaClaw accuracy rises by up to 11.8 points.
How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
October 1, 2026 research area Methods and Algorithms
Recent autonomous machine learning engineering (MLE) agents have made significant progress on public leaderboards. Often motivated by progress stagnation over long-horizon cycles and limited Large Language Model (LLM) primitives, modern MLE agents are deployed on top of increasingly elaborate machinery: multi-agent orchestrators, dedicated retrieval subagents, and more. While such harnesses expand, the use of more primitive but improved coding…
AgentBuilder: Exploring Scaffolds for Prototyping User Experiences of Interface Agents
January 9, 2026 research area Human-Computer Interaction
Interface agents powered by generative AI models (referred to as “agents”) can automate actions based on user commands. An important aspect of developing agents is their user experience (i.e., agent experience). There is a growing need to provide scaffolds for a broader set of individuals beyond AI engineers to prototype agent experiences, since they can contribute valuable perspectives to designing agent experiences. In this work, we explore the…
Discover opportunities in Machine Learning.
Our research in machine learning breaks new ground every day.
Sources
Related stories
Automating coherent long-form video generation
Yale Song and Yiwen Song, Research Scientists, Google We introduce a unified multi-agent framework that autonomously generates temporally consistent, long-form video narratives, overcoming the identity drift and cascading failures of current linear AI pipelines. Recent.
ToolGrad: Efficient tool-use dataset generation with textual "gradients"
Zhongyi Zhou, Research Scientist, and Ruofei Du, Interactive Perception & Graphics Lead, Google XR ToolGrad is a data generation framework that reverses the traditional paradigm by first generating tool-use answers before user queries. We show this design enables LLMs to achieve.
NVIDIA DGX Spark 64GB Gives Developers More Ways to Build and Scale Local AI
Local AI is becoming more useful by the token. As AI agents move from experiments into everyday development, increasingly capable open models are shrinking to fit on more devices, giving builders more to run locally.
DeskForge Corpus Improves Computer-Use Agent Accuracy
A new corpus of annotated desktop observations, DeskForge, enables computer-use agents to achieve higher accuracy in complex desktop scenes.
China Telecom's Xing4.0 Runs a 29B Coding Agent on 19GB Locally
Subtopic Mixture Of Experts · Long Context · Vision Language China Telecom released Xing4.0-29B-A4B , a 29B MoE with only 4B active parameters. Native 256K context extensible to 512K using MLA attention and 64 routed experts.
Red Hat Shrinks Nemotron 3.5 Lightning's 30B Agent Model by Half With FP8
Red Hat AI released an FP8 quantized build of NVIDIA Nemotron 3.5 Lightning 30B A3B. Cuts GPU memory and disk by roughly 50% versus the BF16 reference weights.
Fine-tune a search agent with multi-turn RL on Amazon SageMaker AI
Search agents powered by large language models (LLMs) are transforming how enterprises retrieve information.
How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering? - Apple Machine Learning Research research area Methods and Algorithms content type paper published October 2026 How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?