Compressing Streaming Neural Audio Encoders via Latent-Space Distillation
Compressing Streaming Neural Audio Encoders via Latent-Space Distillation - Apple Machine Learning Research research area Methods and Algorithms , research area Speech and Natural Language Processing content type paper published September 2026 Compressing Streaming Neural Audio Encoders via Latent-Space Distillation Authors Prasanth Yadla‡, Mohammad Samragh Razlighi‡, Dongseong Hwang, Mingbin Xu, Yuanyuan Zhang, Chung-Cheng Chiu, Yongqiang Wang†**, Yuan Liu§**, Zhen Huang,...
- We train only the student encoder to regress the teacher’s per-frame latent under a squared-error objective, with a single affine layer absorbing the teacher–student width mismatch.
- Because the target precedes both the quantizer and the language-model bridge, one recipe covers both token interfaces we support, and applies both to a tokenizer pretrained alone and to one jointly trained with a language model.
- At 2.8× compression the distilled student stays within 1.9% relative WER of its teacher on five of six teacher–student pairs without any fine-tuning, and improves on an independently trained tokenizer of identical capacity by 3.9% relative.
- Apple Machine Learning Research research area Methods and Algorithms , research area Speech and Natural Language Processing content type paper published September 2026 Compressing Streaming Neural Audio Encoders via Latent-Space Distillation Authors Prasanth Yadla‡, Mohammad Samragh Razlighi‡, Dongseong Hwang, Mingbin Xu, Yuanyuan Zhang, Chung-Cheng Chiu, Yongqiang Wang†, Yuan Liu§, Zhen Huang, Xiaodan Zhuang System-wide Dictation on Apple devices runs entirely on-device, and the speech it transcribes reaches the foundation model through a tokenizer: an encoder that maps short windows of waveform onto the representation the language model reads. Because that model is sparsely activated under Instruction-Following Pruning, only a small subset of its experts occupies DRAM at any time, so the always-on tokenizer competes for the same memory, and its parameter count bears directly on power and latency. In this work we study how to compress such a tokenizer by distillation, taking as the supervision target neither the discrete token nor the output distribution but the pre-quantizer latent the model actually consumes—the last representation the two token interfaces share. We train only the student encoder to regress the teacher’s per-frame latent under a squared-error objective, with a single affine layer absorbing the teacher–student width mismatch. Because the target precedes both the quantizer and the language-model bridge, one recipe covers both token interfaces we support, and applies both to a tokenizer pretrained alone and to one jointly trained with a language model. At 2.8× compression the distilled student stays within 1.9% relative WER of its teacher on five of six teacher–student pairs without any fine-tuning, and improves on an independently trained tokenizer of identical capacity by 3.9% relative. Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why July 9, 2026 research area Methods and Algorithms , research area Speech and Natural Language Processing On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions this signal is beneficial and under which it is detrimental. Which teacher model should be used, and in the case of self-distillation, which specific context should serve as the supervisory signal? Does the optimal choice vary from one token to the next? At present, addressing these questions typically… July 1, 2025 research area Methods and Algorithms , research area Speech and Natural Language Processing conference ICML We propose a distillation scaling law that estimates distilled model performance based on a compute budget and its allocation between the student and teacher. Our findings mitigate the risks associated with large-scale distillation by enabling compute-optimal allocation for both the teacher and student to maximize student performance. We provide compute-optimal distillation recipes for two key scenarios: when a teacher already exists, and when a… Discover opportunities in Machine Learning. Our research in machine learning breaks new ground every day.
Sources
Related stories
Language Discrimination Improves Linguistic Learning in Multilingual Speech Models
Language Discrimination Improves Linguistic Learning in Multilingual Speech Models - Apple Machine Learning Research research area Speech and Natural Language Processing content type paper published October 2026 Language Discrimination Improves Linguistic Learning in Multilingual Speech Models Authors Maureen de Seyssel, Jie Chi*, Zakaria Aldeneh* Multilingual self-supervised speech models can benefit from sharing information across languages, but under a matched total...
The State of Chinese Physical AI
🎧 Audio version (or download audio for later): Good Morning, Dermot McGrath is an Irish entrepreneur and advisor based in Shanghai with over a decade…
On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study
On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study - Apple Machine Learning Research research area Methods and Algorithms , research area Speech and Natural Language Processing conference EMNLP content type paper published September 2026 On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study Authors Iuri Macocco†, Pau Rodríguez Lopez, Arno Blaas, Luca Zappella, Marco Baroni†*, Xavier Suau...
The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models
The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models - Apple Machine Learning Research research area Methods and Algorithms , research area Speech and Natural Language Processing conference NeurIPS content type paper published September 2026 The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models Authors Xavier Suau, Alex Ferrando de las Morenas,...