Language Discrimination Improves Linguistic Learning in Multilingual Speech Models
Language Discrimination Improves Linguistic Learning in Multilingual Speech Models - Apple Machine Learning Research research area Speech and Natural Language Processing content type paper published October 2026 Language Discrimination Improves Linguistic Learning in Multilingual Speech Models Authors Maureen de Seyssel, Jie Chi*, Zakaria Aldeneh* Multilingual self-supervised speech models can benefit from sharing information across languages, but under a matched total...
- Using a controlled English/French HuBERT setting, we test two interventions which strengthen language discrimination: an auxiliary language classifier and per-language k-means targets.
- Models like wav2vec 2.0 and HuBERT have achieved state-of-the-art results in tasks such as speech recognition, particularly in monolingual settings.
- Applied to XLM-R (Conneau et al, 2020) across pretraining checkpoints… Discover opportunities in Machine Learning.
- Apple Machine Learning Research research area Speech and Natural Language Processing content type paper published October 2026 Language Discrimination Improves Linguistic Learning in Multilingual Speech Models Authors Maureen de Seyssel, Jie Chi*, Zakaria Aldeneh* Multilingual self-supervised speech models can benefit from sharing information across languages, but under a matched total pretraining data budget they still fall short of monolingual models. We show that strengthening the model’s ability to discriminate languages during pretraining reduces and, on some measures, closes this multilingual gap on continuous phonetic and higher-level linguistic measures, while preserving substantial cross-language sharing. Using a controlled English/French HuBERT setting, we test two interventions which strengthen language discrimination: an auxiliary language classifier and per-language k-means targets. Across interventions, continuous-feature phone discrimination error (phone-ABX,↓) decreases from 11.6% in the bilingual baseline to 10.4% (monolingual: 10.8%), while lexical performance (sWUGGY,↑) increases from 52.1% to 56.7% (monolingual: 58.5%) and prosodic performance (ProsAudit, lexical subtask,↑) from 68.9% to 72.9% (monolingual: 72.6%). Across HuBERT training stages, the strongest gains on most linguistic measures occur when language discrimination is introduced in the first iteration, whereas later or repeated interventions yield smaller improvements and are accompanied by increased language-wise segregation. These results support a causal role for language discrimination in reducing the additional cost of multilingual learning. Leveraging Audio-Visual Data to Reduce the Multilingual Gap in Self-Supervised Speech Models September 25, 2025 research area Speech and Natural Language Processing conference ICASSP Self-supervised learning (SSL) has made significant advances in speech representation learning. Models like wav2vec 2.0 and HuBERT have achieved state-of-the-art results in tasks such as speech recognition, particularly in monolingual settings. However, multilingual SSL models tend to underperform their monolingual counterparts on each individual language, especially in multilingual scenarios with few languages such as the bilingual setting. In… Discriminating Form and Meaning in Multilingual Models with Minimal-Pair ABX Tasks June 13, 2025 research area Speech and Natural Language Processing conference EMNLP We introduce a set of training-free ABX-style discrimination tasks to evaluate how multilingual language models represent language identity (form) and semantic content (meaning). Inspired from speech processing, these zero-shot tasks measure whether minimal differences in representation can be reliably detected. This offers a flexible and interpretable alternative to probing. Applied to XLM-R (Conneau et al, 2020) across pretraining checkpoints… Discover opportunities in Machine Learning. Our research in machine learning breaks new ground every day.
Sources
Related stories
On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study
On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study - Apple Machine Learning Research research area Methods and Algorithms , research area Speech and Natural Language Processing conference EMNLP content type paper published September 2026 On the Effectiveness-Fluency Trade-Off in LLM Conditioning: A Systematic Study Authors Iuri Macocco†, Pau Rodríguez Lopez, Arno Blaas, Luca Zappella, Marco Baroni†*, Xavier Suau...
The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models
The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models - Apple Machine Learning Research research area Methods and Algorithms , research area Speech and Natural Language Processing conference NeurIPS content type paper published September 2026 The Communication Bottleneck: A Round-Trip Study of Tree-Structured Expression Serialization in Language Models Authors Xavier Suau, Alex Ferrando de las Morenas,...
Gemini 3.8 text-to-speech says hello
Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS are our most expressive audio generation models yet. Generate custom character voices and direct scene dialogue across Google AI Studio, Gemini API, Gemini Enterprise, Gemini Notebook, and Google Vids.
Model Auditing: Faithfulness of Post-hoc Explanations Investigated
A study published on arXiv cs.CV investigates the faithfulness of post-hoc explanations for a pedestrian detection model across different domains.