Speech and signals

Tue, Oct 273 papers

arXiv

Beyond Accuracy: What Are Speech Emotion Recognition Models Learning?

Prince Baah-Mensah, Dennis Owusu Asamoah

Speech Emotion Recognition (SER) is important in human-machine interactions, enabling machines to detect and classify human emotions from solely human speech.Speech Emotion Recognition (SER) is important in human-machine interactions, enabling machines to detect and classify human emotions from solely human speech.

Preprint: empirical study
arXiv

JIT-MPE: Just-in-Time Multi-Pitch Estimation under Output Deadlines

Yoshiyuki Kobayashi

Live pitch displays, score followers and accompaniment systems must know which pitches are sounding within tens of milliseconds, but instrument-agnostic multi-pitch…Live pitch displays, score followers and accompaniment systems must know which pitches are sounding within tens of milliseconds, but instrument-agnostic multi-pitch estimation (MPE) has mainly been designed and evaluated offline.

Preprint: new model

Mon, Oct 261 paper

arXiv

Efficient Streaming Audio-Visual Target Speaker Extraction for Real-World Acoustic Scenes

Wendi Sang, Kai Li, Yifan Li et al.

Streaming audio-visual target speaker extraction (AVTSE) is essential for latency-sensitive applications. However, current causal systems face two coupled gaps.Streaming audio-visual target speaker extraction (AVTSE) is essential for latency-sensitive applications. However, current causal systems face two coupled gaps.

Overall pipeline of Falcon for audio-visual target speaker extraction.
Preprint: new model

Fri, Oct 231 paper

arXiv

Measuring the Verifier's Floor: How Much Apparent Rare-Word Text-to-Speech Error Is ASR Error?

Kirill Borodin, Vasilii Kudryavtsev, Nikita Vasiliev et al.

Text-to-speech (TTS) is usually evaluated by transcribing its output with an automatic speech recognition (ASR) system, so the word error rate also counts recognizer…Text-to-speech (TTS) is usually evaluated by transcribing its output with an automatic speech recognition (ASR) system, so the word error rate also counts recognizer errors.

Preprint: empirical study

Thu, Oct 221 paper

arXiv

Decodability Is Not Controllability: Local and Anisotropic Speaker Control in Continuous-Latent TTS

Yuanzhong Zheng, Wei Huang, Ming Ma et al.

High linear decodability does not entail causal controllability in continuous-latent text-to-speech (TTS).High linear decodability does not entail causal controllability in continuous-latent text-to-speech (TTS). In VoxCPM2, position-matched probes and equal-norm interventions across 30 speakers and 60 directed dyads expose this…

Preprint: empirical study

Wed, Oct 212 papers

arXiv

Slots Are Not Speakers: Training-Free Speaker Linking for Frozen Streaming Diarizers

Aditya Srinivas Menon, Kumarmanas Nethil, Kavya Manohar

This paper presents a causal, training-free linker that allows a frozen streaming diarizer to track more speakers than it has output slots.This paper presents a causal, training-free linker that allows a frozen streaming diarizer to track more speakers than it has output slots.

Preprint: method
arXiv

RealComm: Benchmarking and Adapting Audio Deepfake Detection over Real Communication Channels

Li Wang, Jindong Wang, Wan Lin et al.

Acoustic capture, device processing, and transmission coding in real calls jointly alter speech signals and may affect the reliability of audio deepfake detection.Acoustic capture, device processing, and transmission coding in real calls jointly alter speech signals and may affect the reliability of audio deepfake detection.

Preprint: benchmark or dataset

Mon, Oct 191 paper

arXiv

Beyond the Benchmark: Conditioning, Domains and Failure Modes of Qwen3-TTS

Séverin Baroudi, Yanis Labrak, Petr Motlicek et al.

Qwen3-TTS reports state-of-the-art zero-shot voice cloning, but its technical report leaves open how the model should be conditioned, how it behaves outside read speech…Qwen3-TTS reports state-of-the-art zero-shot voice cloning, but its technical report leaves open how the model should be conditioned, how it behaves outside read speech, and how it fails.

Preprint: empirical study

Thu, Oct 151 paper

arXiv

MUSEBENCH: A Benchmark for AI-Generated Music Detection and Artifact Analysis

Yi Chen Liu, Syed Irfan Ali Meerza, Jiawei Yu et al.

Advances in generative AI have made it possible to produce music that is nearly indistinguishable from human compositions, raising the question of whether such works can…Advances in generative AI have made it possible to produce music that is nearly indistinguishable from human compositions, raising the question of whether such works can be reliably distinguished from human compositions.

Preprint: benchmark or dataset

Mon, Oct 123 papers

arXiv

OpenSVBench: Diagnosing Speaker Verification Generalization Across Deployment Scenarios

Qituan Shangguan, Bosen Xu, Shuai Wang

Speaker verification (SV) systems now achieve very low error rates on standard public protocols, but those results do not fully describe deployment generalization.Speaker verification (SV) systems now achieve very low error rates on standard public protocols, but those results do not fully describe deployment generalization.

Preprint: benchmark or dataset
arXiv

O'ido: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator

Daniel Varela Sánchez, Guillermo Martín-Coello Juárez

Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands.Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands. We present O'ido, an inference engine that runs open-vocabulary English speech recognition entirely on an…

Preprint: system
arXiv

What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings

Mateusz Modrzejewski, Tomasz Radzikowski

Open audio-language models are used as judges of generated music: given a clip and its text prompt, the model rates the clip, and the judge is validated by how well its…Open audio-language models are used as judges of generated music: given a clip and its text prompt, the model rates the clip, and the judge is validated by how well its ratings correlate with expert ratings.

Preprint: empirical study

Thu, Oct 81 paper

arXiv

Backdooring Acoustic Foundation Models for Physically Realizable Triggers

Zebin Yun, Eyal Ronen, Mahmood Sharif

Acoustic foundation models (AFMs) have democratized acoustic applications, enabling powerful models for tasks ranging from speech recognition to speaker verification with…Acoustic foundation models (AFMs) have democratized acoustic applications, enabling powerful models for tasks ranging from speech recognition to speaker verification with minimal resources.

Preprint: method

Fri, Oct 21 paper

arXiv

Collapse, Not Invariance: Diagnosing Auxiliary Objectives in Speech Anti-Spoofing

Ksenia Lysikova, Kirill Borodin, Maxim Maslov et al.

Speech anti-spoofing countermeasures degrade when the generator, codec or channel changes, and a common remedy is an auxiliary objective that shapes the embedding space…Speech anti-spoofing countermeasures degrade when the generator, codec or channel changes, and a common remedy is an auxiliary objective that shapes the embedding space; whether it does is invisible to EER, a pure ranking metric.

Preprint: empirical study

Thu, Oct 11 paper

arXiv

Improving Predicted MOS Scores, Not Perceived Quality: Multi-Predictor Test-Time Optimization of Enhanced Speech

Tsubasa Ochiai, Marc Delcroix, Nahomi Kusunoki et al.

Non-intrusive MOS predictors are widely used instead of subjective listening tests to evaluate and rank speech enhancement (SE) systems.Non-intrusive MOS predictors are widely used instead of subjective listening tests to evaluate and rank speech enhancement (SE) systems. If they accurately reflect perceived quality, raising their scores should lead to…

Preprint: empirical study