Rate-dependent Acoustic Modeling for Large Vocabulary Conversational Speech Recognition

Abstract

Variations in rate of speech (ROS) produce changes in both spectral features and word pronunciations that affect automatic speech recognition (ASR) systems. To deal with these ROS effects, we propose to use parallel, rate-specific, acoustic models: one for fast speech, the other for slow speech. Rate switching is permitted at word boundaries, to allow modeling within-sentence speech rate variation, which is common in conversational speech. Due to the parallel structure of rate- specific models and the maximum likelihood decoding method, we do not need high-quality ROS estimation before recognition, which is usually hard to achieve. In this paper, we evaluate our approach on a large-vocabulary conversational speech recognition (LVCSR) task over the telephone, with several minimal pair comparisons based on different baseline systems. Experiments show that on a development set for the 2000 Hub-5 evaluation, introducing word-level ROS-dependent models results in a 1.9% absolute win over a baseline system without multiword pronunciation modeling, and a 0.7% absolute win over a baseline system that incorporates a 4.0% absolute win from multiword pronunciation modeling. The combination of rate-dependent acoustic models with rate-dependent pronunciations obtained by using a data-driven approach is also explored and shown to produce an additional win.