ASR Error Detection Using Recurrent Neural Network Language Model and Complementary ASR

Abstract

Detecting automatic speech recognition (ASR) errors can play an important role for effective human-computer spoken dialogue system, as recognition errors can hinder accurate system understanding of user intents.  Our goal is to locate errors in an utterance so that the dialogue manager can pose appropriate clarification questions to the users.  We propose two approaches to improve ASR error detection:  (1) using recurrent neural network language models to capture long-distance word context within and across previous utterances; (2) using a complementary ASR system. The intuition is that when two complementary ASR systems disagree on a region in an utterance, this region is most likely an error.  We train a neural network predictor of errors using a variety of features.  We performed experiments on both English and Iraqi Arabic ASR and observed significant improvement in error detection using the proposed methods.

Index Terms— ASR error detection, recurrent neural network language model, deep neural network acoustic model, complementary ASR