The Use of Word N-grams and Parts of Speech for Hierarchical Cluster Language Modeling

Abstract

We present extensions to the work of backoff hierarchical class n-gram language modeling of Zitouni et al. (2003) by studying the efficacy of exploring the use of parts of speech (POS) information in hierarchical word clustering. We propose two approaches. One is to use POS n-gram contextual distributions of a target word for clustering. The other is to generate a class tree for each group of words sharing the same POS. The resulting class tree and a set of class trees, from the two approaches, respectively, are then employed in the hierarchical cluster language modeling. We evaluate the two approaches on SRI Arabic conversational telephone speech recognition system and show that the approach of building a set of POS-specific class trees achieves a 3 pct. relative improvement on perplexity compared to the model of Zitouni et al. and a 8 pct. relative improvement on perplexity over the baseline standard word n-grams. When used for N-best rescoring, our approach also outperforms the model of Zitouni et al. and the baseline and achieves significant word error rate (WER) reductions.