Bilingual Recurrent Neural Networks for Improved Statistical Machine Translation

Abstract

Recurrent Neural Networks (RNN) have been successfully applied for improved speech recognition and statistical machine translation (SMT) for N-best list re-ranking.  In SMT, we investigate using bilingual word-aligned sentences to train a bilingual recurrent neural network model.  We employ a bagof-word representation of a source sentence as additional input features in model training.  Experimental results show that our proposed approach performs consistently better than recurrent neural network language model trained only on targetside text in terms of machine translation performance.  We also investigate other input representation of a source sentence based on latent semantic analysis.