Citation
Feifan Liu, Gokhan Tur, Dilek Hakkani-Tür, Hong Yu, Towards spoken clinical-question answering: evaluating and adapting automatic speech-recognition systems for spoken clinical questions, Journal of the American Medical Informatics Association, Volume 18, Issue 5, September 2011, Pages 625–630, https://doi.org/10.1136/amiajnl-2010-000071
Abstract
Objective
To evaluate existing automatic speech-recognition (ASR) systems to measure their performance in interpreting spoken clinical questions and to adapt one ASR system to improve its performance on this task.
Design and measurements
The authors evaluated two well-known ASR systems on spoken clinical questions: Nuance Dragon (both generic and medical versions: Nuance Gen and Nuance Med) and the SRI Decipher (the generic version SRI Gen). The authors also explored language model adaptation using more than 4000 clinical questions to improve the SRI system’s performance, and profile training to improve the performance of the Nuance Med system. The authors reported the results with the NIST standard word error rate (WER) and further analyzed error patterns at the semantic level.
Results
Nuance Gen and Med systems resulted in a WER of 68.1% and 67.4% respectively. The SRI Gen system performed better, attaining a WER of 41.5%. After domain adaptation with a language model, the performance of the SRI system improved 36% to a final WER of 26.7%.
Conclusion
Without modification, two well-known ASR systems do not perform well in interpreting spoken clinical questions. With a simple domain adaptation, one of the ASR systems improved significantly on the clinical question task, indicating the importance of developing domain/genre-specific ASR systems.
Keywords: Automated learning, discovery, text and data mining methods, other methods of information extraction, natural-language processing, knowledge bases, knowledge representations, knowledge acquisition and knowledge management, discovery, and text and data mining methods, natural-language processing, automated learning, processing, and display, analysis, image representation, controlled terminologies and vocabularies, ontologies, machine learning, spoken clinical question answering, language model adaptation, automatic speech recognition
