Core technologies and applications
SRI’s Speech Technology and Research (STAR) Laboratory brings together a multidisciplinary mix of engineers, computer scientists and linguists. Together our experts build systems for a wide range of applications including signal processing; data indexing and mining; and computer-aided learning.
Speech recognition
Noise robustness
Speech production and perception-based features
Keyword spotting
Prosodic modeling and disfluencies
Speech & audio analytics
Voice biometrics
Language/accent identification
Speaker and speaker-state characterization
Audio event detection
Speaker diarization
Machine translation
Speech-to-Speech translation
Cross-lingual information retrieval
Machine-mediated cross-lingual communication
Natural language understanding
Human-computer interaction
Dialog systems and virtual personal assistants (VPAs)
Error detection and recovery
Semantic and syntactic parsing
Information extraction
Multi-lingual information extraction
Topic and event identification
Summarization;
Question answering
Our work
more +
-
Nuance Partners with SCIENTIA Puerto Rico
SRI spin-out Nuance Communications to expand access its Dragon Medical One for the island’s physicians and nurses
-
Aaron Lawson talks about the STAR Lab at SRI
Aaron Lawson is Assistant Lab Director at SRI’s Speech Technology and Research (STAR) lab. STAR lab brings together a multidisciplinary mix of engineers, computer scientists and linguists. Together their experts build systems for a wide range of applications including signal processing; data indexing and mining; and computer-aided learning. Join us to learn about how STAR […]
-
Faking videos is easy, deep nostalgia shows
Lifewire features SRI’s STAR Lab in a discussion around AI and techniques powering deepfakes
Speech and Natural language leadership
Featured researchers
Platforms
Open Language Interface for Voice Exploitation (OLIVE)
Novel speech processing technology leverages AI algorithms to enable speech activity detection in high levels of noise and distortion.
SenSay
Real-time speaker state platform estimates speaker state—such as emotion, sentiment, cognition, health, mental health and communication quality—in a range of end applications.
DynaSpeak® speech recognition engine
Small-footprint, high-accuracy engine incorporates patented techniques that increase recognition performance using speaker adaptation, microphone adaptation, end-of- speech detection, distributed speech recognition and noise robustness.
EduSpeak® speech recognition toolkit
Toolkit specifically designed for language-learning applications and other educational and training software. Works for both adult and child voices, it excels at recognizing native and non-native speakers.
SRI Language Modeling (SRILM)
Toolkit helps build and apply statistical language models for speech recognition, statistical tagging and segmentation, and machine translation. Can be downloaded and used free of charge.
Publications
more +
-
Toward Fail-Safe Speaker Recognition: Trial-Based Calibration with a Reject Option
The output scores of most speaker recognition systems are not directly interpretable as stand-alone values. For this reason, a calibration step is usually performed on the scores to convert them into proper likelihood ratios (LR), which have a clear probabilistic interpretation. The standard calibration approach transforms the system scores using a linear function trained using data selected to closely match the evaluation conditions. This selection, though, is not feasible when the evaluation conditions are unknown. In previous work, we proposed a calibration approach for this scenario called trialbased calibration (TBC). TBC trains a separate calibration model for each test trial using data that is dynamically selected from a candidate training set to match the conditions of the trial. In this work, we extend the TBC method, proposing (1) a new similarity metric for selecting training data that results in significant gains over the one proposed in the original work, (2) a new option that enables the system to reject a trial when not enough matched data is available for training the calibration model, and (3) the use of regularization to improve the robustness of the calibration models trained for each trial. We test the proposed algorithms on a development set composed of several conditions and on the FBI multi-condition speaker recognition dataset, and we demonstrate that the proposed approach reduces calibration loss to values close to 0 for most conditions when matched calibration data is available for selection and that it can reject most trials for which relevant calibration data is unavailable.
-
Resilient Data Augmentation Approaches to Multimodal Verification in the News Domain
With the advent of generative adversarial networks and misinformation in social media, there has been increased interest in multimodal verification. Image-text verification typically involves determining whether a caption and an image correspond with each other. Building on multimodal embedding techniques, we show that data augmentation via two distinct approaches improves results: entity linking and cross-domain local similarity scaling. We refer to the approaches as resilient because we show state-of-the-art results against manipulations specifically designed to thwart the exact multimodal embeddings we are using as the basis for all of our features.
-
Natural Language Access: When Reasoning Makes Sense
Natural language is one of the more appealing ways by which people can interact with computers, but up to now its application has been severely constrained. We argue that to use natural language effectively, we must have both a deep understanding of the subject domain and a general-purpose reasoning capability. We illustrate the issues with SAP-QUEST, a proof-of-concept system for natural language question answering over a set of data sources for business enterprise applications, but the argument can be applied more generally to dialogue-style interfaces over a variety of subject domains.
-
Wideband Spectral Monitoring Using Deep Learning
We present a system to perform spectral monitoring of a wide band of 666.5 MHz, located within a range of 6 GHz of Radio Frequency (RF) bandwidth, using state-of-the-art deep learning approaches.
-
Dual orexin and MCH neuron-ablated mice display severe sleep attacks and cataplexy
Orexin/hypocretin-producing and melanin-concentrating hormone-producing (MCH) neurons are co-extensive in the hypothalamus and project throughout the brain to regulate sleep/wakefulness.
-
Mapping Individual to Group Level Collaboration Indicators Using Speech Data
Automatic detection of collaboration quality from the students’ speech could support teachers in monitoring group dynamics, diagnosing issues, and developing pedagogical intervention plans.
-
Robust Speaker Recognition from Distant Speech under Real Reverberant Environments Using Speaker Embeddings
This article focuses on speaker recognition using speech acquired using a single distant or far-field microphone in an indoors environment. This study differs from the majority of speaker recognition research, which focuses on speech acquisition over short distances, such as when using a telephone handset or mobile device or far-field microphone arrays, for which beamforming can enhance distant speech signals. We use two large-scale corpora collected by retransmitting speech data in reverberant environments with multiple microphones placed at different distances. We first characterize three different speaker recognition systems ranging from a traditional universal background model (UBM) i-vector system to a state-of-the-art deep neural network (DNN) speaker embedding system with a probabilistic linear discriminant analysis (PLDA) back-end. We then assess the impact of microphone distance and placement, background noise, and loudspeaker orientation on the performance of speaker recognition system for distant speech data. We observe that the recently introduced DNN speaker embedding based systems are far more robust compared to i-vector based systems, providing a significant relative improvement of up to 54% over the baseline UBM i-vector system, and 45.5% over prior DNN-based speaker recognition technology.
-
Analysis of Complementary Information Sources in the Speaker Embeddings Framework
Deep neural network (DNN)-based speaker embeddings have resulted in new, state-of-the-art text-independent speaker recognition technology. However, very limited effort has been made to understand DNN speaker embeddings. In this study, our aim is analyzing the behavior of the speaker recognition systems based on speaker embeddings toward different front-end features, including the standard Mel frequency cepstral coefficients (MFCC), as well as power normalized cepstral coefficients (PNCC), and perceptual linear prediction (PLP). Using a speaker recognition system based on DNN speaker embeddings and probabilistic linear discriminant analysis (PLDA), we compared different approaches to leveraging complementary information using score-, embeddings-, and feature-level combination. We report our results for Speakers in the Wild (SITW) and NIST SRE 2016 datasets. We found that first and second embeddings layers are complementary in nature. By applying score and embedding-level fusion we demonstrate relative improvements in equal error rate of 17% on NIST SRE 2016 and 10% on SITW over the baseline system.
-
Structure-based lead optimization to improve antiviral potency and ADMET properties of phenyl-1H-pyrrole-carboxamide entry inhibitors targeted to HIV-1 gp120
We are continuing our concerted effort to optimize our first lead entry antagonist, NBD-11021, which targets the Phe43 cavity of the HIV-1 envelope glycoprotein gp120, to improve antiviral potency and ADMET properties. In this report, we present a structure-based approach that helped us to generate working hypotheses to modify further a recently reported advanced lead entry antagonist, NBD-14107, which showed significant improvement in antiviral potency when tested in a single-cycle assay against a large panel of Env-pseudotyped viruses. We report here the synthesis of twenty-nine new compounds and evaluation of their antiviral activity in a single-cycle and multi-cycle assay to derive a comprehensive structure-activity relationship (SAR). We have selected three inhibitors with the high selectivity index for testing against a large panel of 55 Env-pseudotyped viruses representing a diverse set of clinical isolates of different subtypes. The antiviral activity of one of these potent inhibitors, 55 (NBD-14189), against some clinical isolates was as low as 63 nM. We determined the sensitivity of CD4-binding site mutated-pseudoviruses to these inhibitors to confirm that they target HIV-1 gp120. Furthermore, we assessed their ADMET properties and compared them to the clinical candidate attachment inhibitor, BMS-626529. The ADMET data indicate that some of these new inhibitors have comparable ADMET properties to BMS-626529 and can be optimized further to potential clinical candidates.
