Core technologies and applications
SRI’s Speech Technology and Research (STAR) Laboratory brings together a multidisciplinary mix of engineers, computer scientists and linguists. Together our experts build systems for a wide range of applications including signal processing; data indexing and mining; and computer-aided learning.
Speech production and perception-based features
Prosodic modeling and disfluencies
Speech & audio analytics
Speaker and speaker-state characterization
Audio event detection
Cross-lingual information retrieval
Machine-mediated cross-lingual communication
Natural language understanding
Dialog systems and virtual personal assistants (VPAs)
Error detection and recovery
Semantic and syntactic parsing
Multi-lingual information extraction
Topic and event identification
Aaron Lawson is Assistant Lab Director at SRI’s Speech Technology and Research (STAR) lab. STAR lab brings together a multidisciplinary mix of engineers, computer scientists and linguists. Together their experts build systems for a wide range of applications including signal processing; data indexing and mining; and computer-aided learning. Join us to learn about how STAR […]
Speech and Natural language leadership
Novel speech processing technology leverages AI algorithms to enable speech activity detection in high levels of noise and distortion.
The output scores of most speaker recognition systems are not directly interpretable as stand-alone values. For this reason, a calibration step is usually performed on the scores to convert them into proper likelihood ratios (LR), which have a clear probabilistic interpretation. The standard calibration approach transforms the system scores using a linear function trained using data selected to closely match the evaluation conditions. This selection, though, is not feasible when the evaluation conditions are unknown. In previous work, we proposed a calibration approach for this scenario called trialbased calibration (TBC). TBC trains a separate calibration model for each test trial using data that is dynamically selected from a candidate training set to match the conditions of the trial. In this work, we extend the TBC method, proposing (1) a new similarity metric for selecting training data that results in significant gains over the one proposed in the original work, (2) a new option that enables the system to reject a trial when not enough matched data is available for training the calibration model, and (3) the use of regularization to improve the robustness of the calibration models trained for each trial. We test the proposed algorithms on a development set composed of several conditions and on the FBI multi-condition speaker recognition dataset, and we demonstrate that the proposed approach reduces calibration loss to values close to 0 for most conditions when matched calibration data is available for selection and that it can reject most trials for which relevant calibration data is unavailable.
With the advent of generative adversarial networks and misinformation in social media, there has been increased interest in multimodal verification. Image-text verification typically involves determining whether a caption and an image correspond with each other. Building on multimodal embedding techniques, we show that data augmentation via two distinct approaches improves results: entity linking and cross-domain local similarity scaling. We refer to the approaches as resilient because we show state-of-the-art results against manipulations specifically designed to thwart the exact multimodal embeddings we are using as the basis for all of our features.
Natural language is one of the more appealing ways by which people can interact with computers, but up to now its application has been severely constrained. We argue that to use natural language effectively, we must have both a deep understanding of the subject domain and a general-purpose reasoning capability. We illustrate the issues with SAP-QUEST, a proof-of-concept system for natural language question answering over a set of data sources for business enterprise applications, but the argument can be applied more generally to dialogue-style interfaces over a variety of subject domains.
We present a system to perform spectral monitoring of a wide band of 666.5 MHz, located within a range of 6 GHz of Radio Frequency (RF) bandwidth, using state-of-the-art deep learning approaches.
Orexin/hypocretin-producing and melanin-concentrating hormone-producing (MCH) neurons are co-extensive in the hypothalamus and project throughout the brain to regulate sleep/wakefulness.
Automatic detection of collaboration quality from the students’ speech could support teachers in monitoring group dynamics, diagnosing issues, and developing pedagogical intervention plans.
Robust Speaker Recognition from Distant Speech under Real Reverberant Environments Using Speaker Embeddings
This article focuses on speaker recognition using speech acquired using a single distant or far-field microphone in an indoors environment. This study differs from the majority of speaker recognition research, which focuses on speech acquisition over short distances, such as when using a telephone handset or mobile device or far-field microphone arrays, for which beamforming can enhance distant speech signals. We use two large-scale corpora collected by retransmitting speech data in reverberant environments with multiple microphones placed at different distances. We first characterize three different speaker recognition systems ranging from a traditional universal background model (UBM) i-vector system to a state-of-the-art deep neural network (DNN) speaker embedding system with a probabilistic linear discriminant analysis (PLDA) back-end. We then assess the impact of microphone distance and placement, background noise, and loudspeaker orientation on the performance of speaker recognition system for distant speech data. We observe that the recently introduced DNN speaker embedding based systems are far more robust compared to i-vector based systems, providing a significant relative improvement of up to 54% over the baseline UBM i-vector system, and 45.5% over prior DNN-based speaker recognition technology.
Deep neural network (DNN)-based speaker embeddings have resulted in new, state-of-the-art text-independent speaker recognition technology. However, very limited effort has been made to understand DNN speaker embeddings. In this study, our aim is analyzing the behavior of the speaker recognition systems based on speaker embeddings toward different front-end features, including the standard Mel frequency cepstral coefficients (MFCC), as well as power normalized cepstral coefficients (PNCC), and perceptual linear prediction (PLP). Using a speaker recognition system based on DNN speaker embeddings and probabilistic linear discriminant analysis (PLDA), we compared different approaches to leveraging complementary information using score-, embeddings-, and feature-level combination. We report our results for Speakers in the Wild (SITW) and NIST SRE 2016 datasets. We found that first and second embeddings layers are complementary in nature. By applying score and embedding-level fusion we demonstrate relative improvements in equal error rate of 17% on NIST SRE 2016 and 10% on SITW over the baseline system.
Structure-based lead optimization to improve antiviral potency and ADMET properties of phenyl-1H-pyrrole-carboxamide entry inhibitors targeted to HIV-1 gp120
We are continuing our concerted effort to optimize our first lead entry antagonist, NBD-11021, which targets the Phe43 cavity of the HIV-1 envelope glycoprotein gp120, to improve antiviral potency and ADMET properties. In this report, we present a structure-based approach that helped us to generate working hypotheses to modify further a recently reported advanced lead entry antagonist, NBD-14107, which showed significant improvement in antiviral potency when tested in a single-cycle assay against a large panel of Env-pseudotyped viruses. We report here the synthesis of twenty-nine new compounds and evaluation of their antiviral activity in a single-cycle and multi-cycle assay to derive a comprehensive structure-activity relationship (SAR). We have selected three inhibitors with the high selectivity index for testing against a large panel of 55 Env-pseudotyped viruses representing a diverse set of clinical isolates of different subtypes. The antiviral activity of one of these potent inhibitors, 55 (NBD-14189), against some clinical isolates was as low as 63 nM. We determined the sensitivity of CD4-binding site mutated-pseudoviruses to these inhibitors to confirm that they target HIV-1 gp120. Furthermore, we assessed their ADMET properties and compared them to the clinical candidate attachment inhibitor, BMS-626529. The ADMET data indicate that some of these new inhibitors have comparable ADMET properties to BMS-626529 and can be optimized further to potential clinical candidates.