During the last several years, i-vectors 
have become the dominant approach to text-independent Speaker Verification (SV). In i-vector based systems, utterances of arbitrary duration are mapped onto a low-dimensional subspace modelling both speaker and channel variability, which is estimated in an unsupervised way. The back-end classifier that is usually employed is a Probabilistic Linear Discriminant Analysis (PLDA) model which performs a linear disentanglement of the two dominant types of variability and enables the evaluation of likelihood ratios . The i-vector/PLDA approach, when trained and evaluated on large text-independent datasets (such as those provided by NIST ) has shown a remarkable consistency over the years in attaining state-of-the-art performance. More recently, neural architectures (e.g. x-vectors 
) have managed to outperform i-vectors in most text-independent SV benchmarks, by employing recent advances in deep learning and aggressive data augmentation.
In parallel, the great potential of voice biometrics in commercial applications and forensics has increased the need for methods yielding state-of-the-art results with utterances of short duration. However, a straightforward application of the i-vector/PLDA model to short utterances has been proven to be an inadequate solution . When utterances become shorter, variations due to differences in phonetic content can no longer be averaged out, as happens with utterances of long duration (e.g. 1min). There have been several efforts to propagate the i-vector uncertainty to the PLDA model, but they were only partially successful and the results were inconsistent across datasets [8, 7, 9].
Due to the moderate performance of i-vectors in the particular setting, text-dependent SV started attracting much attention. Text-dependent SV reduces the phonetic variations of short utterances by constraining their vocabulary to either (a) a fixed phrase, (b) a set of predefined phrases, or (c) random sequences of words coming from a specific domain, such as digits. The first two approaches yield superior performance in general, due to the matched order of acoustic events between training and run-time utterances, which prevents random and hard to model co-articulation effects from appearing. On the other hand, when speakers utter a predefined pass-phrase, the system becomes vulnerable to spoofing attacks (e.g. replay attacks), which have become a major threat to speaker recognition systems . Text-prompted SV with random sequences of words from a specific domain is less vulnerable to replay attacks (yet not immune to attacks created by Text-To-Speech and Voice Conversion systems111In this case, creating a random sequence of words from a prerecorded audio is more difficult due to co-articulation effects of words on each other, but not impossible.) and it is employed as a means to perform liveness detection.
In this paper, we primarily work with RSR2015 part III, aiming at enhancing the i-vector paradigm in text-prompted speaker recognition [12, 13]. One of our main motivations is to develop a method for utilizing the i-vector uncertainty tailored to text-dependent and text-prompted SV. We show that by introducing the concept of average uncertainty, a simple and effective linear digit-specific transform can be derived, which can compensate for the i-vector uncertainty without the computational burden in training and evaluation introduced by other uncertainty propagation methods [7, 14]. Building on top of our previous framework on text-dependent SV with fixed phrases ([15, 16]) and text-prompted case , we use digit-specific HMMs and i-vector extractors and we report an extensive experimentation with respect the front-end features (including bottleneck features), channel compensation, uncertainty-aware transforms, and backend approaches. To the best of our knowledge, the results we report are the best published on the challenging RSR2015 part III and constitute a strong baseline for newer deep learning methods (e.g. [18, 19]).
The rest of this paper is organized as follows. In Section II, we provide a detailed review of the proposed approaches to text-dependent SV and i-vector uncertainty modelling. In Section III, our digit-specific subsystems are explained. In Section IV we discuss different methods for performing uncertainty-aware channel compensation. The description of the dataset, experimental setup and results are given in Sections V and VI. Finally, Section VII we provide a brief conclusion of our work and directions for future work.
Ii Related Work
In this section we present and discuss some of the recent approaches that are related to our work, with emphasis to those involving text-dependent speaker verification and uncertainty modelling.
Ii-a Text-dependent speaker verification
There are several interesting approaches to text-dependent SV that have been proposed over the last few years. In , the authors examine a pass-phrase based system which is evaluated on the (proprietary) Wells Fargo text-dependent dataset. NIST datasets were also used for UBM training to overcome the small development set constraint. In 
, experiments are conducted on the same dataset and the authors propose the use of a separate Gaussian Mixture Model (GMM) mean supervector for each digit, adapted from a common UBM. Extracted supervectors undergo Nuisance Attribute Projection (NAP) and are passed to a Support Vector Machine (SVM) classifier to compute the scores. The authors show that their method outperforms the one proposed in on the same dataset.
In [22, 23, 24], the authors propose a Joint Factor Analysis (JFA) approach to address the problem of SV with random digit strings, using RSR2015 part III for training and testing. JFA is employed as a feature extractor, built on top of a tied-mixture model, i.e. an HMM with shared Gaussians and digit-specific sets of weights. The tied-mixture model serves for segmenting utterances into digits, as well as for collecting digit-specific Baum-Welch statistics for JFA modelling. JFA features are either local (i.e. one per digit) or global (i.e. a single per recording), and in the former case each local feature in the test utterance is scored against the corresponding ones in the enrolment utterances. In , JFA features are passed to a joint density back-end (alternative to PLDA), while in  the i-vector mechanics are used to incorporate the uncertainty in the back-end. Finally, in , the authors apply the same uncertainty-aware back-end to individual Gaussian mixture components, resulting in 20% error rate reduction on RSR2015 part III.
In the aforementioned approaches, a digit-independent or adapted UBM is employed, spanning the whole acoustic space. However, obtaining a robust estimate of a JFA speaker vector (i.e. -vector) using merely the tiny amount of information contained in a single digit (as happens with the local JFA features) proved to be very hard, making subspace methods to yield inferior results compared to supervector-size features (i.e. -vector). To address this data scarcity problem, a new scheme for using i-vector in text-prompted SV is introduced in , where word-specific UBMs and i-vector extractors are employed. These UBMs and i-vector extractors are of small size (64-component and 175-dimensional, respectively) as they cover only the phonetic content of each individual word. Following a similar approach, in [15, 16] it is shown that i-vectors, when extracted using phrase-specific UBMs and i-vector extractors yield superior performance compared to JFA front-end features.
Ii-B Modelling i-vector uncertainty
The use of i-vector uncertainty in the back-end may yield notable improvement in SV with short utterances and several methods for making use of it have been proposed. In [8, 7] the authors introduce a modified version of PLDA for propagating the i-vector uncertainty to the PLDA model and they derive an EM algorithm for PLDA training using utterances of arbitrary durations. Similarly, the use of the i-vector uncertainty in PLDA is investigated in , taking into account only the uncertainty in the test utterances (i.e. assuming long training and enrollment utterances). The authors in [25, 26] speed-up the uncertainty propagation method by grouping i-vectors together based on their reliability and by finding a representative posterior covariance matrix for each group. In , the authors incorporate the uncertainty associated with front-end features into the i-vector extraction framework. Finally, in 
, an extension of uncertainty decoding using simplified PLDA scoring and modified imputation is proposed. The authors also employ the uncertainty decoding technique in Linear Discriminant Analysis (LDA) in.
Iii Digit-specific HMMs, i-vector extractors and scoring
In this section we present our HMM-based UBM which we use to extract Baum-Welch statistics and the method for using these statistics to train digit-dependent i-vector extractors. The scheme extends our previous method developed on Persian months dataset . In  we proposed a simple but effective scheme based on separate i-vector extractor for each word with a common i-vector pipeline. In this paper, the above scheme is adapted to random digit strings and enhanced by exploring different methods for modelling uncertainty and combining it with channel compensation.
Note that in RSR2015 part III, the sequence of digits in each utterance is assumed to be known and therefore can be used during training and evaluation , but our methods can be extended to settings where the digit sequences should be estimated by an ASR system.
Iii-a Digit-specific HMMs
It is generally agreed that HMMs are a more natural solution to text-dependent SV than GMMs . The partition of the Gaussians into HMM states permits us to capture the speaker characteristics over segments corresponding to phrases and words, rather than over merely spectral areas, as happens with a UBM. The HMM corresponding to digit is parametrized by a collection of state-specific GMM emission distributions
and a transition probability matrix. We index HMM states by and Gaussian components of the GMM corresponding to HMM state by .
We initialize a collection of digit dependent HMMs, each having states and Gaussian components in each state . We use the subscripts to indicate their dependence on the digit and state , respectively, although in practice we use a fixed number of and . The overall number of Gaussian components of the digit-specific HMM is . HMM training and segmentation into digits is performed using Viterbi training (i.e. a single alignment of frames to HMM states is considered), by concatenating the corresponding digit-dependent HMMs and utilizing the “left-to-right, no skips” structure. Therefore, the concatenated HMM corresponding to an utterance with digits has states and overall Gaussian components, and it is constructed by concatenating the corresponding digit-dependent HMMs.
Once the HMMs are trained, we jointly perform (a) segmentation of utterance into digits, and (b) segmentation of each frame sequence assigned to a digit to digit-specific HMM states . We apply Viterbi-based forced alignment to assign frames to HMM states and hence we estimate hard assignment of frames to HMM states. Then, given the estimated alignments to digit-specific HMM states , frame posteriors corresponding to GMM components of the specific state are computed as follows,
where is the Kronecker delta function, andthe dependence on is kept implicit.
The frame posteriors, together with the corresponding Gaussian components are used to extract zero and first order centralized statistics, and , which are computed by the following equations
In the above equations, is the number of frames of the utterance, is the index of mixture component of the digit-specific mixture model, is the frame at time and
is the posterior probability that theframe has been emitted by the component. Note that once the frame-posteriors are calculated the HMM structure is no longer required for extracting Baum-Welch statistics. Therefore, is used for indexing components of the flattened HMM, i.e. a GMM corresponding to the concatenated state-specific GMMs, having overall Gaussian components and rescaled weights so that they sum up to 1. The flattened HMM plays the role of the UBM in text-independent speaker recognition.
Iii-B Digit Dependent i-vector Extractor
Due to the use of digit-specific HMMs as UBMs for collecting Baum-Welch statistics, all the following structures should also be digit-specific. This includes i-vector extractors, transforms applied to i-vectors as well as trainable back-ends (e.g. PLDA).
The supervector of an utterance associated with a digit is assumed to be generated from the following equation
where is a low rank matrix representing the subspace spanning the dominant variability in the supervector space, and is the supervector corresponding to the digit-specific flattened HMM. Moreover, is a latent variable with standard normal distribution as a prior. Given the Baum-Welch statistics of an utterance, the posterior distribution of is normal with mean and covariance matrix estimated as follows
where and are zero and centralized first order statistics (using the means of the corresponding digit-specific HMM), and is a block diagonal covariance matrix obtained from the corresponding digit-specific HMM.
Iii-C Digit-specific scoring
After extracting digit-specific i-vectors and applying a set of transforms (to be discussed in Sect. IV), scoring is also implemented in a digit-specific fashion, i.e.
where superscripts and indicate enrollment and test respectively, is the set of digits appearing in the test utterance of the trial ( in RSR2015 part III), and
is a similarity measure on the i-vector space (e.g. cosine similarity, PLDA-based log-likelihood ratio, a.o.), which is a function of parameters(e.g. transforms for channel compensation, PLDA parameters) and can also include score normalization. Finally, we use to denote the averaged enrollment i-vectors of the digit , since there might be more than one i-vectors of the same digit in the enrollment side (e.g. three in RSR2015 part III).
This scoring rule is identical to the “local” approach, proposed in . The rationale is to break-down the utterances into segments of limited phonetic content (e.g. words, digits) in order to suppress the phonetic variability between enrollment and test segments. A caveat is that certain segments of the enrollment utterances are not used in each trial, as the test utterance may not contain all the words appearing in the enrollment. In the case of RSR2015 part III, about 50% of the enrolment number of frames is used in each trial, since .
Iii-D Differences between the proposed method and tied-mixture models
, the two methods are substantially different. Aside from differences in (a) subspace modelling (i-vectors vs. JFA features), (b) linear transforms applied to i- or y-vectors, and (c) back-ends (cosine distance vs joint-density models) there are differences in the way frames are assigned to Gaussian components. We propose digit-specific HMMs ofGaussian components each, without sharing them between digits or states, while in the tied-mixture approach, all Gaussian components are shared between digits, with the weights being the only digit-specific set of parameters. As a results, digit-specific i-vectors (or -vectors ) using tied-mixture models are extracted over highly sparse Baum-Welch statistics, and are therefore characterized by high posterior uncertainty. Moreover, in the tied-mixture model approach the HMM structure is merely employed for segmenting utterances into digits, while we propose digit-specific HMMs to segment each digit into subword units. As a result, the Gaussian components are localized in the joint temporal and spectral domain, while in the tied-mixture approach they are merely localized in the spectral domain, via a standard UBM.
Iv i-vector uncertainty and channel compensation
Due to the unsupervised way the i-vector extractors are trained, the i-vector space contains both speaker and session variability. Since only speaker information is useful to verify a speaker, a strategy for removing undesirable session effects is required. In parallel, in short duration SV the problem of increased uncertainty should also be addressed. To this end, we proposed three methods for channel and uncertainty compensation, which are explained in this section. Fig. 1 illustrates the block diagram of the whole system, where all the examined compensation methods are depicted. In this figure, based on the selected method for uncertainty and channel compensation, one of the parallel switches is activated.
Iv-a Between and within-class covariance and uncertainty
It is well known that the total variability covariance matrix can be decomposed into between-class and within-class covariance matrices, and as follows
However, by defining in the above way we are essentially treating i-vectors as point estimates. In order to take into account the uncertainty in the i-vector estimates, we should redefine the total variability as follows S^u_tot = 1n∑_i=1^n E[(y_i - ¯y)(y_i -¯y)^T ] = 1n∑_i=1^n (E[y_i] - ¯y)(E[y_i] -¯y)^T + cov(y_i) = S_tot + S_u , where is the average uncertainty of the i-vectors and is the overall number of i-vectors. The uncertainty in estimating is negligible as it is equal to . It is interesting to note that is used in the i-vector extractor and in JFA during the minimum divergence estimation, where the latent variables are transformed in such a way so that . In other words, the covariance of the aggregated posterior is set equal to the covariance of the prior distribution by transforming accordingly . The principal components of correspond to the directions with the highest uncertainty.
On the other hand, when dealing with short utterances, becomes comparable to and it would be interesting to make use of it when performing channel compensation. We should moreover note that by decomposing into expected within and between-class covariance
we may consider as being part of the within-class covariance, i.e.
This is due to the fact that the uncertainty contained in is smaller compared to , where the average number of i-vectors per speaker, since
where is the average uncertainty of i-vectors of speaker .
Iv-B Digit dependent Uncertainty and Channel Compensation
We examine here our three different proposed approaches, as well as regularized LDA for applying session and uncertainty compensation. In all cases, the transformed vectors are obtains as .
Iv-B1 Uncertain LDA
LDA is a standard technique to compensate for inter-session variability by finding a set of speaker-discriminant non-orthogonal directions and projecting the i-vectors onto the subspace they define . LDA minimizes the within-class variability while maximizing the between-class variability. Using the expectations of these matrices, the objective function of LDA becomes
is the projection matrix. By solving the above equation using generalized eigenvalue decomposition, uncertainty-aware channel compensation can be applied to i-vectors.
Iv-B2 Digit dependent uncertain WCCN
Within-Class Covariance Normalization (WCCN) is a popular technique for channel compensation that uses the Cholesky decomposition of the inverse within class covariance matrix (10) to project the input features. In speaker recognition, it is used typically before applying length normalization or cosine distance scoring . The uncertain version of WCCN is as follows
where is the projection matrix.
Iv-B3 Digit Dependent Uncertainty Normalization
Finally, we propose a novel technique which we call Uncertainty Normalization. In this case, we are using only the average uncertainty and we ignore the clustering structure of i-vectors into speakers. It is an unsupervised method and therefore it does not require multiple recordings per speaker. The rationale is to project the i-vectors onto a space that down-scales directions exhibiting high uncertainty, since their estimates are less reliable. Similarly to uncertain WCCN, it is defined as follows
where is the projection matrix.
Iv-B4 Regularized LDA
LDA has the constraint of reducing the dimensionality to at most where is the number of classes. Yet, in RSR2015 the number of training speakers is smaller than the i-vector dimension. To overcome this limitation and avoid dimensionality reduction we add a simple regularization term to . The regularized version of LDA yields better results than standard LDA in text-independent task too . In our experiments, we combine Regularized LDA with Uncertain WCCN and Uncertainty Normalization.
V Experimental Setup
We used the RSR2015 part III dataset for almost all our experiments. In this dataset, there are 157 males and 143 females speakers, divided into three disjoint speaker subsets: background, development and evaluation, of about 100 speakers each. Each speaker model is enrolled with 3 10-digit utterances, recorded with the same handset, while each speaker contributes 3 different speaker models. Test utterances contain a quasi-random string of 5 digits, one out of 52 unique strings. Six commercial mobile devices were used for the recordings that took place under a typical office environment. All utterances are in English, while speakers are balanced in such a way so that they form a representative sample of the Singaporean population [12, 24].
Apart from the RSR2015 part III, two clean parts of 16 kHz LibriSpeech dataset are used for training a DNN model and performing experiments with Bottleneck (BN) features (namely Train-Clean-100 and Train-Clean-360 ). The dataset contains English speech which is automatically aligned and segmented.
V-B Baseline and state-of-the-art
As a baseline method, we refer to the experiments performed by CRIM () where both subspace and supervector domain methods are investigated. For fair comparison, we used the same setup as in [22, 23, 24] and our baseline results are copied from the reference paper. The number of trials can be found in Table I.
To the best of our knowledge, the current state-of-the-art in RSR2015 part III is the model presented in . The proposed system makes use of a DNN trained either on Fisher data or on RSR2015. Two main approaches are examined, namely DNN posteriors with MFCC features and tandem features, i.e. bottleneck features concatenated with MFCCs.
We use 60-dimensional PLP or MFCC, extracted using HTK with a similar configuration: 25 ms Hamming windowed frames with 15 ms overlap. For each utterance, the features are normalized using Cepstral Mean and Variance Normalization (CMVN). A separate silence model is used for performing supervised Voice Activity Detection (VAD). Silent frames are removed after applying Viterbi alignment.
In addition to the cepstral features, a set of experiments is performed to examine the effectiveness of bottleneck (BN) and tandem features in the text-prompted task. To this end, a neural network is trained following the stacked architecture described and evaluated in[34, 35]. Based on the reported results in , this architecture exhibits very good performance in text-dependent SV. The output layer (softmax) has about 9000 senones, its input has 30 frames context around the current frame and it is trained using cross-entropy loss. Finally, the 80-dimensional BN features are concatenated to the cepstral features and used as input features to the i-vector pipeline.
V-D Model dimensions and gender dependence
Digit-specific HMMs with 8 states and 8 components per state are used as UBM, while the i-vector dimensionality is set to 300. Gender independent UBMs and i-vector extractors are trained using only the background set of RSR2015. The background set is also used for training gender dependent LDA transforms as well as for score normalization. LDA and score normalization are applied in a digit-dependent manner. The MSR open source toolbox was used as a base for developing our code .
V-E Scoring method
In our proposed system we use score-normalized cosine distance. As Eq. (7) shows, for each digit-dependent test i-vector we extract, its cosine similarity with the average of the corresponding digit-dependent i-vectors from the enrolment speaker utterances is computed and the total score of the utterance is evaluated as the average score . It is worth mentioning that the proposed verification system uses a simple scoring method while other uncertainty-aware approaches typically require more complicated and computationally demanding methods, such as PLDA with uncertainty propagation .
V-F Score and Length Normalization
Score normalization is essential when cosine distance scoring is employed . After experimenting with several score normalization methods, we found that S-Norm yields the best performance. Therefore, for all the reported experiments and unless explicitly stated, S-Norm is applied in a gender and digit dependent manner, using the training set for collecting the cohort set of speakers.
Although implicit in cosine distance scoring, length normalization helps towards obtaining more Gaussian-like distributions 
. It is therefore useful to apply it before LDA (and after uncertainty normalization), as the latter assumes Gaussian distributed class-means and class-conditional observations.
The evaluation metrics we report to assess the performance of the proposed methods are the Equal Error Rate (EER) and the Detection Cost Functions (DCFs) defined for NIST-SRE08 and NIST-SRE10, namely old Normalized DCF () and new Normalized DCF ().
Vi-a Baseline, state-of-the-art and our methods
Table II shows the comparison between the proposed methods and several flavors of the baseline system. We select the best single system on this dataset from  and fusion results of single systems with different combinations. -vector and -vector are JFA-features with and without speaker subspace, respectively.
We also report results using speaker embeddings (x-vectors ), which define the state-of-the-art in text-independent speaker recognition. The model attains state-of-the-art results on the Speakers In-The-Wild benchmark (namely 2.32% EER on Eval Core ). The x-vector architecture is trained using a large dataset with more than 7K speakers (VoxCeleb 1 and 2 [39, 40]) compared to the 97 speakers used to train the i-vector extractors. All results reported are derived using identical evaluation set-up, ensuring a fair comparison.
In addition, Fig. 2 shows the DET curves of some selected systems from Table II for female speakers. In the third and fourth sections of this table we report results for the systems with PLP and MFCC features. We observe that for both genders, MFCCs outperform PLPs in almost all experiments. Based on these results, the system with MFCC features is considered as the best single system. Moreover, score-level fusion results of the two systems are given the fifth section of Table II.
Vi-B Uncertainty Normalization, Channel Compensation and Score Normalization
As Table II shows, the proposed uncertainty normalization methods attain the best results. Hence, it is worth further analyzing its performance, e.g. by deactivating channel compensation (i.e. Regularized LDA) and score normalization.
In Table III, we report results using several such combinations, as well as an experiment with PLDA. First of all, we observe that the contribution of Regularized LDA is rather minor compared to uncertainty normalization. This result is rather surprising; it shows that state-of-the-art performance can be attained even without explicit channel modelling, i.e. without the need of collecting multiple training recording coming from different channels, sessions or handsets, per speaker. We mention again that in RSR2015 part III the enrolment utterances for a given speaker are coming from a single handset, which is different to the ones used in the test utterances .
Finally, we examine the effectiveness of Gaussian PLDA as a backend. To this end, we train digit-dependent PLDA models using the RSR2015 part III training set. After experimentation, we found that the combination of uncertainty normalization, regularized LDA and number of speaker factors equal to 50 yields the best performance, while S-Norm does not yield any further gains. However, even the best PLDA configuration is clearly inferior to that attained by cosine distance. We believe that the failure of PLDA is due to the small number of training speakers in RSR2015, which prevents us from estimating robustly the speaker subspace.
Vi-C Comparison with x-vector
The embedding extractor is implemented using the standard Kaldi recipe, and it is trained on VoxCeleb 1 and 2 (containing more that 7K speakers) . The PLDA model used for evaluating LLRs is also trained on VoxCeleb, while we also report results where the RSR2015 training set is employed for PLDA training or adaptation. For enrolling the speakers, three utterances are concatenated and a single x-vector is extracted and for evaluation utterances, and each sequence is represented by an x-vector. The results in the third row of Table II show that the best performance is attained by training PLDA on VoxCeleb without any adaptation. However, our proposed method performs notably better than this. To improve the performance of x-vectors, recently proposed methods for applying domain adaptation to the x-vector extractor (e.g. using Generative Adversarial Networks [41, 42]) are worth exploring, in order to reduce the mismatch in channel and accent between VoxCeleb and RSR2015.
Vi-D Using bottleneck features
Neural approaches using DNNs trained for ASR have resulted in significant improvements in SV, especially in text-independent SV, where the text is unknown and DNNs help towards assigning frames to ASR recognition units (e.g. senones) . Some recent works apply DNNs to text-dependent SV and report notable improvements  
. Hence, it is worth examining the performance of bottleneck and tandem features extracted from a DNN in text-prompted case. Since the best performance among uncertainty and channel compensation methods is attained by uncertainty normalization followed by regularized LDA, we report the results using only this method. TableIV shows the results obtained by 80-dimensional bottleneck feature vector, by their concatenation with MFCC feature vector (i.e. tandem features) and by their fusion with other cepstral features. The results show that although the performance of bottleneck features without any fusion is poor, fusing tandem features with other cepstral features yields significant improvements. The reason for this degradation could be the randomness in digit sequence compared to the fixed sequences of other text-dependent tasks, as well as the fact that we did not use in-domain RSR2015 data to fine-tune the network. It is also apparent from the results that tandem features yield more notable improvement for female speakers.
In Table V we examine the performance of our best single-feature model (i.e. with MFCC) by varying the number of states per HMM . As we observe, the performance is rather insensitive to , being slightly higher for . However, we choose to use for the rest of the experiments, since their differences are minor and the algorithm becomes less computationally and memory demanding.
Vi-E The effect of length normalization
It is generally agreed that applying length normalization before LDA improves its performance. In order to reexamine its positive effect we perform an experiment to compare the performance of length normalization followed by LDA and LDA without length normalization. The system with MFCC features and uncertainty normalization is used as the single system in this experiment. Table VI shows that although cosine similarity scoring applies length normalization implicitly, applying length normalization before LDA and after uncertainty compensation is beneficial. As discussed above, length normalization makes vectors more normally distributed, which is in line with the Gaussian assumptions of LDA.
Vi-F Results on phrases using the RedDots corpus
Although we developed our method primarily for words as recognition units, we can evaluate it on short phrases in a similar way. For experimentation on phrases, RSR2015 part I used to be a standard option, however it is now considered as a too easy corpus . RedDots is more challenging in terms of channel variability, mostly due to (a) the longer time intervals between successive recordings of the same speaker, and (b) the higher levels of background noise .
The main caveat of RedDots is the lack of a training set, due to the small number of participants (49 males and 13 females, with only 35 males and 6 females having target trials). This shortcoming prevents us from evaluating our method on the whole set of RedDots phrases, since training utterances of the evaluation phrases are compulsory in order to train our models. Nevertheless, two of the RedDots phrases (namely the 33rd and the 34th) are also contained in RSR2015 Part I, enabling us to train our models on the corresponding training utterances of RSR2015.
In Table VII we report the performance on male-only trials, as the number of female speakers is too small for drawing any conclusion. The results are averaged over the 33rd and 34th RedDots phrases. Our focus is on the Impostor-Correct results, i.e. with non-targets trials containing the correct phrase, since we believe ASR-based methods are more adequate to estimate whether or not the uttered phrases match the prompted ones. The results show that uncertainty normalization is more effective than regularized LDA, attaining drastic relative improvement in low false acceptance operating points (63% in and 45% in ) and 30% relative improvement in terms of EER. These improvements are attained without any channel compensation, i.e. without requiring repetitions of the same phrase from each training speaker. Finally, by combining uncertainty normalization with regularized LDA a further small improvement is attained.
The comparison of our results with the tied-mixture model approach (Table II) shows that our single system outperforms the baseline by a large margin. In fact, its performance is superior not only to all single baseline systems, but also to the fusion of all systems in . Furthermore, the use of i-vectors rather than supervector size features (-vectors) makes our methods significantly faster. Additionally, memory requirements for each speaker are considerably lower than those of the baseline. Moreover, our system attains higher performance compared to the current state-of-the-art (which is based on DNNs ), even when a single system is used and without training on any external dataset (Fisher dataset is used to train the tandem feature system in ).
In terms of uncertainty and channel compensation methods, uncertainty normalization followed by length normalization and LDA is the more effective combination (Table IV). The results in Table II show a consistency with respect to features (MFCC and PLP) and gender, while the experiments on RedDots reaffirm the effectiveness of the proposed sequence of transforms, yielding drastic improvements especially in the low false alarm area (Table VII). In terms of front-end features, MFCC perform consistently better than PLP in both genders, while bottleneck features seem to be marginally effective, and only when fused with MFCC. Bottleneck features perform very well in text-independent speaker recognition, especially when used as a means to assign frames into UBM components  . However, there is a severe mismatch between the way frames are assigned in text-independent speaker recognition and our proposed HMM-based method. For example, the large context window used in the former does necessarily provide fine-grained temporal localization, required in order to segment each digit into states. More recent end-to-end methods may be more effective ways of using DNNs for text-dependent speaker recognition than bottleneck features (e.g. , ), with the caveat that they require large amounts of in-domain data, which are not available in RSR2015 part III.
Vi-H Scaling-up to larger vocabulary
In cases where a larger vocabulary can be employed the proposed method may suffer from data fragmentation. The number of overall training examples should scale linearly with the number of words, as no parameter sharing is assumed between the word-specific models. In such cases, introducing parameter sharing between the models (especially between the several word-specific HMMs and i-vector extractors) should be considered. Although such a setting is beyond the scope of this work, one may start with a typical large-size UBM/i-vector system (e.g. with 2048 Gaussian components) trained on text-independent datasets or on the available in-domain dataset. Then for each word in the vocabulary, the most dominant Gaussian components should be selected and their means should possibly be re-estimated e.g. via mean-only MAP adaptation, with the remaining components being removed. Word-specific i-vector extractors on top of the word-specific UBM can then be derived by (a) keeping only those rows corresponding to the most dominant Gaussian components, and (b) refining the matrix by applying e.g. minimum divergence training (i.e. without re-estimating the subspace). One may also consider starting from a higher dimensional i-vector extractor (e.g. 600) and selecting the most dominant dimensions for each word-specific extractor.
Vii Conclusions and Future Work
In this paper, we developed a system for text-prompted speaker verification using random digit strings. The core of the system comprises a set of digit-specific HMMs, which we employ in order to perform segmentation of utterances into digits, alignment of frames to HMM states and extraction of Baum-Welch statistics. On top of these HMMs, digit-specific i-vector extractors are trained, enabling us to compare digit-specific i-vectors that appear in both enrolment and test utterances using simple cosine distance scoring with score normalization. Furthermore, we investigated three different methods for compensating channel and uncertainty and we concluded that the novel uncertainty normalization technique followed by LDA yields consistently superior performance. The proposed system outperforms the baseline by a large margin and yields superior performance compared to the current state-of-the-art, which is based on DNNs.
We also examined the use of bottleneck features and different types of cepstral features. The experiments showed that although the performance of cepstral features is superior to that of bottleneck features, fusion with other cepstral features leads to further notable improvement. Our final set of experiments were conducted on whole phrases. To this end, the challenging RedDots corpus is used . The results we reported reaffirm the effectiveness of uncertainty normalization, yielding an impressive 63% relative improvement in terms of .
For future work, we are interested in fitting certain elements of the proposed approach to end-to-end neural architectures. Recently emerged approaches in text-independent speaker recognition combine end-to-end deep learning methods with implicit modeling of acoustic units via multi-head attention and learnable dictionaries or with mimicking the i-vector/PLDA framework [18, 49, 50]. We expect that the proposed method will contribute to this research direction, by demonstrating the potential of digit-specific HMMs and i-vector extractors.
Finally, we should note that the channel and uncertainty compensation approaches examined here may also be applicable to speaker embeddings. Modeling the uncertainty in x-vectors is less straightforward compared to i-vectors. However, recent advances in Bayesian deep learning demonstrate that model averaging via dropouts is a means for quantifying the uncertainty of extracted representations . As a result, uncertainty normalization may also be relevant to neural representations, such as x-vectors.
Themos Stafylakis is funded by the European Commission program Horizon 2020, under grant agreement no. 706668 (Talking Heads).
-  N. Dehak, P. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, “Front-end factor analysis for speaker verification,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 4, pp. 788–798, 2011.
-  S. Ioffe, “Probabilistic linear discriminant analysis,” in Proc. Computer Vision–ECCV 2006. New York, NY, USA: Springer, 2006, pp. 531–542.
-  M. Senoussaoui, P. Kenny, P. Dumouchel, and F. Castaldo, “Well-calibrated heavy tailed Bayesian speaker verification for microphone speech,” in Proc. ICASSP. IEEE, 2011, pp. 4824–4827.
-  G. R. Doddington, M. A. Przybocki, A. F. Martin, and D. A. Reynolds, “The NIST speaker recognition evaluation–overview, methodology, systems, results, perspective,” Speech Communication, vol. 31, no. 2, pp. 225–254, 2000.
-  D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust dnn embeddings for speaker recognition,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 5329–5333.
-  D. Snyder, D. Garcia-Romero, G. Sell, A. McCree, D. Povey, and S. Khudanpur, “Speaker recognition for multi-speaker conversations using x-vectors,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 5796–5800.
-  T. Stafylakis, P. Kenny, P. Ouellet, J. Perez, M. Kockmann, and P. Dumouchel, “Text-dependent speaker recognition using PLDA with uncertainty propagation,” in Proc. Interspeech, 2013, pp. 3684–3688.
-  P. Kenny, T. Stafylakis, P. Ouellet, M. J. Alam, and P. Dumouchel, “PLDA for speaker verification with utterances of arbitrary duration,” Proc. ICASSP, pp. 7649–7653, 2013.
-  S. Cumani, O. Plchot, and P. Laface, “On the use of i-vector posterior distributions in probabilistic linear discriminant analysis,” IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), vol. 22, no. 4, pp. 846–857, 2014.
-  N. Evans, M. Sahidullah, J. Yamagishi, M. Todisco, K. A. Lee, H. Delgado, T. Kinnunen et al., “The 2nd automatic speaker verification spoofing and countermeasures challenge (asvspoof 2017) database,” 2017.
-  M. Todisco, X. Wang, V. Vestman, M. Sahidullah, H. Delgado, A. Nautsch, J. Yamagishi, N. Evans, T. Kinnunen, and K. A. Lee, “Asvspoof 2019: Future horizons in spoofed and fake audio detection,” arXiv preprint arXiv:1904.05441, 2019.
-  A. Larcher, K. A. Lee, B. Ma, and H. Li, “Text-dependent speaker verification: Classifiers, databases and RSR2015,” Speech Communication, vol. 60, pp. 56–77, 2014.
-  A. Larcher, K. A. Lee, and B. Ma, “The RSR2015: Database for text-dependent speaker verification using multiple pass-phrases,” in Proc. Interspeech, 2012.
-  P. Kenny, T. Stafylakis, J. Alam, V. Gupta, and M. Kockmann, “Uncertainty modeling without subspace methods for text-dependent speaker recognition,” in Proc. Odyssey-The Speaker and Language Recognition Workshop, 2016, pp. 16–23.
-  H. Zeinali, L. Burget, H. Sameti, O. Glembek, and O. Plchot, “Deep neural networks and hidden Markov models in i-vector-based text-dependent speaker verification,” in Proc. Odyssey-The Speaker and Language Recognition Workshop, 2016, pp. 24–30.
-  H. Zeinali, H. Sameti, and L. Burget, “HMM-based phrase-independent i-vector extractor for text-dependent speaker verification,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 25, no. 7, pp. 1421–1435, 2017.
-  H. Zeinali, E. Kalantari, H. Sameti, and H. Hadian, “Telephony text-prompted speaker verification using i-vector representation,” in Proc. ICASSP. IEEE, 2015, pp. 4839–4843.
-  Y. Zhu, T. Ko, D. Snyder, B. Mak, and D. Povey, “Self-attentive speaker embeddings for text-independent speaker verification,” Proc. Interspeech 2018, pp. 3573–3577, 2018.
-  Z. Huang, S. Wang, and K. Yu, “Angular softmax for short-duration text-independent speaker verification,” Proc. Interspeech 2018, pp. 3623–3627, 2018.
-  H. Aronowitz, “Text dependent speaker verification using a small development set,” in Proc. Odyssey-The Speaker and Language Recognition Workshop, 2012, pp. 312–316.
-  S. Novoselov, T. Pekhovsky, A. Shulipa, and A. Sholokhov, “Text-dependent GMM-JFA system for password based speaker verification,” in Proc. ICASSP. IEEE, 2014, pp. 729–737.
-  P. Kenny, T. Stafylakis, J. Alam, and M. Kockmann, “An i-vector backend for speaker verification,” in Proc. Interspeech, 2015, pp. 2307–2310.
-  T. Stafylakis, P. Kenny, J. Alam, and M. Kockmann, “JFA for Speaker Recognition with Random Digit Strings,” in Proc. Interspeech, 2015.
-  T. Stafylakis, J. Alam, and P. Kenny, “Text dependent speaker recognition with random digit strings,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 7, pp. 1194–1203, 2016.
-  W.-w. Lin, M.-W. Mak, and J.-T. Chien, “Fast scoring for plda with uncertainty propagation via i-vector grouping,” Computer Speech & Language, vol. 45, pp. 503–515, 2017.
-  W. Lin and M.-W. Mak, “Fast scoring for plda with uncertainty propagation,” in Proc. Odyssey-The Speaker and Language Recognition Workshop, 2016, pp. 31–38.
-  D. Ribas, E. Vincent, and J. R. Calvo, “Uncertainty propagation for noise robust speaker recognition: the case of NIST-SRE,” in Proc. Interspeech, 2015.
-  R. Saeidi and P. Alku, “Accounting for uncertainty of i-vectors in speaker recognition using uncertainty propagation and modified imputation,” in Proc. Interspeech, 2015.
-  R. Saeidi, R. Astudillo, and D. Kolossa, “Uncertain LDA: Including observation uncertainties in discriminative transforms,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 7, pp. 1479–1488, 2015.
-  T. Stafylakis, P. Kenny, M. J. Alam, and M. Kockmann, “Speaker and channel factors in text-dependent speaker recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 1, pp. 65–78, 2016.
-  H. Zeinali, H. Sameti, and N. Maghsoodi, “SUT Submission for NIST 2016 Speaker Recognition Evaluation: Description and Analysis,” in Proc. ROCLING, 2017.
-  V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in Proc. ICASSP. IEEE, 2015, pp. 5206–5210.
-  J. Zhong, W. Hu, F. Soong, and H. Meng, “Dnn i-vector speaker verification with short, text-constrained test utterances,” in Proc. Interspeech, 2017, pp. 1507–1511.
-  M. Karafiát, F. Grézl, K. Veselỳ, M. Hannemann, I. Szőke, and J. Černockỳ, “But 2014 babel system: Analysis of adaptation in nn based systems,” in Proc. Interspeech, 2014.
-  H. Zeinali, H. Sameti, L. Burget et al., “Text-dependent speaker verification based on i-vectors, neural networks and hidden markov models,” Computer Speech & Language, vol. 46, pp. 53–71, 2017.
-  S. O. Sadjadi, M. Slaney, and L. Heck, “MSR identity toolbox v1. 0: A MATLAB toolbox for speaker-recognition research,” Speech and Language Processing Technical Committee Newsletter, vol. 1, no. 4, 2013.
-  D. Garcia-Romero and C. Y. Espy-Wilson, “Analysis of i-vector length normalization in speaker recognition systems.” in in Proc. Interspeech, 2011, pp. 249–252.
-  M. McLaren, L. Ferrer, D. Castan, and A. Lawson, “The speakers in the wild (sitw) speaker recognition database.” in Interspeech, 2016, pp. 818–822.
-  A. Nagrani, J. S. Chung, and A. Zisserman, “Voxceleb: a large-scale speaker identification dataset,” arXiv preprint arXiv:1706.08612, 2017.
-  J. S. Chung, A. Nagrani, and A. Zisserman, “Voxceleb2: Deep speaker recognition,” arXiv preprint arXiv:1806.05622, 2018.
-  J. Rohdin, T. Stafylakis, A. Silnova, H. Zeinali, L. Burget, and O. Plchot, “Speaker verification using end-to-end adversarial language adaptation,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 6006–6010.
-  P. S. Nidadavolu, J. Villalba, and N. Dehak, “Cycle-gans for domain adaptation of acoustic features for speaker recognition,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 6206–6210.
-  Y. Lei, N. Scheffer, L. Ferrer, and M. McLaren, “A novel scheme for speaker recognition using a phonetically-aware deep neural network,” in Proc. ICASSP. IEEE, 2014, pp. 1695–1699.
-  K. A. Lee, A. Larcher, G. Wang, P. Kenny, N. Brümmer, D. v. Leeuwen, H. Aronowitz, M. Kockmann, C. Vaquero, B. Ma et al., “The reddots data collection for speaker recognition,” in Sixteenth Annual Conference of the International Speech Communication Association, 2015.
-  A. Lozano-Diez, A. Silnova, P. Matejka, O. Glembek, O. Plchot, J. Pešán, L. Burget, and J. Gonzalez-Rodriguez, “Analysis and optimization of bottleneck features for speaker recognition,” in Proceedings of Odyssey, vol. 2016, 2016, pp. 352–357.
T. Fu, Y. Qian, Y. Liu, and K. Yu, “Tandem deep features for text-dependent speaker verification,” inFifteenth Annual Conference of the International Speech Communication Association, 2014.
-  S.-X. Zhang, Z. Chen, Y. Zhao, J. Li, and Y. Gong, “End-to-end attention based text-dependent speaker verification,” in Spoken Language Technology Workshop (SLT), 2016 IEEE. IEEE, 2016, pp. 171–178.
-  F. Chowdhury, Q. Wang, I. L. Moreno, and L. Wan, “Attention-based models for text-dependent speaker verification,” arXiv preprint arXiv:1710.10470, 2017.
-  W. Cai, J. Chen, and M. Li, “Exploring the encoding layer and loss function in end-to-end speaker and language recognition system,” arXiv preprint arXiv:1804.05160, 2018.
-  J. Rohdin, A. Silnova, M. Diez, O. Plchot, P. Matějka, and L. Burget, “End-to-end dnn based speaker recognition inspired by i-vector and plda,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 4874–4878.
Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing
model uncertainty in deep learning,” in
International Conference on Machine Learning (ICML), 2016, pp. 1050–1059.