Finding relevant information in input features that is necessary for an output/task
has seen a surge of interest in the computer vision[1, 2, 3, 4, 5]
and reinforcement learning communities[6, 7, 8].  proposed the information bottleneck approach to address the problem and [10, 11] used the idea to improve model generalization. Our previous work proposed a correlational method to find regions of speech spectrograms that are important to a listener’s correctly identifying the words it contains, and we applied it to both humans and automatic speech recognition (ASR) systems [12, 13, 14]
. These “importance maps” or “saliency maps” reveal how the ASR uses speech features to derive the recognition. In this paper, we propose a method to evaluate the quality of predicted importance maps and apply them to saliency maps estimated for an ASR “listener.”
The saliency map in speech has a similar meaning to the saliency map in computer vision. However, unlike in vision, where ground truth can be obtained from eye-tracking systems, in speech, we do not have a corresponding ‘ear-tracking’ system. With the human saliency map as the gold label, the predicted visual saliency map can be evaluated using different metrics, such as area under the curve (AUC) , correlation coefficient , similarity , or information gain . For speech, where the human saliency map is unavailable, we propose a method to assess the quality of a predicted saliency map in this paper. The main idea of our approach is that the better the predicted saliency map, the higher the accuracy when the ASR uses only information from the important regions of the spectrogram. Similarly, if the important regions are removed from an observation, the ASR should have low accuracy.
To the best of our knowledge, we are among the first, if not the first, to propose a method to evaluate the saliency map of running sentences, a structured prediction problem. In computer vision, there is related work on evaluation methods for saliency maps in simple classification problems without ground truth.  proposed the MoRF method (Most Relevant First) to evaluate saliency maps by measuring model performance degradation when the most relevant pixels are replaced by random values.  introduced the complementary LeRF method (Least Relevant First), where the least relevant features are removed.  recommended evaluating with a score measuring the area between the MoRF and LeRF curves created when the number of pixels is varied.
, we propose here an evaluation metric, the SSBM, that measures accuracy degradation when the most or least important time-frequency points are replaced with white noise in a structured prediction setting. A fundamental difference between our approach and these others is that they evaluate the accuracy of a single simple classifier, such as an image classifier. Our approach, on the other hand, compares the recognition performance of a target word to that of the other words in the sentence. We also characterize these values in comparison to the amount of speech energy preserved in a mixture.
The main idea of our method is to evaluate the quality of the predicted time-frequency importance regions for an utterance. Denote the predicted importance maps in the speech spectrogram from method for word as , a binary matrix indicating whether time-frequency point is important for the recognition of (1) or not (0). If the ASR can correctly recognize the speech using only the regions where instead of using all the spectrogram points, and if it cannot recognize the speech when presented with only the regions where , then we can conclude that method has successfully identified the important regions. To do this, we perform two tests. In the first case, we add noise everywhere in a sentence except the predicted important regions of , which is equivalent to dropping the least relevant features (LeRF). In the second case, we add noise to the predicted important regions for , equivalent to dropping the most relevant features (MoRF). If the ASR can recognize in the first case and not the second, then the importance region is correct. In addition, we also make sure that in these same two cases, the other words in the sentence are not correctly recognized when including the regions, and that the other words in the sentence are correctly recognized when excluding the regions.
We define a new metric that we call the structured saliency benchmark (SSBM) to evaluate the accuracy of the analyzed words with respect to the accuracy of other words in the sentence and the predicted important speech energies.
where is the accuracy of analyzed word , is the averaged accuracy of the other words, is the percentage of the least important energy that is dropped (dropped energy is divided by utterance energy), and is the percentage of the most important energy that is dropped. Thus, represents the accuracy of the analyzed word per unit (percentage) of energy, with the accuracy of other words as a penalty. We can see that if the importance maps of are correct, then when the least important energy for is removed, the accuracy of , , should be high while the accuracy of other words, , should be low. Additionally, for two different importance maps with the same and , the map corresponding to higher (more unimportant energy is dropped) should be better as should the one with the lower (less important energy is preserved).
In this paper, we analyze the importance maps of two different approaches. The first is a bubble analysis method where a time-frequency point is predicted to be important when its audibility in noise is significantly correlated with speech intelligibility [12, 13]. The second is an energy-based baseline, where a time-frequency point in the spectrogram is predicted to be important when its energy is larger than a certain threshold.
2.1 Bubble analysis approach
The bubble analysis method [12, 13] identifies important regions by adding many instances of random noise to clean speech, then finding the spectrogram points that are revealed when the ASR recognizes the noisy speech correctly and hidden by noise when the ASR fails to recognize the utterance. Specifically, the noisy utterances are generated by adding many instances of random white noise to the clean speech to make these utterances inaudible. However, the noise level is decreased inside randomly placed oval-shaped bubbles to reveal the speech information inside. Denote as the intelligibility, which has value one or zero (binary) when the ASR correctly or incorrectly recognizes the th word in the th noisy mixture of the th clean utterance. In addition, the audibility is defined as a continuous variable that represents the inverse of the amount of noise added to a time-frequency point in a spectrogram, varying between zero (maximum noise) and one (no noise). A point-biserial correlation is computed between and . The significance (
-value) of this correlation is examined under a two-sided t-test for every time-frequency point in the spectrogram. The importance map is defined as the set of time-frequency points that have positive correlation and -values smaller than a specific threshold.
2.2 Energy-based approach
In the energy-based approach, a time-frequency point in the spectrogram is considered important when its energy in a smoothed version of the spectrogram is greater than a certain threshold. Specifically, the linear frequency spectrogram has pre-emphasis applied, is converted to a mel spectrogram with 30 bins, and then is converted back to a linear frequency axis. The importance map of a word is then the set of high energy spectrogram points that are between the start and end frame of the target word in the forced alignment of the clean utterance produced by Kaldi.
2.3 LeRF and MoRF noise masks
The LeRF mask is created by adding maximum noise to unimportant regions while adding minimum noise to important regions. There is a transition between the two as shown in the top plot of Figure 1. The intention is that when maximum noise is added outside the important regions of a specific word, then the ASR should still be able to recognize this word, but should not be able to recognize the other words in the sentence. The procedure is slightly different for the two mask prediction algorithms, so each is described separately below.
The bubble analysis LeRF mask , at a single point is
where is the threshold, is the p-value of time-frequency points in the spectrogram, is a parameter controlling the size of the transition region while and control the minimum and maximum value of the mask, respectively.
The green line in the top plot in Figure 1 illustrates mask values for , . In addition, , leading to a minimum mask value of and maximum value of . As shown in this figure, a time-frequency point with a -value larger than has noise level (maximum noise), while a point with a -value smaller than has noise level . Additionally, a visualization of a complete mask with threshold is shown in the second row of Figure 2.
The bubble analysis MoRF mask is derived in a similar way as equations (3) and (4), however with . The red line in the top plot of Figure 1 shows the MoRF mask with the same parameters as the green line. Besides, visualization of the mask are shown in the top plot of Figure 3. In general, we can observe that the MoRF and LeRF mask are almost complementary to each other, but are not exactly because we always have the masks provide decay smoothly towards 0 to mirror the logarithmic nature of intensity perception.
Similarly, the LeRF mask for the energy-based approach is created by adding maximum noise to the time-frequency region with energy lower than a specific threshold in decibels (unimportant regions). The important regions have minimum noise added, except the transition area. The mask is defined as
where is the absolute magnitude of the time-frequency point in the spectrogram and is the threshold in magnitude. An example of the mask with a specific threshold dB is illustrated in the bottom plot of Figure 1 and Figure 4.
The energy-based MoRF mask is formed by adding maximum noise to the time-frequency region with energy bigger than a specific threshold. The mask is derived the same as equations (5) and (6) except with .
To create the noisy speech, we multiply the spectrogram of a white noise signal by the mask and add the masked noise to the clean speech. Examples of the mask and the masked noisy speech are shown in the second and third rows of Figure 2.
3 Experimental setup
We utilize the AMI dataset , which includes 100 hours of English meeting recordings. We selected the Individual Headset Microphone (IHM) channels for our experiment. We followed the standard train/test split and chose 100 sentences from the test set where the recognizer achieved 100% accuracy without additional noise added to be our set of clean speech. We created 1000 noisy mixtures for every clean utterance, leading to a dataset of 100,000 mixtures for the bubble analysis method.
We use Kaldi 
as the ASR to perform the experiments. We choose the standard model in AMI recipe s5b with a time-delay neural network (TDNN) acoustic model and an-gram language model from the SRI Language Modeling Toolkit (SRILM) 
) from the preceding layer. The time-domain utterances are sampled at 16 kHz and are transformed into the frequency domain using an STFT with window length 64 ms, and hop length 16 ms.
For the bubble analysis technique, we choose , and . We perform experiments with 25 different values of threshold that are spaced evenly on a log scale from to . For the energy-based technique, we use the same values of , however we use 21 different values of thresholds , spaced evenly on a linear scale from to with a step size of .
Here, we compare the bubble analysis and the energy-based approaches according to LeRF and MoRF curves and SSBM scores. Figure 6 allows a direct comparison between the two mask methods by characterizing each masked signal by the proportion of speech energy in the entire utterance that it obscures. This proportion could vary for different words at the same threshold, so this plot averages over masks that have the same proportion when rounded to the nearest percent.
The top plot of Figure 6 shows the accuracy of analyzed words when the least important features are dropped, averaged over the entire dataset. Perfect performance in this case would be in the top right corner, obscuring almost all of the speech while preserving recognition accuracy. In general, we can see that the bubble analysis method (blue line) achieves slightly higher accuracy than the energy-based method (orange line). For instance, by dropping the least important energy, the energy-based and bubble analysis masks have average word accuracy and , respectively.
The bottom plot of Figure 6 shows the accuracy of analyzed words when the most important features are dropped on all 100 sentences. A perfect MoRF mask would be in the bottom left corner of the bottom plot, obscuring almost none of the speech while destroying recognition accuracy. This plot demonstrates that the bubble analysis method is better at reducing recognition accuracy than the energy-based method when both drop the same amount of important speech energy. In both plots, the orange lines are shorter than the blue lines because the important regions of a word are restricted to be between the start frame and end frame in the energy-based approach.
Figure 7 shows the SSMB scores (green lines) at various thresholds for both methods. For the bubble analysis method in the top row, we can see that the threshold of obtains the best SSBM score of 6.5. This means that the increase in LeRF accuracy at higher thresholds is not worth the decrease in MoRF accuracy. For the energy-based method in the second row, the threshold of dB achieves the highest SSMB score of 4.7, which is worse than that of the bubble analysis method. Thus, the bubble analysis method produces better importance maps than the energy-based approach according to the LeRF and MoRF curves and the SSBM score.
4.1 Other discussions
First, we can see that the ASR does not need to observe all of the speech energy of a word to correctly identify it. For illustration, the ASR can recognize the word “actually” with a threshold as low as on bubble analysis LeRF mask as in Figure 5 (blue line). This mask and its corresponding noisy speech are illustrated in the second and third row of Figure 2. As we can see, the mask only spans from 400 Hz to 3200 Hz. Surprisingly, the clean speech lacks energy at those frequencies, but this does not prevent the ASR from correctly identifying the word.
Second, the threshold identifying which time-frequency points are important is varied across word. For example, in Figure 5 (blue line), the ASR needs to use all time-frequency points with -value to correctly identify the word “but”, however, the ASR must use all spectrogram points with -value to recognize the words “more”.
Figure 8 shows a possible explanation of why the threshold for correct recognition varies across words. It shows the threshold at which a target words transitions from correct to incorrect recognition as a function of word length in phonemes. We can see that longer words typically require a higher LeRF threshold, meaning more speech is revealed, while they typically require a lower MoRF threshold, meaning less speech is obscured. Similar trends were observed with word length measured in syllables and characters.
5 Conclusion and future work
In this paper, we proposed an evaluation metric for structured saliency maps, where we measure the word accuracy when either keeping or dropping the most important regions. A gap in this accuracy is measured between the analyzed word and other words in the sentence with respect to the predicted important speech energies. Additionally, we perform a large scale saliency map analysis with a bubble analysis method and energy-based baseline on sentences from the AMI meeting corpus. According to several metrics, the bubble analysis approach achieves a better importance map than its alternative. In the future, we will extend this evaluation to different methods to compare different speech importance maps and use these importance maps to enhance speech recognition robustness in noisy conditions. We also hope that this speech saliency evaluation metric can facilitate a community evaluation on the topic of speech saliency, similar to those that have been organized around visual saliency .
-  M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in European conference on computer vision. Springer, 2014, pp. 818–833.
D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K.-R.
MÃžller, “How to explain individual classification decisions,”
Journal of Machine Learning Research, vol. 11, no. Jun, pp. 1803–1831, 2010.
-  D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg, “Smoothgrad: removing noise by adding noise,” arXiv preprint arXiv:1706.03825, 2017.
-  M. T. Ribeiro, S. Singh, and C. Guestrin, “” why should i trust you?” explaining the predictions of any classifier,” in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135–1144.
M. Ancona, E. Ceolini, C. Öztireli, and M. Gross, “A unified view of
gradient-based attribution methods for deep neural networks,” in
NIPS Workshop on Interpreting, Explaining and Visualizing Deep Learning-Now What?(NIPS 2017). ETH Zurich, 2017.
-  S. Mohamed and D. J. Rezende, “Variational information maximisation for intrinsically motivated reinforcement learning,” in Advances in neural information processing systems, 2015, pp. 2125–2133.
-  A. Goyal, Y. Bengio, M. Botvinick, and S. Levine, “The variational bandwidth bottleneck: Stochastic evaluation on an information budget,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=Hye1kTVFDS
-  A. Goyal, R. Islam, D. Strouse, Z. Ahmed, M. Botvinick, H. Larochelle, Y. Bengio, and S. Levine, “Infobot: Transfer and exploration via the information bottleneck,” arXiv preprint arXiv:1901.10902, 2019.
-  N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” arXiv preprint physics/0004057, 2000.
-  A. Achille and S. Soatto, “Information dropout: learning optimal representations through noise,” 2016.
-  A. A. Alemi, I. Fischer, J. V. Dillon, and K. Murphy, “Deep variational information bottleneck,” arXiv preprint arXiv:1612.00410, 2016.
-  M. I. Mandel, S. E. Yoho, and E. W. Healy, “Measuring time-frequency importance functions of speech with bubble noise,” Journal of the Acoustical Society of America, vol. 140, pp. 2542–2553, 2016. [Online]. Available: http://m.mr-pc.org/work/jasa16.pdf
-  V. A. Trinh and M. I. Mandel, “Directly comparing the listening strategies of humans and machines,” IEEE/ACM Transactions on Audio, Speech, and Language Processing Under Review, 2020.
-  V. A. Trinh, B. McFee, and M. I. Mandel, “Bubble cooperative networks for identifying important speech cues,” Proc. Interspeech 2018, pp. 1616–1620, 2018.
-  B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, “Visual correlates of fixation selection: Effects of scale and time,” Vision research, vol. 45, no. 5, pp. 643–659, 2005.
-  T. Jost, N. Ouerhani, R. Von Wartburg, R. Müri, and H. Hügli, “Assessing the contribution of color in visual attention,” Computer Vision and Image Understanding, vol. 100, no. 1-2, pp. 107–123, 2005.
-  Z. Bylinskii, T. Judd, A. Borji, L. Itti, F. Durand, A. Oliva, and A. Torralba, “Mit saliency benchmark,” 2015.
-  M. Kümmerer, T. S. Wallis, and M. Bethge, “Information-theoretic model comparison unifies saliency metrics,” Proceedings of the National Academy of Sciences, vol. 112, no. 52, pp. 16 054–16 059, 2015.
-  W. Samek, A. Binder, G. Montavon, S. Lapuschkin, and K.-R. Müller, “Evaluating the visualization of what a deep neural network has learned,” IEEE transactions on neural networks and learning systems, vol. 28, no. 11, pp. 2660–2673, 2016.
-  K. Schulz, L. Sixt, F. Tombari, and T. Landgraf, “Restricting the flow: Information bottlenecks for attribution,” arXiv preprint arXiv:2001.00396, 2020.
-  J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, W. Kraaij, M. Kronenthal et al., “The ami meeting corpus: A pre-announcement,” in International Workshop on Machine Learning for Multimodal Interaction. Springer, 2005, pp. 28–39.
-  D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz et al., “The kaldi speech recognition toolkit,” in Proc. ASRU, 2011.
-  A. Stolcke, “Srilm-an extensible language modeling toolkit,” in Seventh international conference on spoken language processing, 2002.
-  M. Kümmerer, Z. Bylinskii, T. Judd, A. Borji, L. Itti, F. Durand, A. Oliva, and A. Torralba, “Mit/tübingen saliency benchmark,” https://saliency.tuebingen.ai/.