1 Foundations
Stochastic finite automata are used in many language and speech processing tasks, including speech recognition and synthesis, language and pronunciation modeling, tagging and parsing, among many others. They arise naturally in optimization and decoding tasks for probabilistic sequence models.
1.1 FST background
A weighted finite state transducer (WFST) is a tuple where and are finite sets (the input and output alphabet, respectively), is a semiring of weights, is a finite set of states, is the initial state, is a function mapping a state to its final weight, is a finite set of edges, and is a function mapping an edge to its weight. Given an edge define canonical projections , , , and .
A path is a finite sequence of consecutive edges such that for . In this case we write and , and we extend the weight function to paths:
Associated with a path is an input string as well as a similarly defined output string. The set of all successful paths labeled with given strings and is . The behavior of the WFST is the function , and its grand total weight is :
(1) 
A stochastic FST (SFST) is a WFST defined over the semiring of nonnegative real numbers . An SFST is globally normalized if ; in that case is a probability measure for the countable sample space . Normalization is impossible if the grand total diverges, due to cycles with path weights . An SFST is locally normalized if
An SFST can be brought into locally normalized form (implying global normalization) using the WFST weight pushing algorithm [1, pp. 241–43]. This is illustrated in Figure 1.
On the left, the grand total weight is ; on the right, .
From an SFST a random path can be drawn by starting at state , drawing an outgoing edge with probability , and iterating this process at the next state . At any state , terminate with probability .
Claim: If is a random path through , then is a random string pair distributed according to . Reason: The morphism defined for successful paths by has preimage , which is measurable by (1). By construction, is an arbitrary path in this countable set of disjoint events and was drawn with probability .
In other words, we can sample random string pairs by sampling random paths. This allows us to gloss over the fact that an SFST gives rise to both a probability space over paths and a probability space over string pairs. We write to refer to a random string pair with distribution .
It follows that sampling from SFSTs is unaffected by WFST optimizations which leave the behavior of an automaton intact. This includes weighted epsilonremoval, determinization, and minimization [1]; as well as disambiguation [2]. This means that and are equal as random elements, for any combination of optimizations . We can in effect sample by sampling paths from . As we will see below, this can be efficient in situations where determinization or disambiguation would lead to an exponential size increase.
Sampling from SFSTs can be effective even when operations would change their behavior. Isomorphisms such as FST reversal or inversion are a particularly simple case: If we want to sample from the reverse of , we can either construct and then sample from ; or, equivalently, sample directly from and then reverse the sampled paths/strings.
More generally, this strategy also works for certain morphisms of string pairs (such as morphisms of arcs that only change arc labels, including FST projection) as well as composition with unweighted functional FSTs. For example, sampling from is equivalent to drawing a random string pair from and returning .
1.2 Epsiloncycle conflation
The strategy of sampling random string pairs by sampling random paths can be inefficient for SFSTs with cyclic epsilon graphs, especially in the presence of highprobability epsilon cycles. If the probability of the epsilonloop at state 1 in Figure 1 is , then a random path will contain repetitions of that edge on average, which can be problematic for small , i.e. loop probabilities close to . Sampling strings by sampling paths is not efficient in this situation, as the path sampling algorithm has to repeatedly expand the epsilonloop despite the fact that it contributes no symbols to the string labeling.
One remedy is to apply the weighted epsilonremoval algorithm [1, pp. 233–37]. We illustrate this using an SFST with a nontrivial stronglyconnected component (SCC) of its epsilongraph, highlighted in Figure 2. The result of epsilonremoval is depicted in the lefthand side of Figure 3. Because epsilonremoval eliminated all epsilontransitions, new outgoing transitions were added with appropriate weights for all outgoing transitions from any state in the original epsilonSCC.
As we had seen above, in the context of sampling the presence of epsilontransitions is not necessarily problematic. It would suffice to simply ensure that the epsilongraph is acyclic. This can be achieved by a modification of the epsilonremoval algorithm, which we call epsiloncycle conflation. Rather than removing all epsilontransitions, this algorithm merely replaces every set of epsilonpaths through an epsilonSCC with a single epsilontransition. This is illustrated in the righthand side of Figure 3; pseudocode appears in Figure 4.
The main property of the epsiloncycle conflation algorithm is that it splits each state within an epsilonSCC into two states, by adding a new state that is paired with an existing state. Transitions from within the SCC are removed (line 16); transitions from outside the SCC are redirected to reach the new state (line 18). Each SCC is replaced with a complete bipartite graph: for each newly split state, epsilontransitions are added to all other original states within the SCC (forloop starting on line 4). In other words, the bipartite graph that replaces the SCC encodes the allpairs algebraic distance within the SCC. The algorithm references standard subroutines for computing the SCCs (line 2), for computing the singlesource algebraic distance (line 6), and for trimming useless states (line 21), as well as standard WFST accessor functions for adding and deleting states and arcs. It also uses a simple arcfilter called EpsilonSccArcFilter (line 5 and 13), which is a predicate that returns true iff a given arc is an epsilontransition within the specified SCC. It is assumed that the algebraic distance subroutine ShortestDistance can take this filter as an argument, so as to restrict it to the specified SCC within the epsilongraph.^{1}^{1}1This functionality is readily available in the OpenFst library [3], which forms the basis of our implementation.
Unlike epsilonremoval, epsiloncycle conflation only affects the states and arcs within each epsilonSCC. It does not add or remove any nonepsilon transitions or epsilontransitions not fully within epsilonSCCs. If the sizes of the epsilonSCCs are , epsiloncycle conflation adds at most additional states and no more than additional transitions. A worst case arises if the entire FST is the cycle graph with states and epsilonarcs, in which case epsiloncycle conflation will result in states and arcs before trimming.
From here on we assume that any SFST we sample from is locally normalized and epsiloncycle free. If it is not, we first apply epsiloncycle conflation, followed by weight pushing.
2 CTC decoding by sampling
We now turn to an illustration of sampling from stochastic automata in the context of the decoding problem for Connectionist Temporal Classification (CTC) [4]. The CTC decoding problem arises because the sequences generated as CTC outputs do not correspond directly to the final label sequences. CTC output is a dimensional matrix
where each vector
represents the probability distribution at time over symbols from a finite alphabet of labels with cardinality augmented with a special blank symbol .A standard way (e.g. [5]) of conceptualizing CTC output is as an equivalent weighted finite state acceptor (WFSA). Figure 5 shows a WFSA view of a CTC lattice corresponding to a sequence of length of symbols from . The cost of transitioning into state having accepted symbol is .
From a path through the CTC lattice we get to a final labeling sequence by first collapsing contiguous runs of repeated symbols and then removing blank symbols. For example, the path labeled with "" gives rise to the labeling "". Following [4] we denote the pathtolabeling function as .
Crucially, the pathtolabeling function can be computed by an unweighted functional FST, which we also refer to as . In order to collapse contiguous runs, the FST only needs to remember the last symbol seen, i.e. it has the structure of a bigram model over its input labels. This is illustrated in the top FST in Figure 6, where e.g. state 1 is reached whenever the input label ‘a’ is read in any state. On reaching state 1 from another state, ‘a’ is also output, but subsequent occurrences of ‘a’ are mapped to the empty string. A similar construction was used for the Token WFST in [5].
For a given SFST over a label alphabet and pathtolabeling transducer
, the posterior predictive distribution over labeling sequences
corresponds to the outputprojection of the WFST composition :(2) 
In maximum a posteriori (MAP) decoding we are asked to find . It is easy to define more complex decoding tasks over the labeling distribution from (2).
The key point of this paper is that we can sample efficiently from the labeling distribution for many choices of and , including but not limited to CTC decoding, where exact MAP decoding via disambiguation or determinization is generally only tractable for unrealistically small WFSTs. When involves only operations discussed in Section 1 above, we know that pathsampling is effective. All we have to do is sample paths from the SFST pass them through the unweighted functional FST , and read labeling strings off the output tape.
This immediately leads us to a naive decoding strategy: Sample a large number of labeling strings and return the labeling which occurs most frequently in the sample, i.e. the sample mode. The advantage is that this is very easy to implement. On the other hand, the naive strategy often needs to sample a large number of paths, both to ensure that there are no unexplored modes and to distinguish labelings that are nearly equally likely.
The naive strategy can be improved by evaluating the probability from (2) for a given labeling or set of labelings . We do this by expressing the preimage as a WFST (see the bottom FST in Figure 6) and measuring that preimage using . For singleton the number of states and arcs of the preimage FST is linear in by construction. Evaluating amounts to computing the grand total weight from (1) of using standard WFST operations. It is important to note that this efficiently sums over an implicitly represented WFST that compactly encodes an exponentially larger set of paths. This means that, while we could compute lower bounds for via sampling and path probabilities alone, those bounds are so loose as to be of little practical value, due to the large number of paths for the same labeling.
The ability to evaluate (2) is useful in several ways. For example, if we know that for a particular labeling , then must necessarily be the mode of . Generalizing from here gives us our samplingbased CTC decoding strategy (pseudocode appears in Figure 7): Repeatedly draw using pathbased sampling (lines 12–13), compute
(line 19) and track the most probable labeling seen (lines 3–4 and 21–22) as well as the probability mass of all seen labelings (lines 10 and 20). We always include the labeling for the most likely path (line 2), a fast and useful heuristic already noted in
[4]. If the probability of the most probable seen labeling exceeds the unseen probability mass (lines 5 and 23), then that labeling must be the global mode. For efficiency the decoding procedure takes parameters specifying a maximum number of random draws and a threshold for early stopping.There are many ways to conceptualize when to stop sampling (line 25), most generally as an optimal stopping problem [6], which subsumes sequential probability ratio tests. We want to stop sampling when we are reasonably certain that we have already seen the global mode.^{2}^{2}2
The early stopping strategies are applicable because we can draw iid samples from the posterior predictive distribution. If instead we sampled correlated variates sequentially, there would be a substantial risk that e.g. a Markov chain sampler might get stuck near a local maximum.
Since in this strategy we can observe the probabilities of labelings, we could hypothesize that there is an unseen mode that occurs with probability, which is a random variable that we assume follows a
distribution. The probability of the unseen mode would have to exceed the highest labeling probability encountered so far, and cannot exceed the unseen probability mass . We can stop sampling as soon as this becomes sufficiently unlikely, when .While the evaluation of labeling probabilities provides useful information that guides early stopping, it is often not necessary to compute for every labeling string sampled randomly. Like the naive sampling strategy, our decoding algorithm keeps a count of labelings (lines 7–8 and 14–16). The count (line 16) of the current labeling , together with other information, is passed to a predicate ComputeProbability indicating whether should be evaluated during a given iteration (the probability computation strategy), which can be implemented in a variety of ways. If it is always true, probabilities are computed for every distinct labeling, which can improve early stopping. If it is always false and if the most frequent labeling in is returned, this becomes the naive sampling algorithm. ComputeProbability could also be implemented (similar to the negation of the early stopping criterion) as where is a random variable with a distribution, so we compute only when this value is likely to exceed . Finally, the simple criterion works well in our experience: only evaluate the probability of a labeling when seeing it for the second time.
The decoding algorithm can thus be finetuned for many practical considerations, and many additional variations are possible, such as sampling in batches and only occasionally testing for early stopping. In the following section we compare different decoding strategies, including the decoding algorithm from Figure 7 and the naive sampling algorithm.
3 Evaluation and discussion
Our experiments focus on continuous phoneme recognition, which is a sequencetosequence task where the input sequence corresponds to the acoustic features and the output is a sequence of categorical labels representing phonemes from a finite inventory. CTCbased approaches have been successfully applied to this task [7, 8, 9, 10]. We trained a CTC phoneme recognizer for Argentinian Spanish^{3}^{3}3Our corpus is available at http://openslr.org/61/. based on the architecture described in [11]. We computed CTC output matrices on a test set of 90 unseen utterances and turned those into CTC lattices.
Decoding strategy,  Mode  Avg. paths  Avg. probs.  

probability computation strategy  found  sampled  computed  
1  , n/a  83%  
2  , n/a  83%  
3  , never  82%  600  0 
4  , never  94%  6000  0 
5  , never  77%  0  0 
6  , always  99%  36  27 
7  , always  100%  53  40 
8  , if  100%  53  7 
We compared different decoding strategies in terms of their ability to find the global mode of the posterior predictive labeling distribution. We first ran an exhaustive search to locate the global mode. We then compared different decoding strategies in terms of how often they were able to locate the global mode. The results are summarized in Table 1
. The first two rows refer to the CTC beam search decoding algorithm of TensorFlow
[12], which we ran with its default beamwidth of and then again with a much larger beamwidth of , which did not affect the results of beam search.The third and fourth row of Table 1 show results for the naive sampling strategy, which simply returns the mode of a large sample. Because no early stopping is employed, the average number of paths sampled per utterance is identical to the parameter of Figure 7. In the naive sampling strategy the number of probability computations is zero by design. This strategy can match or exceed beamsearch in its ability to locate the global mode, but this comes at a substantial cost, as a large number of paths needs to be sampled. We experimented with early stopping for naive sampling before deciding against it: while early stopping can ensure that there are no unseen modes, a reduction in sample size made it much harder to correctly identify the global mode within the sample.
The fifth row of Table 1 is the bestpath heuristic mentioned in [4]. It only considers the labeling of the most likely path and does not sample any additional paths. While it performs worst in this comparison, it is a useful heuristic due to speed.
The last three rows of Table 1 are nontrivial instances of our decoding algorithm. We fixed a confidence threshold of . Larger values result in fewer overall computations, at the expense of possibly not finding the global mode. Smaller values increase the overall computations. We then chose a maximum number of draws, which always leads to early stopping: the algorithm from Figure 7 either finds the global mode (lines 6 and 24) or stops early with confidence (line 26). When running with a sample limit of paths, our algorithm always found the global mode in our experiments. Even with a much smaller limit of at most random draws, our algorithm still did well. Finally, as the last row shows, the average number of probability evaluations can be greatly reduced by computing labeling probabilities only when a labeling is encountered more than once. The last three strategies shown here all outperform beam search.
4 Conclusion
We have discussed sufficient conditions under which sampling random paths from transformations of stochastic automata yields random strings from the transformed automata. We have shown that this can be used to sample labeling sequences from CTC posterior lattices. In a comparison on a connected phoneme recognition problem, samplingbased decoding informed by posterior probabilities was able to locate posterior modes reliably with minimal search effort.
References
 [1] M. Mohri, “Weighted automata algorithms,” in Handbook of Weighted Automata, M. Droste, W. Kuich, and H. Vogler, Eds. Springer, 2009, pp. 213–254.
 [2] M. Mohri and M. D. Riley, “A Disambiguation Algorithm for Weighted Automata,” Theoretical Computer Science, vol. 679, pp. 53–68, 2017.
 [3] C. Allauzen, M. Riley, J. Schalkwyk, W. Skut, and M. Mohri, “OpenFst: A General and Efficient Weighted FiniteState Transducer Library,” in CIAA 2007: Implementation and Application of Automata, 2007, pp. 11–23.

[4]
A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks,” in
Proc. 23rd International Conference on Machine Learning (ICML)
, Pittsburgh, Pennsylvania, Jun. 2006, pp. 369–376.  [5] Y. Miao, M. Gowayyed, and F. Metze, “EESEN: Endtoend speech recognition using deep RNN models and WFSTbased decoding,” in Automatic Speech Recognition and Understanding Workshop (ASRU), Scottsdale, Arizona, Dec. 2015.
 [6] T. S. Ferguson, “Optimal stopping and applications,” 2008, http://www.math.ucla.edu/~tom/Stopping/.
 [7] S. Fernández, A. Graves, and J. Schmidhuber, “Phoneme recognition in TIMIT with BLSTMCTC,” arXiv preprint arXiv:0804.3269, 2008.
 [8] A. Graves, A. R. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Canada, May 2013, pp. 6645–6649.
 [9] H. Sak, F. de Chaumont Quitry, T. Sainath, K. Rao et al., “Acoustic modelling with CDCTCSMBR LSTM RNNs,” in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Scottsdale, Arizona, Dec. 2015, pp. 604–609.
 [10] G. Pundak and T. N. Sainath, “Lower Frame Rate Neural Network Acoustic Models,” in Interspeech 2016, San Francisco, USA, Sep. 2016, pp. 22–26.
 [11] O. Adams, T. Cohn, G. Neubig, H. Cruz, S. Bird, and A. Michaud, “Evaluating phonemic transcription of lowresource tonal languages for language documentation,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Japan, 2018, pp. 3356–3365.
 [12] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al., “TensorFlow: A System for LargeScale Machine Learning,” in Proc. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16), 2016, pp. 265–283.
Comments
There are no comments yet.