DeepAI AI Chat
Log In Sign Up

Searching for Search Errors in Neural Morphological Inflection

by   Martina Forster, et al.
ETH Zurich

Neural sequence-to-sequence models are currently the predominant choice for language generation tasks. Yet, on word-level tasks, exact inference of these models reveals the empty string is often the global optimum. Prior works have speculated this phenomenon is a result of the inadequacy of neural models for language generation. However, in the case of morphological inflection, we find that the empty string is almost never the most probable solution under the model. Further, greedy search often finds the global optimum. These observations suggest that the poor calibration of many neural models may stem from characteristics of a specific subset of tasks rather than general ill-suitedness of such models for language generation.


page 1

page 2

page 3

page 4


Exact Hard Monotonic Attention for Character-Level Transduction

Many common character-level, string-to-string transduction tasks, e.g., ...

String Transduction with Target Language Models and Insertion Handling

Many character-level tasks can be framed as sequence-to-sequence transdu...

Do RNN States Encode Abstract Phonological Processes?

Sequence-to-sequence models have delivered impressive results in word fo...

Evaluating Contextualized Language Models for Hungarian

We present an extended comparison of contextualized language models for ...

Neural Generation for Czech: Data and Baselines

We present the first dataset targeted at end-to-end NLG in Czech in the ...

Evaluating Neural Morphological Taggers for Sanskrit

Neural sequence labelling approaches have achieved state of the art resu...

Back to Patterns: Efficient Japanese Morphological Analysis with Feature-Sequence Trie

Accurate neural models are much less efficient than non-neural models an...