DeepAI AI Chat
Log In Sign Up

Inflected Forms Are Redundant in Question Generation Models

by   Xingwu Sun, et al.

Neural models with an encoder-decoder framework provide a feasible solution to Question Generation (QG). However, after analyzing the model vocabulary we find that current models (both RNN-based and pre-training based) have more than 23% inflected forms. As a result, the encoder will generate separate embeddings for the inflected forms, leading to a waste of training data and parameters. Even worse, in decoding these models are vulnerable to irrelevant noise and they suffer from high computational costs. In this paper, we propose an approach to enhance the performance of QG by fusing word transformation. Firstly, we identify the inflected forms of words from the input of encoder, and replace them with the root words, letting the encoder pay more attention to the repetitive root words. Secondly, we propose to adapt QG as a combination of the following actions in the encode-decoder framework: generating a question word, copying a word from the source sequence or generating a word transformation type. Such extension can greatly decrease the size of predicted words in the decoder as well as noise. We apply our approach to a typical RNN-based model and UniLM to get the improved versions. We conduct extensive experiments on SQuAD and MS MARCO datasets. The experimental results show that the improved versions can significantly outperform the corresponding baselines in terms of BLEU, ROUGE-L and METEOR as well as time cost.


page 1

page 2

page 3

page 4


Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

This paper tackles the reduction of redundant repeating generation that ...

Transformer Based Bengali Chatbot Using General Knowledge Dataset

An AI chatbot provides an impressive response after learning from the tr...

Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models

Encoder-decoder models typically only employ words that are frequently u...

Generating Chinese Classical Poems with RNN Encoder-Decoder

We take the generation of Chinese classical poem lines as a sequence-to-...

Code-switching pre-training for neural machine translation

This paper proposes a new pre-training method, called Code-Switching Pre...

Generating a Common Question from Multiple Documents using Multi-source Encoder-Decoder Models

Ambiguous user queries in search engines result in the retrieval of docu...

Joint Copying and Restricted Generation for Paraphrase

Many natural language generation tasks, such as abstractive summarizatio...