KEYword based Sampling (KEYS) for Large Language Models

by   Jyothir S V, et al.

Question answering (Q/A) can be formulated as a generative task (Mitra, 2017) where the task is to generate an answer given the question and the passage (knowledge, if available). Recent advances in QA task is focused a lot on language model advancements and less on other areas such as sampling(Krishna et al., 2021), (Nakano et al., 2021). Keywords play very important role for humans in language generation. (Humans formulate keywords and use grammar to connect those keywords and work). In the research community, very little focus is on how humans generate answers to a question and how this behavior can be incorporated in a language model. In this paper, we want to explore these two areas combined, i.e., how sampling can be to used generate answers which are close to human-like behavior and factually correct. Hence, the type of decoding algorithm we think should be used for Q/A tasks should also depend on the keywords. These keywords can be obtained from the question, passage or internet results. We use knowledge distillation techniques to extract keywords and sample using these extracted keywords on top of vanilla decoding algorithms when formulating the answer to generate a human-like answer. In this paper, we show that our decoding method outperforms most commonly used decoding methods for Q/A task


page 1

page 2

page 3

page 4


Tag and Correct: Question aware Open Information Extraction with Two-stage Decoding

Question Aware Open Information Extraction (Question aware Open IE) take...

How much should you ask? On the question structure in QA systems

Datasets that boosted state-of-the-art solutions for Question Answering ...

Fluent Response Generation for Conversational Question Answering

Question answering (QA) is an important aspect of open-domain conversati...

Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding

This work aims at decreasing the end-to-end generation latency of large ...

Prompting Vision Language Model with Knowledge from Large Language Model for Knowledge-Based VQA

Knowledge-based visual question answering is a very challenging and wide...

Learning to Apply Schematic Knowledge to Novel Instances

Humans have schematic knowledge of how certain types of events unfold (e...

GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Ranking

Retrieval-enhanced text generation, which aims to leverage passages retr...

Please sign up or login with your details

Forgot password? Click here to reset