Parameter-Efficient Abstractive Question Answering over Tables or Text

04/07/2022
by   Vaishali Pal, et al.
1

A long-term ambition of information seeking QA systems is to reason over multi-modal contexts and generate natural answers to user queries. Today, memory intensive pre-trained language models are adapted to downstream tasks such as QA by fine-tuning the model on QA data in a specific modality like unstructured text or structured tables. To avoid training such memory-hungry models while utilizing a uniform architecture for each modality, parameter-efficient adapters add and train small task-specific bottle-neck layers between transformer layers. In this work, we study parameter-efficient abstractive QA in encoder-decoder models over structured tabular data and unstructured textual data using only 1.5 modality. We also ablate over adapter layers in both encoder and decoder modules to study the efficiency-performance trade-off and demonstrate that reducing additional trainable parameters down to 0.7 results. Our models out-perform current state-of-the-art models on tabular QA datasets such as Tablesum and FeTaQA, and achieve comparable performance on a textual QA dataset such as NarrativeQA using significantly less trainable parameters than fine-tuning.

READ FULL TEXT

page 8

page 13

research
10/16/2021

EncT5: Fine-tuning T5 Encoder for Non-autoregressive Tasks

Encoder-decoder transformer architectures have become popular recently w...
research
11/30/2022

A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering

Question Answering (QA) is a growing area of research, often used to fac...
research
05/03/2022

DrugEHRQA: A Question Answering Dataset on Structured and Unstructured Electronic Health Records For Medicine Related Queries

This paper develops the first question answering dataset (DrugEHRQA) con...
research
05/15/2023

Parameter-Efficient Fine-Tuning with Layer Pruning on Free-Text Sequence-to-Sequence modeling

The increasing size of language models raises great research interests i...
research
10/12/2020

End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems

We propose an end-to-end approach for synthetic QA data generation. Our ...
research
06/29/2023

Unified Language Representation for Question Answering over Text, Tables, and Images

When trying to answer complex questions, people often rely on multiple s...
research
05/23/2022

QASem Parsing: Text-to-text Modeling of QA-based Semantics

Several recent works have suggested to represent semantic relations with...

Please sign up or login with your details

Forgot password? Click here to reset