Complex QA and language models hybrid architectures, Survey

by   Xavier Daull, et al.

This paper reviews the state-of-the-art of language models architectures and strategies for "complex" question-answering (QA, CQA, CPS) with a focus on hybridization. Large Language Models (LLM) are good at leveraging public data on standard problems but once you want to tackle more specific complex questions or problems (e.g. How does the concept of personal freedom vary between different cultures ? What is the best mix of power generation methods to reduce climate change ?) you may need specific architecture, knowledge, skills, methods, sensitive data protection, explainability, human approval and versatile feedback... Recent projects like ChatGPT and GALACTICA have allowed non-specialists to grasp the great potential as well as the equally strong limitations of LLM in complex QA. In this paper, we start by reviewing required skills and evaluation techniques. We integrate findings from the robust community edited research papers BIG, BLOOM and HELM which open source, benchmark and analyze limits and challenges of LLM in terms of tasks complexity and strict evaluation on accuracy (e.g. fairness, robustness, toxicity, ...) as a baseline. We discuss some challenges associated with complex QA, including domain adaptation, decomposition and efficient multi-step QA, long form and non-factoid QA, safety and multi-sensitivity data protection, multimodal search, hallucinations, explainability and truthfulness, temporal reasoning. We analyze current solutions and promising research trends, using elements such as: hybrid LLM architectural patterns, training and prompting strategies, active human reinforcement learning supervised with AI, neuro-symbolic and structured knowledge grounding, program synthesis, iterated decomposition and others.


page 1

page 2

page 3

page 4


Conversational Question Answering: A Survey

Question answering (QA) systems provide a way of querying the informatio...

QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering

The problem of answering questions using knowledge from pre-trained lang...

Question Answering over Curated and Open Web Sources

The last few years have seen an explosion of research on the topic of au...

Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models

Reasoning about time is of fundamental importance. Many facts are time-d...

chatIPCC: Grounding Conversational AI in Climate Science

Large Language Models (LLMs) have made significant progress in recent ye...

A Survey on Multi-hop Question Answering and Generation

The problem of Question Answering (QA) has attracted significant researc...

Uncertainty-Aware Text-to-Program for Question Answering on Structured Electronic Health Records

Question Answering on Electronic Health Records (EHR-QA) has a significa...

Please sign up or login with your details

Forgot password? Click here to reset