Robust and Scalable Differentiable Neural Computer for Question Answering

07/07/2018
by   Jörg Franke, et al.
0

Deep learning models are often not easily adaptable to new tasks and require task-specific adjustments. The differentiable neural computer (DNC), a memory-augmented neural network, is designed as a general problem solver which can be used in a wide range of tasks. But in reality, it is hard to apply this model to new tasks. We analyze the DNC and identify possible improvements within the application of question answering. This motivates a more robust and scalable DNC (rsDNC). The objective precondition is to keep the general character of this model intact while making its application more reliable and speeding up its required training time. The rsDNC is distinguished by a more robust training, a slim memory unit and a bidirectional architecture. We not only achieve new state-of-the-art performance on the bAbI task, but also minimize the performance variance between different initializations. Furthermore, we demonstrate the simplified applicability of the rsDNC to new tasks with passable results on the CNN RC task without adaptions.

READ FULL TEXT
research
07/21/2020

Distributed Memory based Self-Supervised Differentiable Neural Computer

A differentiable neural computer (DNC) is a memory augmented neural netw...
research
01/25/2018

Finding ReMO (Related Memory Object): A Simple Neural Architecture for Text based Reasoning

To solve the text-based question and answering task that requires relati...
research
04/23/2019

Improving Differentiable Neural Computers Through Memory Masking, De-allocation, and Link Distribution Sharpness Control

The Differentiable Neural Computer (DNC) can learn algorithmic and quest...
research
05/25/2019

Neural Stored-program Memory

Neural networks powered with external memory simulate computer behaviors...
research
06/12/2018

A Question-Answering framework for plots using Deep learning

Deep Learning has managed to push boundaries in a wide variety of tasks....
research
07/17/2017

Visual Question Answering with Memory-Augmented Networks

This paper exploits a memory-augmented neural network to predict accurat...
research
06/13/2023

Improving Opinion-based Question Answering Systems Through Label Error Detection and Overwrite

Label error is a ubiquitous problem in annotated data. Large amounts of ...

Please sign up or login with your details

Forgot password? Click here to reset