Phase Conductor on Multi-layered Attentions for Machine Comprehension

10/28/2017
by   Rui Liu, et al.
0

Attention models have been intensively studied to improve NLP tasks such as machine comprehension via both question-aware passage attention model and self-matching attention model. Our research proposes phase conductor (PhaseCond) for attention models in two meaningful ways. First, PhaseCond, an architecture of multi-layered attention models, consists of multiple phases each implementing a stack of attention layers producing passage representations and a stack of inner or outer fusion layers regulating the information flow. Second, we extend and improve the dot-product attention function for PhaseCond by simultaneously encoding multiple question and passage embedding layers from different perspectives. We demonstrate the effectiveness of our proposed model PhaseCond on the SQuAD dataset, showing that our model significantly outperforms both state-of-the-art single-layered and multiple-layered attention models. We deepen our results with new findings via both detailed qualitative analysis and visualized examples showing the dynamic changes through multi-layered attention models.

READ FULL TEXT
research
04/07/2020

Two Results on Layered Pathwidth and Linear Layouts

Layered pathwidth is a new graph parameter studied by Bannister et al (2...
research
04/07/2023

Multi-Layered Unseen Garments Draping Network

While recent AI-based draping networks have significantly advanced the a...
research
10/06/2018

Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences

Learning a matching function between two text sequences is a long standi...
research
05/17/2023

Towards Multi-Layered 3D Garments Animation

Mimicking realistic dynamics in 3D garment animations is a challenging t...
research
11/22/2022

Layered-Garment Net: Generating Multiple Implicit Garment Layers from a Single Image

Recent research works have focused on generating human models and garmen...
research
12/03/2020

Source location on multilayer networks

Nowadays it is not uncommon to have to deal with dissemination on multi-...
research
02/02/2019

Parametric FEM for Shape Optimization applied to Golgi Stack

The thesis is about an application of the shape optimization to the morp...

Please sign up or login with your details

Forgot password? Click here to reset