Residual Attention Net for Superior Cross-Domain Time Sequence Modeling

01/13/2020
by   Seth H. Huang, et al.
0

We present a novel architecture, residual attention net (RAN), which merges a sequence architecture, universal transformer, and a computer vision architecture, residual net, with a high-way architecture for cross-domain sequence modeling. The architecture aims at addressing the long dependency issue often faced by recurrent-neural-net-based structures. This paper serves as a proof-of-concept for a new architecture, with RAN aiming at providing the model a higher level understanding of sequence patterns. To our best knowledge, we are the first to propose such an architecture. Out of the standard 85 UCR data sets, we have achieved 35 state-of-the-art results with 10 results matching current state-of-the-art results without further model fine-tuning. The results indicate that such architecture is promising in complex, long-sequence modeling and may have vast, cross-domain applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/25/2023

RestNet: Boosting Cross-Domain Few-Shot Segmentation with Residual Transformation Network

Cross-domain few-shot segmentation (CD-FSS) aims to achieve semantic seg...
research
11/15/2022

Hybrid Transformers for Music Source Separation

A natural question arising in Music Source Separation (MSS) is whether l...
research
04/09/2019

Seq2Biseq: Bidirectional Output-wise Recurrent Neural Networks for Sequence Modelling

During the last couple of years, Recurrent Neural Networks (RNN) have re...
research
10/18/2022

Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs

The extraction of aspect terms is a critical step in fine-grained sentim...
research
03/25/2022

A Cross-Domain Approach for Continuous Impression Recognition from Dyadic Audio-Visual-Physio Signals

The impression we make on others depends not only on what we say, but al...
research
04/05/2019

NL-FIIT at SemEval-2019 Task 9: Neural Model Ensemble for Suggestion Mining

In this paper, we present neural model architecture submitted to the Sem...
research
10/06/2019

Parallel Segregation-Integration Networks for Shared-account Cross-domain Sequential Recommendations

Sequential Recommendation (SR) has been attracting a growing attention f...

Please sign up or login with your details

Forgot password? Click here to reset