Attention Augmented Convolutional Transformer for Tabular Time-series

Time-series classification is one of the most frequently performed tasks in industrial data science, and one of the most widely used data representation in the industrial setting is tabular representation. In this work, we propose a novel scalable architecture for learning representations from tabular time-series data and subsequently performing downstream tasks such as time-series classification. The representation learning framework is end-to-end, akin to bidirectional encoder representations from transformers (BERT) in language modeling, however, we introduce novel masking technique suitable for pretraining of time-series data. Additionally, we also use one-dimensional convolutions augmented with transformers and explore their effectiveness, since the time-series datasets lend themselves naturally for one-dimensional convolutions. We also propose a novel timestamp embedding technique, which helps in handling both periodic cycles at different time granularity levels, and aperiodic trends present in the time-series data. Our proposed model is end-to-end and can handle both categorical and continuous valued inputs, and does not require any quantization or encoding of continuous features.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/26/2023

Improving Position Encoding of Transformers for Multivariate Time Series Classification

Transformers have demonstrated outstanding performance in many applicati...
research
11/03/2020

Tabular Transformers for Modeling Multivariate Time Series

Tabular datasets are ubiquitous in data science applications. Given thei...
research
02/20/2023

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

Deep learning-based algorithms, e.g., convolutional networks, have signi...
research
02/26/2021

Beyond Convolutions: A Novel Deep Learning Approach for Raw Seismic Data Ingestion

Traditional seismic processing workflows (SPW) are expensive, requiring ...
research
08/21/2018

Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks

Recently, researchers have started applying convolutional neural network...
research
09/12/2022

Classification of hazard event via language fractal

HAZOP is a safety paradigm undertaken to reveal hazards in industry, its...
research
03/26/2018

Locality-Sensitive Hashing for Earthquake Detection: A Case Study Scaling Data-Driven Science

In this work, we report on a novel application of Locality Sensitive Has...

Please sign up or login with your details

Forgot password? Click here to reset