DeepAI AI Chat
Log In Sign Up

Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series

by   Kristoffer Wickstrøm, et al.
University of Tromsø the Arctic University of Norway

The lack of labeled data is a key challenge for learning useful representation from time series data. However, an unsupervised representation framework that is capable of producing high quality representations could be of great value. It is key to enabling transfer learning, which is especially beneficial for medical applications, where there is an abundance of data but labeling is costly and time consuming. We propose an unsupervised contrastive learning framework that is motivated from the perspective of label smoothing. The proposed approach uses a novel contrastive loss that naturally exploits a data augmentation scheme in which new samples are generated by mixing two data samples with a mixing component. The task in the proposed framework is to predict the mixing component, which is utilized as soft targets in the loss function. Experiments demonstrate the framework's superior performance compared to other representation learning approaches on both univariate and multivariate time series and illustrate its benefits for transfer learning for clinical time series.


Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification

Learning time-series representations when only unlabeled data or few lab...

Utilizing Expert Features for Contrastive Learning of Time-Series Representations

We present an approach that incorporates expert knowledge for time-serie...

Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding

Time series are often complex and rich in information but sparsely label...

Time Series Contrastive Learning with Information-Aware Augmentations

Various contrastive learning approaches have been proposed in recent yea...

Contrastive Learning for Time Series on Dynamic Graphs

There have been several recent efforts towards developing representation...

Variational PSOM: Deep Probabilistic Clustering with Self-Organizing Maps

Generating visualizations and interpretations from high-dimensional data...

Cross Reconstruction Transformer for Self-Supervised Time Series Representation Learning

Unsupervised/self-supervised representation learning in time series is c...