Multi-Modal Prototype Learning for Interpretable Multivariable Time Series Classification

by   Gaurav R. Ghosal, et al.

Multivariable time series classification problems are increasing in prevalence and complexity in a variety of domains, such as biology and finance. While deep learning methods are an effective tool for these problems, they often lack interpretability. In this work, we propose a novel modular prototype learning framework for multivariable time series classification. In the first stage of our framework, encoders extract features from each variable independently. Prototype layers identify single-variable prototypes in the resulting feature spaces. The next stage of our framework represents the multivariable time series sample points in terms of their similarity to these single-variable prototypes. This results in an inherently interpretable representation of multivariable patterns, on which prototype learning is applied to extract representative examples i.e. multivariable prototypes. Our framework is thus able to explicitly identify both informative patterns in the individual variables, as well as the relationships between the variables. We validate our framework on a simulated dataset with embedded patterns, as well as a real human activity recognition problem. Our framework attains comparable or superior classification performance to existing time series classification methods on these tasks. On the simulated dataset, we find that our model returns interpretations consistent with the embedded patterns. Moreover, the interpretations learned on the activity recognition dataset align with domain knowledge.


page 9

page 11


UMSNet: An Universal Multi-sensor Network for Human Activity Recognition

Human activity recognition (HAR) based on multimodal sensors has become ...

Multivariable times series classification through an interpretable representation

Multivariate time series classification is a task with increasing import...

Learning Interpretable Shapelets for Time Series Classification through Adversarial Regularization

Times series classification can be successfully tackled by jointly learn...

TS-MoCo: Time-Series Momentum Contrast for Self-Supervised Physiological Representation Learning

Limited availability of labeled physiological data often prohibits the u...

DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing

Mobile sensing applications usually require time-series inputs from sens...

Boosting Crop Classification by Hierarchically Fusing Satellite, Rotational, and Contextual Data

Accurate in-season crop type classification is crucial for the crop prod...

A Multi-Media Exchange Format for Time-Series Dataset Curation

Exchanging data as character-separated values (CSV) is slow, cumbersome ...

Please sign up or login with your details

Forgot password? Click here to reset