DeepAI AI Chat
Log In Sign Up

ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classification

04/29/2019
by   Kathan Kashiparekh, et al.
Tata Consultancy Services
0

Training deep neural networks often requires careful hyper-parameter tuning and significant computational resources. In this paper, we propose ConvTimeNet (CTN): an off-the-shelf deep convolutional neural network (CNN) trained on diverse univariate time series classification (TSC) source tasks. Once trained, CTN can be easily adapted to new TSC target tasks via a small amount of fine-tuning using labeled instances from the target tasks. We note that the length of convolutional filters is a key aspect when building a pre-trained model that can generalize to time series of different lengths across datasets. To achieve this, we incorporate filters of multiple lengths in all convolutional layers of CTN to capture temporal features at multiple time scales. We consider all 65 datasets with time series of lengths up to 512 points from the UCR TSC Benchmark for training and testing transferability of CTN: We train CTN on a randomly chosen subset of 24 datasets using a multi-head approach with a different softmax layer for each training dataset, and study generalizability and transferability of the learned filters on the remaining 41 TSC datasets. We observe significant gains in classification accuracy as well as computational efficiency when using pre-trained CTN as a starting point for subsequent task-specific fine-tuning compared to existing state-of-the-art TSC approaches. We also provide qualitative insights into the working of CTN by: i) analyzing the activations and filters of first convolution layer suggesting the filters in CTN are generically useful, ii) analyzing the impact of the design decision to incorporate multiple length decisions, and iii) finding regions of time series that affect the final classification decision via occlusion sensitivity analysis.

READ FULL TEXT
04/01/2019

Transfer Learning for Clinical Time Series Analysis using Deep Neural Networks

Deep neural networks have shown promising results for various clinical p...
03/25/2019

Depth Augmented Networks with Optimal Fine-tuning

Convolutional neural networks (CNN) have been shown to achieve state-of-...
07/04/2018

Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks

Deep neural networks have shown promising results for various clinical p...
11/06/2014

How transferable are features in deep neural networks?

Many deep neural networks trained on natural images exhibit a curious ph...
08/25/2014

Convolutional Neural Networks for Sentence Classification

We report on a series of experiments with convolutional neural networks ...
10/06/2017

Efficient K-Shot Learning with Regularized Deep Networks

Feature representations from pre-trained deep neural networks have been ...
09/29/2021

Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning

A convolutional neural network (ConvNet) is usually trained and then tes...