Kernel Clustering with Sigmoid-based Regularization for Efficient Segmentation of Sequential Data

06/22/2021
by   Tung Doan, et al.
0

Kernel segmentation aims at partitioning a data sequence into several non-overlapping segments that may have nonlinear and complex structures. In general, it is formulated as a discrete optimization problem with combinatorial constraints. A popular algorithm for optimally solving this problem is dynamic programming (DP), which has quadratic computation and memory requirements. Given that sequences in practice are too long, this algorithm is not a practical approach. Although many heuristic algorithms have been proposed to approximate the optimal segmentation, they have no guarantee on the quality of their solutions. In this paper, we take a differentiable approach to alleviate the aforementioned issues. First, we introduce a novel sigmoid-based regularization to smoothly approximate the combinatorial constraints. Combining it with objective of the balanced kernel clustering, we formulate a differentiable model termed Kernel clustering with sigmoid-based regularization (KCSR), where the gradient-based algorithm can be exploited to obtain the optimal segmentation. Second, we develop a stochastic variant of the proposed model. By using the stochastic gradient descent algorithm, which has much lower time and space complexities, for optimization, the second model can perform segmentation on overlong data sequences. Finally, for simultaneously segmenting multiple data sequences, we slightly modify the sigmoid-based regularization to further introduce an extended variant of the proposed model. Through extensive experiments on various types of data sequences performances of our models are evaluated and compared with those of the existing methods. The experimental results validate advantages of the proposed models. Our Matlab source code is available on github.

READ FULL TEXT

page 2

page 3

page 6

page 8

page 9

page 10

page 11

research
11/03/2017

Analysis of Approximate Stochastic Gradient Using Quadratic Constraints and Sequential Semidefinite Programs

We present convergence rate analysis for the approximate stochastic grad...
research
05/28/2018

Strongly polynomial efficient approximation scheme for segmentation

Partitioning a sequence of length n into k coherent segments is one of t...
research
11/06/2015

Streaming regularization parameter selection via stochastic gradient descent

We propose a framework to perform streaming covariance selection. Our ap...
research
06/11/2020

Optimizing generalization on the train set: a novel gradient-based framework to train parameters and hyperparameters simultaneously

Generalization is a central problem in Machine Learning. Most prediction...
research
12/22/2019

LinearFold: linear-time approximate RNA folding by 5'-to-3' dynamic programming and beam search

Motivation: Predicting the secondary structure of an RNA sequence is use...
research
06/23/2020

Differentiable Segmentation of Sequences

Segmented models are widely used to describe non-stationary sequential d...
research
03/11/2021

BODAME: Bilevel Optimization for Defense Against Model Extraction

Model extraction attacks have become serious issues for service provider...

Please sign up or login with your details

Forgot password? Click here to reset