End-to-End Fine-Grained Action Segmentation and Recognition Using Conditional Random Field Models and Discriminative Sparse Coding

01/29/2018
by   Effrosyni Mavroudi, et al.
0

Fine-grained action segmentation and recognition is an important yet challenging task. Given a long, untrimmed sequence of kinematic data, the task is to classify the action at each time frame and segment the time series into the correct sequence of actions. In this paper, we propose a novel framework that combines a temporal Conditional Random Field (CRF) model with a powerful frame-level representation based on discriminative sparse coding. We introduce an end-to-end algorithm for jointly learning the weights of the CRF model, which include action classification and action transition costs, as well as an overcomplete dictionary of mid-level action primitives. This results in a CRF model that is driven by sparse coding features obtained using a discriminative dictionary that is shared among different actions and adapted to the task of structured output learning. We evaluate our method on three surgical tasks using kinematic data from the JIGSAWS dataset, as well as on a food preparation task using accelerometer data from the 50 Salads dataset. Our results show that the proposed method performs on par or better than state-of-the-art methods.

READ FULL TEXT
research
04/24/2018

Fine-grained Video Classification and Captioning

We describe a DNN for fine-grained action classification and video capti...
research
08/30/2020

Action similarity judgment based on kinematic primitives

Understanding which features humans rely on – in visually recognizing ac...
research
07/20/2022

Spotting Temporally Precise, Fine-Grained Events in Video

We introduce the task of spotting temporally precise, fine-grained event...
research
11/20/2019

CAT: CRF-based ASR Toolkit

In this paper, we present a new open source toolkit for automatic speech...
research
08/04/2020

Learning Discriminative Feature with CRF for Unsupervised Video Object Segmentation

In this paper, we introduce a novel network, called discriminative featu...
research
12/18/2014

Deep Structured Output Learning for Unconstrained Text Recognition

We develop a representation suitable for the unconstrained recognition o...
research
07/23/2019

CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology

This paper presents the submission by the CMU-01 team to the SIGMORPHON ...

Please sign up or login with your details

Forgot password? Click here to reset