ST-CoNAL: Consistency-Based Acquisition Criterion Using Temporal Self-Ensemble for Active Learning

by   Jae Soon Baik, et al.

Modern deep learning has achieved great success in various fields. However, it requires the labeling of huge amounts of data, which is expensive and labor-intensive. Active learning (AL), which identifies the most informative samples to be labeled, is becoming increasingly important to maximize the efficiency of the training process. The existing AL methods mostly use only a single final fixed model for acquiring the samples to be labeled. This strategy may not be good enough in that the structural uncertainty of a model for given training data is not considered to acquire the samples. In this study, we propose a novel acquisition criterion based on temporal self-ensemble generated by conventional stochastic gradient descent (SGD) optimization. These self-ensemble models are obtained by capturing the intermediate network weights obtained through SGD iterations. Our acquisition function relies on a consistency measure between the student and teacher models. The student models are given a fixed number of temporal self-ensemble models, and the teacher model is constructed by averaging the weights of the student models. Using the proposed acquisition criterion, we present an AL algorithm, namely student-teacher consistency-based AL (ST-CoNAL). Experiments conducted for image classification tasks on CIFAR-10, CIFAR-100, Caltech-256, and Tiny ImageNet datasets demonstrate that the proposed ST-CoNAL achieves significantly better performance than the existing acquisition methods. Furthermore, extensive experiments show the robustness and effectiveness of our methods.


page 1

page 2

page 3

page 4


Weighted Ensembles for Active Learning with Adaptivity

Labeled data can be expensive to acquire in several application domains,...

TrustAL: Trustworthy Active Learning using Knowledge Distillation

Active learning can be defined as iterations of data labeling, model tra...

Distilling and Transferring Knowledge via cGAN-generated Samples for Image Classification and Regression

Knowledge distillation (KD) has been actively studied for image classifi...

Prediction stability as a criterion in active learning

Recent breakthroughs made by deep learning rely heavily on large number ...

Spatial Ensemble: a Novel Model Smoothing Mechanism for Student-Teacher Framework

Model smoothing is of central importance for obtaining a reliable teache...

A Self-ensembling Framework for Semi-supervised Knee Osteoarthritis Localization and Classification with Dual-Consistency

Knee osteoarthritis (OA) is one of the most common musculoskeletal disor...

Active Ensemble Deep Learning for Polarimetric Synthetic Aperture Radar Image Classification

Although deep learning has achieved great success in image classificatio...

Please sign up or login with your details

Forgot password? Click here to reset