Co-training for Deep Object Detection: Comparing Single-modal and Multi-modal Approaches

04/23/2021
by   Jose L. Gomez, et al.
10

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human labeling, which is time-consuming and does not scale as we wish. This data labeling bottleneck may be intensified due to domain shifts among image sensors, which could force per-sensor data labeling. In this paper, we focus on the use of co-training, a semi-supervised learning (SSL) method, for obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep object detectors. In particular, we assess the goodness of multi-modal co-training by relying on two different views of an image, namely, appearance (RGB) and estimated depth (D). Moreover, we compare appearance-based single-modal co-training with multi-modal. Our results suggest that in a standard SSL setting (no domain shift, a few human-labeled data) and under virtual-to-real domain shift (many virtual-world labeled data, no human-labeled data) multi-modal co-training outperforms single-modal. In the latter case, by performing GAN-based domain translation both co-training modalities are on pair; at least, when using an off-the-shelf depth estimation model not specifically trained on the translated images.

READ FULL TEXT

page 2

page 4

page 8

page 11

page 12

page 14

research
08/12/2020

Co-training for On-board Deep Object Detection

Providing ground truth supervision to train visual models has been a bot...
research
03/19/2017

Weakly-supervised DCNN for RGB-D Object Recognition in Real-World Applications Which Lack Large-scale Annotated Training Data

This paper addresses the problem of RGBD object recognition in real-worl...
research
03/22/2021

Monocular Depth Estimation through Virtual-world Supervision and Real-world SfM Self-Supervision

Depth information is essential for on-board perception in autonomous dri...
research
05/10/2023

A Multi-modal Approach to Single-modal Visual Place Classification

Visual place classification from a first-person-view monocular RGB image...
research
03/20/2012

Semi-Supervised Single- and Multi-Domain Regression with Multi-Domain Training

We address the problems of multi-domain and single-domain regression bas...
research
04/11/2018

Offline Object Extraction from Dynamic Occupancy Grid Map Sequences

A dynamic occupancy grid map (DOGMa) allows a fast, robust, and complete...
research
01/13/2023

Learning Transformations To Reduce the Geometric Shift in Object Detection

The performance of modern object detectors drops when the test distribut...

Please sign up or login with your details

Forgot password? Click here to reset