Data Augmentation for Object Detection via Differentiable Neural Rendering

by   Guanghan Ning, et al.

It is challenging to train a robust object detector when annotated data is scarce. Existing approaches to tackle this problem include semi-supervised learning that interpolates labeled data from unlabeled data, self-supervised learning that exploit signals within unlabeled data via pretext tasks. Without changing the supervised learning paradigm, we introduce an offline data augmentation method for object detection, which semantically interpolates the training data with novel views. Specifically, our proposed system generates controllable views of training images based on differentiable neural rendering, together with corresponding bounding box annotations which involve no human intervention. Firstly, we extract and project pixel-aligned image features into point clouds while estimating depth maps. We then re-project them with a target camera pose and render a novel-view 2d image. Objects in the form of keypoints are marked in point clouds to recover annotations in new views. It is fully compatible with online data augmentation methods, such as affine transform, image mixup, etc. Extensive experiments show that our method, as a cost-free tool to enrich images and labels, can significantly boost the performance of object detection systems with scarce training data. Code is available at <>.


page 3

page 5

page 6

page 7

page 8

page 11

page 12

page 13


Point-Teaching: Weakly Semi-Supervised Object Detection with Point Annotations

Point annotations are considerably more time-efficient than bounding box...

Self-Supervised Object Detection via Generative Image Synthesis

We present SSOD, the first end-to-end analysis-by synthesis framework wi...

OPERA: Omni-Supervised Representation Learning with Hierarchical Supervisions

The pretrain-finetune paradigm in modern computer vision facilitates the...

A Simple Semi-Supervised Learning Framework for Object Detection

Semi-supervised learning (SSL) has promising potential for improving the...

Compound Figure Separation of Biomedical Images with Side Loss

Unsupervised learning algorithms (e.g., self-supervised learning, auto-e...

Towards Human-Machine Cooperation: Self-supervised Sample Mining for Object Detection

Though quite challenging, leveraging large-scale unlabeled or partially ...

Kit-Net: Self-Supervised Learning to Kit Novel 3D Objects into Novel 3D Cavities

In industrial part kitting, 3D objects are inserted into cavities for tr...

Please sign up or login with your details

Forgot password? Click here to reset