Causal Scene BERT: Improving object detection by searching for challenging groups of data

02/08/2022
by   Cinjon Resnick, et al.
7

Modern computer vision applications rely on learning-based perception modules parameterized with neural networks for tasks like object detection. These modules frequently have low expected error overall but high error on atypical groups of data due to biases inherent in the training process. In building autonomous vehicles (AV), this problem is an especially important challenge because their perception modules are crucial to the overall system performance. After identifying failures in AV, a human team will comb through the associated data to group perception failures that share common causes. More data from these groups is then collected and annotated before retraining the model to fix the issue. In other words, error groups are found and addressed in hindsight. Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated scenes. To keep our interventions on the data manifold, we utilize masked language models. We verify that the prioritized groups found via intervention are challenging for the object detector and show that retraining with data collected from these groups helps inordinately compared to adding more IID data. We also plan to release software to run interventions in simulated scenes, which we hope will benefit the causality community.

READ FULL TEXT

page 3

page 7

page 27

research
12/06/2022

Adaptive Testing of Computer Vision Models

Vision models often fail systematically on groups of data that share com...
research
07/31/2023

Detecting Out-of-distribution Objects Using Neuron Activation Patterns

Object detection is essential to many perception algorithms used in mode...
research
06/06/2018

Causal Interventions for Fairness

Most approaches in algorithmic fairness constrain machine learning metho...
research
05/18/2023

MonoTDP: Twin Depth Perception for Monocular 3D Object Detection in Adverse Scenes

3D object detection plays a crucial role in numerous intelligent vision ...
research
08/03/2021

ODIP: Towards Automatic Adaptation for Object Detection by Interactive Perception

Object detection plays a deep role in visual systems by identifying inst...
research
07/25/2022

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

In modern autonomy stacks, prediction modules are paramount to planning ...
research
04/24/2017

Towards Instance Segmentation with Object Priority: Prominent Object Detection and Recognition

This manuscript introduces the problem of prominent object detection and...

Please sign up or login with your details

Forgot password? Click here to reset