Detecting Everything in the Open World: Towards Universal Object Detection

03/21/2023
by   Zhenyu Wang, et al.
0

In this paper, we formally address universal object detection, which aims to detect every scene and predict every category. The dependence on human annotations, the limited visual information, and the novel categories in the open world severely restrict the universality of traditional detectors. We propose UniDetector, a universal object detector that has the ability to recognize enormous categories in the open world. The critical points for the universality of UniDetector are: 1) it leverages images of multiple sources and heterogeneous label spaces for training through the alignment of image and text spaces, which guarantees sufficient information for universal representations. 2) it generalizes to the open world easily while keeping the balance between seen and unseen classes, thanks to abundant information from both vision and language modalities. 3) it further promotes the generalization ability to novel categories through our proposed decoupling training manner and probability calibration. These contributions allow UniDetector to detect over 7k categories, the largest measurable category size so far, with only about 500 classes participating in training. Our UniDetector behaves the strong zero-shot generalization ability on large-vocabulary datasets like LVIS, ImageNetBoxes, and VisualGenome - it surpasses the traditional supervised baselines by more than 4% on average without seeing any corresponding images. On 13 public detection datasets with various scenes, UniDetector also achieves state-of-the-art performance with only a 3% amount of training data.

READ FULL TEXT
research
09/18/2023

Object2Scene: Putting Objects in Context for Open-Vocabulary 3D Detection

Point cloud-based open-vocabulary 3D object detection aims to detect 3D ...
research
04/04/2023

Towards Open-Vocabulary Video Instance Segmentation

Video Instance Segmentation(VIS) aims at segmenting and categorizing obj...
research
08/01/2018

Open Category Detection with PAC Guarantees

Open category detection is the problem of detecting "alien" test instanc...
research
05/18/2023

Going Denser with Open-Vocabulary Part Segmentation

Object detection has been expanded from a limited number of categories t...
research
03/12/2023

Towards Universal Vision-language Omni-supervised Segmentation

Existing open-world universal segmentation approaches usually leverage C...
research
03/04/2023

CapDet: Unifying Dense Captioning and Open-World Detection Pretraining

Benefiting from large-scale vision-language pre-training on image-text p...
research
11/09/2020

Closing the Generalization Gap in One-Shot Object Detection

Despite substantial progress in object detection and few-shot learning, ...

Please sign up or login with your details

Forgot password? Click here to reset