Localized Vision-Language Matching for Open-vocabulary Object Detection

05/12/2022
by   Maria A. Bravo, et al.
11

In this work, we propose an open-world object detection method that, based on image-caption pairs, learns to detect novel object classes along with a given set of known classes. It is a two-stage training approach that first uses a location-guided image-caption matching technique to learn class labels for both novel and known classes in a weakly-supervised manner and second specializes the model for the object detection task using known class annotations. We show that a simple language model fits better than a large contextualized language model for detecting novel objects. Moreover, we introduce a consistency-regularization technique to better exploit image-caption pair information. Our method compares favorably to existing open-world detection approaches while being data-efficient.

READ FULL TEXT

page 1

page 3

page 7

page 8

page 14

page 15

research
03/28/2022

Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model

Recently, vision-language pre-training shows great potential in open-voc...
research
10/13/2022

ImaginaryNet: Learning Object Detectors without Real Images and Annotations

Without the demand of training in reality, humans can easily detect a kn...
research
03/25/2021

Few-shot Weakly-Supervised Object Detection via Directional Statistics

Detecting novel objects from few examples has become an emerging topic i...
research
11/18/2019

Large Scale Open-Set Deep Logo Detection

We present an open-set logo detection (OSLD) system, which can detect (l...
research
06/08/2023

Multi-Modal Classifiers for Open-Vocabulary Object Detection

The goal of this paper is open-vocabulary object detection (OVOD) x2013 ...
research
10/19/2022

Learning to Discover and Detect Objects

We tackle the problem of novel class discovery, detection, and localizat...
research
03/22/2022

Open-Vocabulary DETR with Conditional Matching

Open-vocabulary object detection, which is concerned with the problem of...

Please sign up or login with your details

Forgot password? Click here to reset