Tell Me What They're Holding: Weakly-supervised Object Detection with Transferable Knowledge from Human-object Interaction

by   Daesik Kim, et al.

In this work, we introduce a novel weakly supervised object detection (WSOD) paradigm to detect objects belonging to rare classes that have not many examples using transferable knowledge from human-object interactions (HOI). While WSOD shows lower performance than full supervision, we mainly focus on HOI as the main context which can strongly supervise complex semantics in images. Therefore, we propose a novel module called RRPN (relational region proposal network) which outputs an object-localizing attention map only with human poses and action verbs. In the source domain, we fully train an object detector and the RRPN with full supervision of HOI. With transferred knowledge about localization map from the trained RRPN, a new object detector can learn unseen objects with weak verbal supervision of HOI without bounding box annotations in the target domain. Because the RRPN is designed as an add-on type, we can apply it not only to the object detection but also to other domains such as semantic segmentation. The experimental results on HICO-DET dataset show the possibility that the proposed method can be a cheap alternative for the current supervised object detection paradigm. Moreover, qualitative results demonstrate that our model can properly localize unseen objects on HICO-DET and V-COCO datasets.


page 2

page 3

page 5

page 6

page 8

page 9

page 10

page 12


Boosting Weakly Supervised Object Detection with Progressive Knowledge Transfer

In this paper, we propose an effective knowledge transfer framework to b...

Beyond Bounding Box: Multimodal Knowledge Learning for Object Detection

Multimodal supervision has achieved promising results in many visual lan...

Activity Driven Weakly Supervised Object Detection

Weakly supervised object detection aims at reducing the amount of superv...

Exploring Bottom-up and Top-down Cues with Attentive Learning for Webly Supervised Object Detection

Fully supervised object detection has achieved great success in recent y...

Domain Generalisation for Object Detection

Domain generalisation aims to promote the learning of domain-invariant f...

Weakly Supervised Knowledge Transfer with Probabilistic Logical Reasoning for Object Detection

Training object detection models usually requires instance-level annotat...

DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detection

This paper presents a DETR-based method for cross-domain weakly supervis...

Please sign up or login with your details

Forgot password? Click here to reset