Look-into-Object: Self-supervised Structure Modeling for Object Recognition

by   Mohan Zhou, et al.

Most object recognition approaches predominantly focus on learning discriminative visual patterns while overlooking the holistic object structure. Though important, structure modeling usually requires significant manual annotations and therefore is labor-intensive. In this paper, we propose to "look into object" (explicitly yet intrinsically model the object structure) through incorporating self-supervisions into the traditional framework. We show the recognition backbone can be substantially enhanced for more robust representation learning, without any cost of extra annotation and inference speed. Specifically, we first propose an object-extent learning module for localizing the object according to the visual patterns shared among the instances in the same category. We then design a spatial context learning module for modeling the internal structures of the object, through predicting the relative positions within the extent. These two modules can be easily plugged into any backbone networks during training and detached at inference time. Extensive experiments show that our look-into-object approach (LIO) achieves large performance gain on a number of benchmarks, including generic object recognition (ImageNet) and fine-grained object recognition tasks (CUB, Cars, Aircraft). We also show that this learning paradigm is highly generalizable to other tasks such as object detection and segmentation (MS COCO). Project page: https://github.com/JDAI-CV/LIO.


page 1

page 2

page 4

page 7


SIM-Trans: Structure Information Modeling Transformer for Fine-grained Visual Categorization

Fine-grained visual categorization (FGVC) aims at recognizing objects fr...

SL3D: Self-supervised-Self-labeled 3D Recognition

There are a lot of promising results in 3D recognition, including classi...

RTMDet: An Empirical Study of Designing Real-Time Object Detectors

In this paper, we aim to design an efficient real-time object detector t...

L3DOR: Lifelong 3D Object Recognition

3D object recognition has been widely-applied. However, most state-of-th...

Learning Canonical 3D Object Representation for Fine-Grained Recognition

We propose a novel framework for fine-grained object recognition that le...

Learning Visual Context by Comparison

Finding diseases from an X-ray image is an important yet highly challeng...

Object-QA: Towards High Reliable Object Quality Assessment

In object recognition applications, object images usually appear with di...

Code Repositories


Destruction and Construction Learning for Fine-grained Image Recognition

view repo


This repo is base on https://github.com/JDAI-CV/DCL for training myself in model classification

view repo

Please sign up or login with your details

Forgot password? Click here to reset