Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition

08/10/2021
by   Ziwei Xu, et al.
5

This paper proposes a novel model for recognizing images with composite attribute-object concepts, notably for composite concepts that are unseen during model training. We aim to explore the three key properties required by the task — relation-aware, consistent, and decoupled — to learn rich and robust features for primitive concepts that compose attribute-object pairs. To this end, we propose the Blocked Message Passing Network (BMP-Net). The model consists of two modules. The concept module generates semantically meaningful features for primitive concepts, whereas the visual module extracts visual features for attributes and objects from input images. A message passing mechanism is used in the concept module to capture the relations between primitive concepts. Furthermore, to prevent the model from being biased towards seen composite concepts and reduce the entanglement between attributes and objects, we propose a blocking mechanism that equalizes the information available to the model for both seen and unseen concepts. Extensive experiments and ablation studies on two benchmarks show the efficacy of the proposed model.

READ FULL TEXT

page 1

page 4

page 9

page 10

research
12/20/2021

Translational Concept Embedding for Generalized Compositional Zero-shot Learning

Generalized compositional zero-shot learning means to learn composed con...
research
03/31/2022

Do Vision-Language Pretrained Models Learn Primitive Concepts?

Vision-language pretrained models have achieved impressive performance o...
research
08/08/2023

Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning

Compositional zero-shot learning (CZSL) aims to recognize unseen composi...
research
04/22/2021

Attribute-Modulated Generative Meta Learning for Zero-Shot Classification

Zero-shot learning (ZSL) aims to transfer knowledge from seen classes to...
research
07/12/2021

Zero-Shot Compositional Concept Learning

In this paper, we study the problem of recognizing compositional attribu...
research
03/27/2022

Discovering Human-Object Interaction Concepts via Self-Compositional Learning

A comprehensive understanding of human-object interaction (HOI) requires...
research
05/15/2019

Task-Driven Modular Networks for Zero-Shot Compositional Learning

One of the hallmarks of human intelligence is the ability to compose lea...

Please sign up or login with your details

Forgot password? Click here to reset