Context-LGM: Leveraging Object-Context Relation for Context-Aware Object Recognition

10/08/2021
by   Mingzhou Liu, et al.
2

Context, as referred to situational factors related to the object of interest, can help infer the object's states or properties in visual recognition. As such contextual features are too diverse (across instances) to be annotated, existing attempts simply exploit image labels as supervision to learn them, resulting in various contextual tricks, such as features pyramid, context attention, etc. However, without carefully modeling the context's properties, especially its relation to the object, their estimated context can suffer from large inaccuracy. To amend this problem, we propose a novel Contextual Latent Generative Model (Context-LGM), which considers the object-context relation and models it in a hierarchical manner. Specifically, we firstly introduce a latent generative model with a pair of correlated latent variables to respectively model the object and context, and embed their correlation via the generative process. Then, to infer contextual features, we reformulate the objective function of Variational Auto-Encoder (VAE), where contextual features are learned as a posterior distribution conditioned on the object. Finally, to implement this contextual posterior, we introduce a Transformer that takes the object's information as a reference and locates correlated contextual factors. The effectiveness of our method is verified by state-of-the-art performance on two context-aware object recognition tasks, i.e. lung cancer prediction and emotion recognition.

READ FULL TEXT

page 1

page 2

page 9

page 10

page 11

page 12

research
11/17/2019

Putting visual object recognition in context

Context plays an important role in visual recognition. Recent studies ha...
research
04/06/2021

When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes

Context is of fundamental importance to both human and machine vision – ...
research
08/16/2019

Context-Aware Emotion Recognition Networks

Traditional techniques for emotion recognition have focused on the facia...
research
03/21/2023

Context De-confounded Emotion Recognition

Context-Aware Emotion Recognition (CAER) is a crucial and challenging ta...
research
02/01/2019

Lift-the-Flap: Context Reasoning Using Object-Centered Graphs

Children benefit from lift-the-flap books by taking on an active role in...
research
04/11/2020

Learning to Manipulate Individual Objects in an Image

We describe a method to train a generative model with latent factors tha...
research
02/04/2020

Context-Aware Recommendations for Televisions Using Deep Embeddings with Relaxed N-Pairs Loss Objective

This paper studies context-aware recommendations in the television domai...

Please sign up or login with your details

Forgot password? Click here to reset