HiCoRe: Visual Hierarchical Context-Reasoning

09/02/2019
by   Pedro H. Bugatti, et al.
0

Reasoning about images/objects and their hierarchical interactions is a key concept for the next generation of computer vision approaches. Here we present a new framework to deal with it through a visual hierarchical context-based reasoning. Current reasoning methods use the fine-grained labels from images' objects and their interactions to predict labels to new objects. Our framework modifies this current information flow. It goes beyond and is independent of the fine-grained labels from the objects to define the image context. It takes into account the hierarchical interactions between different abstraction levels (i.e. taxonomy) of information in the images and their bounding-boxes. Besides these connections, it considers their intrinsic characteristics. To do so, we build and apply graphs to graph convolution networks with convolutional neural networks. We show a strong effectiveness over widely used convolutional neural networks, reaching a gain 3 times greater on well-known image datasets. We evaluate the capability and the behavior of our framework under different scenarios, considering distinct (superclass, subclass and hierarchical) granularity levels. We also explore attention mechanisms through graph attention networks and pre-processing methods considering dimensionality expansion and/or reduction of the features' representations. Further analyses are performed comparing supervised and semi-supervised approaches.

READ FULL TEXT

page 2

page 6

research
12/04/2019

Better Understanding Hierarchical Visual Relationship for Image Caption

The Convolutional Neural Network (CNN) has been the dominant image featu...
research
01/10/2022

Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification

Hierarchical multi-granularity classification (HMC) assigns hierarchical...
research
03/04/2021

Learning Granularity-Aware Convolutional Neural Network for Fine-Grained Visual Classification

Locating discriminative parts plays a key role in fine-grained visual cl...
research
06/22/2017

Fine-Grained Categorization via CNN-Based Automatic Extraction and Integration of Object-Level and Part-Level Features

Fine-grained categorization can benefit from part-based features which r...
research
05/19/2018

Learning Hierarchical Visual Representations in Deep Neural Networks Using Hierarchical Linguistic Labels

Modern convolutional neural networks (CNNs) are able to achieve human-le...
research
07/10/2020

Hypothetical Reasoning via Provenance Abstraction

Data analytics often involves hypothetical reasoning: repeatedly modifyi...

Please sign up or login with your details

Forgot password? Click here to reset