aiTPR: Attribute Interaction-Tensor Product Representation for Image Caption

01/27/2020
by   Chiranjib Sur, et al.
15

Region visual features enhance the generative capability of the machines based on features, however they lack proper interaction attentional perceptions and thus ends up with biased or uncorrelated sentences or pieces of misinformation. In this work, we propose Attribute Interaction-Tensor Product Representation (aiTPR) which is a convenient way of gathering more information through orthogonal combination and learning the interactions as physical entities (tensors) and improving the captions. Compared to previous works, where features are added up to undefined feature spaces, TPR helps in maintaining sanity in combinations and orthogonality helps in defining familiar spaces. We have introduced a new concept layer that defines the objects and also their interactions that can play a crucial role in determination of different descriptions. The interaction portions have contributed heavily for better caption quality and has out-performed different previous works on this domain and MSCOCO dataset. We introduced, for the first time, the notion of combining regional image features and abstracted interaction likelihood embedding for image captioning.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 5

page 6

page 9

page 10

11/22/2019

TPsgtR: Neural-Symbolic Tensor Product Scene-Graph-Triplet Representation for Image Captioning

Image captioning can be improved if the structure of the graphical repre...
02/15/2020

MRRC: Multiple Role Representation Crossover Interpretation for Image Captioning With R-CNN Feature Distribution Composition (FDC)

While image captioning through machines requires structured learning and...
12/17/2018

Feature Fusion Effects of Tensor Product Representation on (De)Compositional Network for Caption Generation for Images

Progress in image captioning is gradually getting complex as researchers...
07/25/2018

Distinctive-attribute Extraction for Image Captioning

Image captioning, an open research issue, has been evolved with the prog...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.