Bringing back simplicity and lightliness into neural image captioning

10/15/2018
by   Jean-Benoit Delbrouck, et al.
0

Neural Image Captioning (NIC) or neural caption generation has attracted a lot of attention over the last few years. Describing an image with a natural language has been an emerging challenge in both fields of computer vision and language processing. Therefore a lot of research has focused on driving this task forward with new creative ideas. So far, the goal has been to maximize scores on automated metric and to do so, one has to come up with a plurality of new modules and techniques. Once these add up, the models become complex and resource-hungry. In this paper, we take a small step backwards in order to study an architecture with interesting trade-off between performance and computational complexity. To do so, we tackle every component of a neural captioning model and propose one or more solution that lightens the model overall. Our ideas are inspired by two related tasks: Multimodal and Monomodal Neural Machine Translation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/09/2018

Image Captioning as Neural Machine Translation Task in SOCKEYE

Image captioning is an interdisciplinary research problem that stands be...
research
01/17/2018

Image Captioning using Deep Neural Architectures

Automatically creating the description of an image using any natural lan...
research
07/02/2019

Neural Image Captioning

In recent years, the biggest advances in major Computer Vision tasks, su...
research
03/12/2016

Image Captioning with Semantic Attention

Automatically generating a natural language description of an image has ...
research
08/05/2021

Neural Twins Talk Alternative Calculations

Inspired by how the human brain employs a higher number of neural pathwa...
research
03/14/2018

Unpaired Image Captioning by Language Pivoting

Image captioning is a multimodal task involving computer vision and natu...
research
10/27/2016

Can Active Memory Replace Attention?

Several mechanisms to focus attention of a neural network on selected pa...

Please sign up or login with your details

Forgot password? Click here to reset