RSVQA: Visual Question Answering for Remote Sensing Data

03/16/2020
by   Sylvain Lobry, et al.
9

This paper introduces the task of visual question answering for remote sensing data (RSVQA). Remote sensing images contain a wealth of information which can be useful for a wide range of tasks including land cover classification, object counting or detection. However, most of the available methodologies are task-specific, thus inhibiting generic and easy access to the information contained in remote sensing data. As a consequence, accurate remote sensing product generation still requires expert knowledge. With RSVQA, we propose a system to extract information from remote sensing data that is accessible to every user: we use questions formulated in natural language and use them to interact with the images. With the system, images can be queried to obtain high level information specific to the image content or relational dependencies between objects visible in the images. Using an automatic method introduced in this article, we built two datasets (using low and high resolution data) of image/question/answer triplets. The information required to build the questions and answers is queried from OpenStreetMap (OSM). The datasets can be used to train (when using supervised methods) and evaluate models to solve the RSVQA task. We report the results obtained by applying a model based on Convolutional Neural Networks (CNNs) for the visual part and on a Recurrent Neural Network (RNN) for the natural language part to this task. The model is trained on the two datasets, yielding promising results in both cases.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 9

page 10

page 11

research
09/24/2021

How to find a good image-text embedding for remote sensing visual question answering?

Visual question answering (VQA) has recently been introduced to remote s...
research
04/07/2023

Multilingual Augmentation for Robust Visual Question Answering in Remote Sensing Images

Aiming at answering questions based on the content of remotely sensed im...
research
06/25/2023

Visual Question Answering in Remote Sensing with Cross-Attention and Multimodal Information Bottleneck

In this research, we deal with the problem of visual question answering ...
research
08/24/2022

ForestEyes Project: Conception, Enhancements, and Challenges

Rainforests play an important role in the global ecosystem. However, sig...
research
05/06/2022

From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data

Visual question answering (VQA) for remote sensing scene has great poten...
research
07/29/2019

X-LineNet: Detecting Aircraft in Remote Sensing Images by a pair of Intersecting Line Segments

In the field of aircraft detection, tremendous progress has been gained ...

Please sign up or login with your details

Forgot password? Click here to reset