Log In Sign Up

Cross-Task Transfer for Geotagged Audiovisual Aerial Scene Recognition

by   Di Hu, et al.

Aerial scene recognition is a fundamental task in remote sensing and has recently received increased interest. While the visual information from overhead images with powerful models and efficient algorithms yields considerable performance on scene recognition, it still suffers from the variation of ground objects, lighting conditions etc. Inspired by the multi-channel perception theory in cognition science, in this paper, for improving the performance on the aerial scene recognition, we explore a novel audiovisual aerial scene recognition task using both images and sounds as input. Based on an observation that some specific sound events are more likely to be heard at a given geographic location, we propose to exploit the knowledge from the sound events to improve the performance on the aerial scene recognition. For this purpose, we have constructed a new dataset named AuDio Visual Aerial sceNe reCognition datasEt (ADVANCE). With the help of this dataset, we evaluate three proposed approaches for transferring the sound event knowledge to the aerial scene recognition task in a multimodal learning framework, and show the benefit of exploiting the audio information for the aerial scene recognition. The source code is publicly available for reproducibility purposes.


Cross-Task Transfer for Multimodal Aerial Scene Recognition

Aerial scene recognition is a fundamental task in remote sensing and has...

AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification

Aerial scene classification, which aims to automatically label an aerial...

Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks

Aerial scene recognition is a fundamental visual task and has attracted ...

MultiScene: A Large-scale Dataset and Benchmark for Multi-scene Recognition in Single Aerial Images

Aerial scene recognition is a fundamental research problem in interpreti...

Transferring Object-Scene Convolutional Neural Networks for Event Recognition in Still Images

Event recognition in still images is an intriguing problem and has poten...

Prediction method of Soundscape Impressions using Environmental Sounds and Aerial Photographs

We investigate an method for quantifying city characteristics based on i...

ERA: A Dataset and Deep Learning Benchmark for Event Recognition in Aerial Videos

Along with the increasing use of unmanned aerial vehicles (UAVs), large ...

Code Repositories


Code for (ECCV 2020)

view repo