Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions

05/14/2020
by   Di Hu, et al.
0

Visual crowd counting has been recently studied as a way to enable people counting in crowd scenes from images. Albeit successful, vision-based crowd counting approaches could fail to capture informative features in extreme conditions, e.g., imaging at night and occlusion. In this work, we introduce a novel task of audiovisual crowd counting, in which visual and auditory information are integrated for counting purposes. We collect a large-scale benchmark, named auDiovISual Crowd cOunting (DISCO) dataset, consisting of 1,935 images and the corresponding audio clips, and 170,270 annotated instances. In order to fuse the two modalities, we make use of a linear feature-wise fusion module that carries out an affine transformation on visual and auditory features. Finally, we conduct extensive experiments using the proposed dataset and approach. Experimental results show that introducing auditory information can benefit crowd counting under different illumination, noise, and occlusion conditions. The dataset and code will be released. Code and data have been made available

READ FULL TEXT

page 2

page 4

page 6

page 10

research
01/10/2020

NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting

In the last decade, crowd counting attracts much attention of researcher...
research
03/24/2021

Repetitive Activity Counting by Sight and Sound

This paper strives for repetitive activity counting in videos. Different...
research
04/19/2018

A-cCCNN: adaptive ccnn for density estimation and crowd counting

Crowd counting, for estimating the number of people in a crowd using vis...
research
12/08/2020

Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting

Crowd counting is a fundamental yet challenging problem, which desires r...
research
09/04/2021

Audio-Visual Transformer Based Crowd Counting

Crowd estimation is a very challenging problem. The most recent study tr...
research
04/17/2020

Sketch and test: visual crowd research using p5.js

Designing visual crowd experiments requires both control and versatility...
research
02/08/2022

A Unified Multi-Task Learning Framework of Real-Time Drone Supervision for Crowd Counting

In this paper, a novel Unified Multi-Task Learning Framework of Real-Tim...

Please sign up or login with your details

Forgot password? Click here to reset