DeepAI AI Chat
Log In Sign Up

VGGSound: A Large-scale Audio-Visual Dataset

04/29/2020
by   Honglie Chen, et al.
9

Our goal is to collect a large-scale audio-visual dataset with low label noise from videos in the wild using computer vision techniques. The resulting dataset can be used for training and evaluating audio recognition models. We make three contributions. First, we propose a scalable pipeline based on computer vision techniques to create an audio dataset from open-source media. Our pipeline involves obtaining videos from YouTube; using image classification algorithms to localize audio-visual correspondence; and filtering out ambient noise using audio verification. Second, we use this pipeline to curate the VGGSound dataset consisting of more than 210k videos for 310 audio classes. Third, we investigate various Convolutional Neural Network (CNN) architectures and aggregation approaches to establish audio recognition baselines for our new dataset. Compared to existing audio datasets, VGGSound ensures audio-visual correspondence and is collected under unconstrained conditions. Code and the dataset are available at http://www.robots.ox.ac.uk/ vgg/data/vggsound/

READ FULL TEXT
06/26/2017

VoxCeleb: a large-scale speaker identification dataset

Most existing datasets for speaker identification contain samples obtain...
07/02/2020

Spot the conversation: speaker diarisation in the wild

The goal of this paper is speaker diarisation of videos collected 'in th...
11/10/2022

AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies

Existing correspondence datasets for two-dimensional (2D) cartoon suffer...
09/14/2020

Themes Informed Audio-visual Correspondence Learning

The applications of short-term user-generated video (UGV), such as Snapc...
02/01/2023

Epic-Sounds: A Large-scale Dataset of Actions That Sound

We introduce EPIC-SOUNDS, a large-scale dataset of audio annotations cap...
12/16/2021

An Audio-Visual Dataset and Deep Learning Frameworks for Crowded Scene Classification

This paper presents a task of audio-visual scene classification (SC) whe...
12/25/2019

Improving Visual Recognition using Ambient Sound for Supervision

Our brains combine vision and hearing to create a more elaborate interpr...