Massive Online Crowdsourced Study of Subjective and Objective Picture Quality

by   Deepti Ghadiyaram, et al.

Most publicly available image quality databases have been created under highly controlled conditions by introducing graded simulated distortions onto high-quality photographs. However, images captured using typical real-world mobile camera devices are usually afflicted by complex mixtures of multiple distortions, which are not necessarily well-modeled by the synthetic distortions found in existing databases. The originators of existing legacy databases usually conducted human psychometric studies to obtain statistically meaningful sets of human opinion scores on images in a stringently controlled visual environment, resulting in small data collections relative to other kinds of image analysis databases. Towards overcoming these limitations, we designed and created a new database that we call the LIVE In the Wild Image Quality Challenge Database, which contains widely diverse authentic image distortions on a large number of images captured using a representative variety of modern mobile devices. We also designed and implemented a new online crowdsourcing system, which we have used to conduct a very large-scale, multi-month image quality assessment subjective study. Our database consists of over 350000 opinion scores on 1162 images evaluated by over 7000 unique human observers. Despite the lack of control over the experimental environments of the numerous study participants, we demonstrate excellent internal consistency of the subjective dataset. We also evaluate several top-performing blind Image Quality Assessment algorithms on it and present insights on how mixtures of distortions challenge both end users as well as automatic perceptual quality prediction models.


page 3

page 4

page 6

page 8

page 9

page 11


Subjective Quality Assessment of Ground-based Camera Images

Image quality assessment is critical to control and maintain the perceiv...

Perceptual Quality Prediction on Authentically Distorted Images Using a Bag of Features Approach

Current top-performing blind perceptual image quality prediction models ...

KonIQ-10k: Towards an ecologically valid and large-scale IQA database

The main challenge in applying state-of-the-art deep learning methods to...

Human vs Objective Evaluation of Colourisation Performance

Automatic colourisation of grey-scale images is the process of creating ...

Deep Neural Networks for Blind Image Quality Assessment: Addressing the Data Challenge

The enormous space and diversity of natural images is usually represente...

Helping Visually Impaired People Take Better Quality Pictures

Perception-based image analysis technologies can be used to help visuall...

Learning from Synthetic Data for Opinion-free Blind Image Quality Assessment in the Wild

Nowadays, most existing blind image quality assessment (BIQA) models 1) ...

Please sign up or login with your details

Forgot password? Click here to reset