PieAPP: Perceptual Image-Error Assessment through Pairwise Preference

by   Ekta Prashnani, et al.

The ability to estimate the perceptual error between images is an important problem in computer vision with many applications. Although it has been studied extensively, however, no method currently exists that can robustly predict visual differences like humans. Some previous approaches used hand-coded models, but they fail to model the complexity of the human visual system. Others used machine learning to train models on human-labeled datasets, but creating large, high-quality datasets is difficult because people are unable to assign consistent error labels to distorted images. In this paper, we present a new learning-based method that is the first to predict perceptual image error like human observers. Since it is much easier for people to compare two given images and identify the one more similar to a reference than to assign quality scores to each, we propose a new, large-scale dataset labeled with the probability that humans will prefer one image over another. We then train a deep-learning model using a novel, pairwise-learning framework to predict the preference of one distorted image over the other. Our key observation is that our trained network can then be used separately with only one distorted image and a reference to predict its perceptual error, without ever being trained on explicit human perceptual-error labels. The perceptual error estimated by our new metric, PieAPP, is well-correlated with human opinion. Furthermore, it significantly outperforms existing algorithms, beating the state-of-the-art by almost 3x on our test set in terms of binary error rate, while also generalizing to new kinds of distortions, unlike previous learning-based methods.


page 2

page 3


Perceptual Attacks of No-Reference Image Quality Models with Human-in-the-Loop

No-reference image quality assessment (NR-IQA) aims to quantify how huma...

Deep Perceptual Image Quality Assessment for Compression

Lossy Image compression is necessary for efficient storage and transfer ...

Enhancing image quality prediction with self-supervised visual masking

Full-reference image quality metrics (FR-IQMs) aim to measure the visual...

Learning from Synthetic Data for Opinion-free Blind Image Quality Assessment in the Wild

Nowadays, most existing blind image quality assessment (BIQA) models 1) ...

Learned Perceptual Image Enhancement

Learning a typical image enhancement pipeline involves minimization of a...

Learning to Observe: Approximating Human Perceptual Thresholds for Detection of Suprathreshold Image Transformations

Many tasks in computer vision are often calibrated and evaluated relativ...

HYPE: Human eYe Perceptual Evaluation of Generative Models

Generative models often use human evaluations to determine and justify p...

Please sign up or login with your details

Forgot password? Click here to reset