E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles

06/10/2019
by   Markus Kettunen, et al.
5

It has been recently shown that the hidden variables of convolutional neural networks make for an efficient perceptual similarity metric that accurately predicts human judgment on relative image similarity assessment. First, we show that such learned perceptual similarity metrics (LPIPS) are susceptible to adversarial attacks that dramatically contradict human visual similarity judgment. While this is not surprising in light of neural networks' well-known weakness to adversarial perturbations, we proceed to show that self-ensembling with an infinite family of random transformations of the input --- a technique known not to render classification networks robust --- is enough to turn the metric robust against attack, while retaining predictive power on human judgments. Finally, we study the geometry imposed by our our novel self-ensembled metric (E-LPIPS) on the space of natural images. We find evidence of "perceptual convexity" by showing that convex combinations of similar-looking images retain appearance, and that discrete geodesics yield meaningful frame interpolation and texture morphing, all without explicit correspondences.

READ FULL TEXT

page 4

page 5

page 6

page 7

research
05/15/2023

Attacking Perceptual Similarity Metrics

Perceptual similarity metrics have progressively become more correlated ...
research
07/27/2023

R-LPIPS: An Adversarially Robust Perceptual Similarity Metric

Similarity metrics have played a significant role in computer vision to ...
research
07/27/2022

Shift-tolerant Perceptual Similarity Metric

Existing perceptual similarity metrics assume an image and its reference...
research
02/08/2016

Generating Images with Perceptual Similarity Metrics based on Deep Networks

Image-generating machine learning models are typically trained with loss...
research
08/16/2020

Visual stream connectivity predicts assessments of image quality

Some biological mechanisms of early vision are comparatively well unders...
research
05/08/2019

PerceptNet: Learning Perceptual Similarity of Haptic Textures in Presence of Unorderable Triplets

In order to design haptic icons or build a haptic vocabulary, we require...
research
04/29/2020

Image Morphing with Perceptual Constraints and STN Alignment

In image morphing, a sequence of plausible frames are synthesized and co...

Please sign up or login with your details

Forgot password? Click here to reset