Learning Robust Representations Of Generative Models Using Set-Based Artificial Fingerprints

06/04/2022
by   Hae Jin Song, et al.
0

With recent progress in deep generative models, the problem of identifying synthetic data and comparing their underlying generative processes has become an imperative task for various reasons, including fighting visual misinformation and source attribution. Existing methods often approximate the distance between the models via their sample distributions. In this paper, we approach the problem of fingerprinting generative models by learning representations that encode the residual artifacts left by the generative models as unique signals that identify the source models. We consider these unique traces (a.k.a. "artificial fingerprints") as representations of generative models, and demonstrate their usefulness in both the discriminative task of source attribution and the unsupervised task of defining a similarity between the underlying models. We first extend the existing studies on fingerprints of GANs to four representative classes of generative models (VAEs, Flows, GANs and score-based models), and demonstrate their existence and attributability. We then improve the stability and attributability of the fingerprints by proposing a new learning method based on set-encoding and contrastive training. Our set-encoder, unlike existing methods that operate on individual images, learns fingerprints from a set of images. We demonstrate improvements in the stability and attributability through comparisons to state-of-the-art fingerprint methods and ablation studies. Further, our method employs contrastive training to learn an implicit similarity between models. We discover latent families of generative models using this metric in a standard hierarchical clustering algorithm.

READ FULL TEXT

page 6

page 8

page 14

page 16

research
10/05/2020

Winning Lottery Tickets in Deep Generative Models

The lottery ticket hypothesis suggests that sparse, sub-networks of a gi...
research
08/02/2017

Combining Generative and Discriminative Approaches to Unsupervised Dependency Parsing via Dual Decomposition

Unsupervised dependency parsing aims to learn a dependency parser from u...
research
08/14/2018

Skill Rating for Generative Models

We explore a new way to evaluate generative models using insights from e...
research
01/21/2022

Evaluating Generalization in Classical and Quantum Generative Models

Defining and accurately measuring generalization in generative models re...
research
12/16/2020

Responsible Disclosure of Generative Models Using Scalable Fingerprinting

Over the past six years, deep generative models have achieved a qualitat...
research
02/17/2022

Attributable-Watermarking of Speech Generative Models

Generative models are now capable of synthesizing images, speeches, and ...
research
04/17/2023

Attributing Image Generative Models using Latent Fingerprints

Generative models have enabled the creation of contents that are indisti...

Please sign up or login with your details

Forgot password? Click here to reset