Privacy Leakage of SIFT Features via Deep Generative Model based Image Reconstruction

by   Haiwei Wu, et al.

Many practical applications, e.g., content based image retrieval and object recognition, heavily rely on the local features extracted from the query image. As these local features are usually exposed to untrustworthy parties, the privacy leakage problem of image local features has received increasing attention in recent years. In this work, we thoroughly evaluate the privacy leakage of Scale Invariant Feature Transform (SIFT), which is one of the most widely-used image local features. We first consider the case that the adversary can fully access the SIFT features, i.e., both the SIFT descriptors and the coordinates are available. We propose a novel end-to-end, coarse-to-fine deep generative model for reconstructing the latent image from its SIFT features. The designed deep generative model consists of two networks, where the first one attempts to learn the structural information of the latent image by transforming from SIFT features to Local Binary Pattern (LBP) features, while the second one aims to reconstruct the pixel values guided by the learned LBP. Compared with the state-of-the-art algorithms, the proposed deep generative model produces much improved reconstructed results over three public datasets. Furthermore, we address more challenging cases that only partial SIFT features (either SIFT descriptors or coordinates) are accessible to the adversary. It is shown that, if the adversary can only have access to the SIFT descriptors while not their coordinates, then the modest success of reconstructing the latent image can be achieved for highly-structured images (e.g., faces) and would fail in general settings. In addition, the latent image can be reconstructed with reasonably good quality solely from the SIFT coordinates. Our results would suggest that the privacy leakage problem can be largely avoided if the SIFT coordinates can be well protected.



There are no comments yet.


page 1

page 4

page 6

page 8

page 9

page 10


Content-Based Image Retrieval Based on Late Fusion of Binary and Local Descriptors

One of the challenges in Content-Based Image Retrieval (CBIR) is to redu...

Training privacy-preserving video analytics pipelines by suppressing features that reveal information about private attributes

Deep neural networks are increasingly deployed for scene analytics, incl...

Multichannel Distributed Local Pattern for Content Based Indexing and Retrieval

A novel color feature descriptor, Multichannel Distributed Local Pattern...

Generative Model for Zero-Shot Sketch-Based Image Retrieval

We present a probabilistic model for Sketch-Based Image Retrieval (SBIR)...

Rewriting a Deep Generative Model

A deep generative model such as a GAN learns to model a rich set of sema...

A Deep Generative Model for Disentangled Representations of Sequential Data

We present a VAE architecture for encoding and generating high dimension...

Analysis and Mitigations of Reverse Engineering Attacks on Local Feature Descriptors

As autonomous driving and augmented reality evolve, a practical concern ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.