Our work addresses the problem of privacy-preserving data release, where the goal is to release useful data while also limiting the exposure of associated sensitive information. Approaches that involve data modification must consider the tradeoff between concealing sensitive information and minimizing distortion to preserve data utility. However, practical optimization of this tradeoff can be challenging when we wish to quantify privacy via statistical measures (such as mutual information) and the actual statistical distributions of data are unknown. In this paper, we propose a data-driven framework involving adversarially trained neural networks to design privacy-preserving data release mechanisms that approach the theoretically optimal privacy-utility tradeoffs.
Privacy-preserving data release is a broad and widely explored field, where the study of principled methods have been well motivated by highly publicized leaks stemming from the inadequacy of simple anonymization techniques, such as reported in [1, 2]. A wide variety of methods to statistically quantify and address privacy have been proposed, such as -anonymity , -diversity , -closeness , and differential privacy . In our work, we focus on an information-theoretic approach where privacy is quantified by the mutual information between the data release and the sensitive information [7, 8, 9, 10, 11]. Unlike the methods mentioned earlier, measuring privacy via mutual information implicitly requires consideration of the underlying statistical distribution of the data. While lack of model knowledge may be a challenging issue to address in practice, entirely ignoring the data distribution can weaken the scope of privacy guarantees. For example, an adversary armed with only mild knowledge about the correlation of the data111Note that even when data samples are inherently independent, the prior knowledge of an adversary could become correlated when conditioned on particular side information. can undermine the practical privacy protection of differential privacy, as noted in examples given by [12, 9, 13, 14].
, where the sensitive and useful data are respectively modeled as random variablesand . We also adopt the extension considered in , where only a (potentially partial and/or noisy) observation of the data is available. In this framework, the design of the privacy-preserving mechanism to release is formulated as the optimization of the tradeoff between minimizing privacy-leakage quantified by the mutual information and minimizing an expected distortion . This non-asymptotic framework has strong connections to generalized rate-distortion problems (see discussion in [8, 9, 14]), as well as related asymptotic privacy frameworks where communication efficiency is also considered in a rate-distortion-privacy tradeoff [7, 10].
In principle, when the data model distribution is known, the design of the optimal privacy-preserving mechanism can be tackled as a convex optimization problem [8, 9]. However, in practice, model knowledge is often missing or inaccurate for realistic data sets, and the optimization becomes intractable for high-dimensional and continuous data. Addressing these challenges, we propose a data-driven approach that optimizes the privacy-preserving mechanism toward the theoretically optimal privacy-utility tradeoffs, by learning from a set of training data rather than requiring model knowledge. We call this approach Privacy-Preserving Adversarial Networks
(PPAN) since the mechanism is realized as a randomized neural network, which is trained along with an adversarial network that attempts to recover the sensitive information from the released data. The key to attaining information-theoretic privacy is that the adversarial network specifically estimates the posterior distribution (rather than only the value) of the sensitive variable given the released data to enable a variational approximation of mutual information. While the adversary is trained to minimize the log-loss with respect to this posterior estimate, the mechanism network is trained toward the dual objectives of minimizing distortion and concealing sensitive information (by maximizing the adversarial loss).
1.1 Related Work
The general concept of adversarial training of neural networks was introduced by , which proposed Generative Adversarial Networks (GAN) for learning generative models that can synthesize new data samples. Since, their introduction, GANs have inspired an enormous number of adversarially trained neural network architectures for a wide variety of purposes .
have also proposed adversarial training frameworks for optimizing privacy-preserving mechanisms, where the adversarial network is realized as a classifier that attempts to recover a discrete sensitive variable. In
, the mechanism is realized as an autoencoder222An autoencoder architecture itself is comprised of two networks, an encoder and a decoder. The input is first processed by the encoder to produce a latent representation (or code), which is then processed by the decoder to produce the final output., and the adversary attempts to predict a binary sensitive variable from the latent representation. In the framework of , a deterministic mechanism is trained with the adversarial network realized as a classifier attempting to predict the sensitive variable from the output of the mechanism. Both of these frameworks additionally propose using an optional predictor network that attempts to predict a useful variable from the output of the mechanism network. Thus, while the adversarial network is trained to recover the sensitive variable, the mechanism and predictor (if present) networks are trained toward multiple objectives: maximizing the loss of the adversary as well as minimizing the reconstruction loss of the mechanism network and/or the prediction loss of the predictor network. However, a significant limitation of both of these approaches is that they consider only deterministic333While  does also consider a “noisy” version of their mechanism, the randomization is limited to only independent, additive noise before or after deterministic filtering. mechanisms, which generally do not achieve the optimal privacy-utility tradeoffs, although neither attempts to address information-theoretic privacy.
The recent, independent work of  proposes a similar adversarial training framework, which also realizes the necessity of and proposes randomized mechanism networks, in order to address the information-theoretically optimal privacy-utility tradeoffs. They also rediscover the earlier realization of  that mutual information privacy arises from an adversary (which outputs a distribution) that is optimized with respect to log-loss. However, their framework does not make the connections to a general variational approximation of mutual information applicable to arbitrary (i.e., discrete, continuous, and/or multivariate) sensitive variable alphabets, and hence their data-driven formulation and empirical evaluation is limited to only binary sensitive variables.
1.2 Contributions and Paper Outline
Our main contributions are summarized as follows:
Our framework, presented in Section 2, provides a data-driven approach for optimizing privacy-preserving data release mechanisms that approaches the information-theoretically optimal privacy-utility tradeoffs. The key to our approach is employing adversarial training to perform a variational approximation of mutual information privacy.
We consider randomized data release mechanisms where the input to the mechanism can be a general observation of the data, e.g., a full or potentially noisy/partial view of the sensitive and useful variables.
In our framework, all of the variables involved can be discrete, continuous, and/or high-dimensional vectors. We describe specific network architectures and sampling methods appropriate for various scenarios in Section2.3. When all of the variables have finite alphabets, we note that the network architectures can be efficiently minimalized to essentially just the matrices describing the conditional distributions, and that replacing sampling with a directly computed expectation improves training performance.
We evaluate our PPAN approach in Section 3 with experiments on discrete and continuous (multivariate Gaussian) synthetic data, and the MNIST handwritten digit dataset. For the synthetic data experiments, we demonstrate that PPAN closely approaches the theoretically optimal privacy-utility tradeoffs.
For multivariate Gaussian data, with partial and full observations, we analytically derive the theoretically-optimal privacy-utility tradeoffs in Section 4, providing the theoretical baseline for our experiments with continuous synthetic data.
2 Problem Formulation and PPAN Methods
2.1 Privacy-Utility Tradeoff Optimization
are modeled as random variables that are jointly distributed according to a data modelover the space . The goal is to design a system that processes the observed data to produce a release that minimizes the privacy-leakage of the sensitive attributes , while also maximizing the utility gained from revealing information about . This system is specified by the release mechanism , with , and thus
forms a Markov chain. Privacy-leakage is quantified by the mutual informationbetween the sensitive attributes and the release . Utility is inversely quantified by the expected distortion444We mainly focus on expected distortion in this work, although the formulation in  actually allows for a more general class of distortion measures. We outline an extension of our approach for distortion measured by conditional entropy in Section 5.1. between the useful attributes and the release , where the distortion function is given by the application. The design of the release mechanism is formulated as the following privacy-utility tradeoff optimization problem,
where the parameter indicates the distortion (or disutility) budget allowed for the sake of preserving privacy.
As noted in , given a fixed data model and distortion function , the problem in (1) is a convex optimization problem, since the mutual information objective is a convex functional of , which is in turn a linear functional of , and the expected distortion is a linear functional of and hence also of . While the treatment in  considers discrete variables over finite alphabets, the formulation of (1) need not be limited those assumptions. Thus, in this work, we seek to also address this problem with high-dimensional, continuous variables.
2.2 Adversarial Training for an Unknown Data Model
Our aim is to solve the privacy-utility tradeoff optimization problem when the data model is unknown but instead a set of training samples is available: . A key to our approach is approximating via a variational lower bound given by  and also used in . This bound is based on the following identity, for any conditional distribution over given values in ,
denotes the Kullback-Leibler divergence. Therefore, since KL divergence is nonnegative,
where the expectations are with respect to , and the parameter can be adjusted to obtain various points on the optimal privacy-utility tradeoff curve. Alternatively, to target a specific distortion budget , the second term in (3) could be replaced with a penalty term , where is made relatively large to penalize exceeding the budget. The expectations in (3) can be conveniently approximated by Monte Carlo sampling over training set batches.
The minimax formulation of (3) can be interpreted and realized in an adversarial training framework (as illustrated by Figure 2), where the variational posterior is viewed as the posterior likelihood estimates of the sensitive attributes made by an adversary observing the release . The adversary attempts to maximize the negative log-loss , which the release mechanism attempts to minimize. The release mechanism and adversary are realized as neural networks, which take as inputs and , respectively, and produce the parameters that specify their respective distributions and within parametric families that are appropriate for the given application. For example, a release mechanism suitable for the release space could be the multivariate Gaussian
where the mean and covariance are determined by a neural network as a function of and controlled by the parameters . For brevity of notation, we will use to denote the distribution defined by the release mechanism network . Similarly, we will let denote the parametric distribution defined the adversary network that is controlled by the parameters . For each training sample tuple , we sample independent releases to approximate the loss term with
The networks are optimized with respect to these loss terms averaged over the training data (or mini-batches)
which approximates the theoretical privacy-utility tradeoff optimization problem as given in (3
), since by the law of large numbers, as,
where the expectation is with respect to . Similarly, the second term in (4) could be replaced with a penalty term to target a specific distortion budget . Similar to GANs , the minimax optimization in (5) can be more practically handled by alternating gradient descent/ascent between the two networks (possibly with multiple inner maximization updates per outer minimization update) rather than optimizing the adversary network until convergence for each release mechanism network update.
2.3 Sampling the Release Mechanism
To allow optimization of the networks via gradient methods, the release samples need to be generated such that the gradients of the loss terms can be readily calculated. Various forms of the release mechanism distribution are appropriate for different applications, and each require their own specific sampling methods. In this section, we outline some of these forms and their associated sampling methods.
2.3.1 Finite Alphabets
When the release space is a finite discrete set, we can forgo sampling altogether and calculate the loss terms via
which replaces the empirical average over samples with the direct expectation over . We found that this direct expectation produced better results than estimation via sampling, such as by applying the Gumbel-softmax categorical reparameterization trick (see [22, 23]).
Further, if and are also finite alphabets, then and can be exactly parameterized by matrices of size and , respectively. Thus, in the purely finite alphabet case, with the variables represented as one-hot vectors, the mechanism and adversary are most efficiently realized as minimal networks with no hidden layers and softmax applied to the output (to yield stochastic vectors).
2.3.2 Gaussian Approximations for Real Variables
A multivariate Gaussian release mechanism can be sampled by employing the reparameterization trick of , which first samples a vector of independent standard normal variables , and then generates , where the parameters are produced by the release mechanism network to specify a conditional Gaussian with mean and covariance .
Extending this technique, a Gaussian Mixture Model (GMM) release mechanism can be realized with a neural networkthat produces the set of parameters , where are the mixture weights. We then sample for each component distribution of the GMM, and compute the loss terms via
which combines the Gaussian sampling reparameterization trick with a direct expectation over the mixture component selection.
2.3.3 Universal Approximators
Another approach, as seen in , is to directly produce the release sample as using a neural network that takes random seed noise as an additional input. The seed noise can be sampled from a simple distribution (e.g., uniform, Gaussian, etc.) and provides the randomization of with respect to . Since the transformations applying the seed noise can be learned, this approach could potentially approximate the universal class of distributions. However, although it is not needed for training, it is generally intractable to produce an explicit expression for as implied by the behavior of the network.
3 Experimental Results
In this section, we present the privacy-utility tradeoffs that are achieved by our PPAN framework in experiments with synthetic and real data. For the synthetic data experiments, we show that the results obtained by PPAN (which does not require model knowledge and instead uses training data) are very close to the theoretically optimal tradeoffs obtained from optimizing (1) with full model knowledge. In the experiments with discrete synthetic data presented in Section 3.1, we also compare PPAN against the approach of , where an approximate discrete distribution is estimated from the training data and used in lieu of the true distribution for the optimization given by (1). For the continuous synthetic data experiments, we consider Gaussian joint distributions over the sensitive, useful, and observed data, for which we can compare the results obtained by PPAN versus the theoretically optimal tradeoffs that we derive in Section 4. We use the MNIST handwritten digits dataset for an example of applying PPAN to real data in Section 3.3, where we demonstrate optimized networks that tradeoff between concealing the digit and reducing image distortion. Table 1
summarizes the data models and distortion metrics that we use in our experiments. Our experiments were implemented using the Chainer deep learning framework, with optimization performed by their implementation of Adam .
|Case||Attribute Model||Observation Model||Distortion Metric|
|Discrete, Sec. 3.1||symmetric pair for , see (7)||and|
|Continuous, Sec. 3.2.2||,|
|Continuous, Sec. 3.2.3||and|
|Continuous, Sec. 3.2.4||,|
3.1 Discrete Synthetic Data
In our experiments with discrete data, we used a toy distribution for which the theoretically optimal privacy-utility tradeoffs have been analytically determined in . Specifically, we consider sensitive and useful attributes that are distributed over the finite alphabets , with , according to the symmetric pair distribution given by
with the parameter . The mutual information of the symmetric pair distribution is given by  as
where is the binary entropy function, and for convenience in later discussion, we define as a function of the distribution parameters and .
3.1.1 Theoretically Optimal Privacy-Utility Tradeoffs
for three specific data observation models, while using probability of error as the distortion metric, i.e.,. In one case, when the observation is the full data, i.e., , the optimal mutual information privacy-leakage as a function of the distortion (probability of error) limit is given by
In another case, when the observation is only the useful attribute, i.e., , the optimal privacy-leakage as a function is given by
We will use these two observation scenarios, full data and useful data only, in our experiments.
3.1.2 Network Architecture and Evaluation
As mentioned in Section 2.3.1
, minimal network architectures can be used for the release mechanism and adversary when all of the variables are finite-alphabet. Each network simply applies a single linear transformation (with no bias term) on the one-hot encoded input, followed by the softmax operation to yield a stochastic vector. The mechanism network takes as inputencoded as a one-hot column vector and outputs
where the network parameters are a real matrix. Note that applying the softmax operation to each column of produces the conditional distribution describing the mechanism. Similarly, the attacker network is realized as
where we use in these experiments.
In Figure 3, we compare the results of PPAN against the theoretical baselines of (8) and (9), as well as against a conventional approach suggested by , where the joint distribution of is estimated from the training data and then used in the convex optimization of (1). We used training samples generated according to the symmetric pair distribution in (7) with and . The PPAN networks were trained for epochs (for the full data observation case) with a minibatch size of , with each network alternatingly updated once per iteration. For the useful data only observation case, epochs were used. For evaluating both the PPAN and conventional approaches, we computed the actual performance of the optimized mechanisms with respect to the true data model, i.e., from the joint distribution combining the optimized with the true .
3.2 Gaussian Synthetic Data
The experiments described previously considered the setting in which the attributes belonged to a finite discrete alphabet. In this section, we consider scalar and multivariate jointly Gaussian sensitive and useful attributes. We evaluate the performance of PPAN on synthetic data generated for this model in various scenarios. The utility metric used here is the mean squared error between the release and the useful attribute.
As we note in Section 4 the optimum release for the scenarios considered here is a random variable which is jointly Gaussian with the attributes. Thus we could potentially use a mechanism network architecture that can realize the procedure described in Section 2.3.2 to generate the release. However, since the form of the optimal release distribution will not be known in practice, we use the universal approximator technique described in Section 2.3.3
. Thus we choose an architecture for the privacy mechanism which can generate real-valued release samples. The mechanism implemented in these experiments consists of three fully connected layers, with the ReLU activation function applied at the outputs of the two hidden layers, and no activation function is used at the output layer. The mechanism takes as input observationand seed noise and generates samples of the release random variable at its output. We can represent this process as the evaluation map of the function , where denotes the parameters of the mechanism network. Each component of the seed noise vector is an i.i.d. sample from Uniform.
The attacker network, with parameters denoted by
, models the posterior probabilityof the sensitive attribute given the release. We assume that
is a normal distribution with meanand covariance matrix , i.e., they are functions of the release
. For the attacker network, we use three fully connected layers to learn the mean and variance. The network takes as input the releaseand outputs the pair of evaluation maps , where the is applied componentwise on the variance vector. The ReLU activation function is applied at the outputs of the two hidden layers, and no activation function is used at the output layer. We use the PPAN mechanism to solve the min-max optimization problem described in (5). We choose in (4), and similar to the previous section, we use the penalty modification of the distortion term, i.e., the loss terms are set to be
The parameter is swept through a linearly spaced range of values. For each value of , we train the adversarial networks and evaluate the performance to obtain an operating point in the privacy-leakage versus distortion plane. The data model is sampled independently to obtain a dataset realization that is used to train and evaluate the PPAN mechanism for each different value of
. In all the scenarios described below, we used 8000 training instances sampled from the given model. For the scalar data experiments, both networks have 5 nodes per hidden layer, while 20 nodes per hidden layer were used for the multivariate data experiments. The PPAN networks were trained using stochastic gradient descent with minibatch size 200 for 250 epochs. In each iteration we do 5 gradient descent steps to update the parameters of the attacker network before updating the mechanism network. We evaluate the performance of PPAN mechanism on an independently generated test set of 4000 samples. We generated the corresponding releases for the test set as, where are seed noise realizations, and
denote the learned parameters for the mechanism network. The attribute model, observation scenario, testing procedure and the values of the other hyperparameters used in our experiments are described in the subsections below.
3.2.1 Estimating Mutual Information Leakage for the Test Set
The operating point of a trained PPAN mechanism is specified by the values of mutual information and distortion between the test set and its corresponding release. We can evaluate the empirical distortion using and . However, evaluating requires us to know the joint distribution in general, and here we have access to only the realizations and . In Section 4 we show that for the experiments considered here, the optimal is jointly Gaussian with . Motivated by this, we estimate in the following manner. We find the empirical covariance matrix of and , denoted as
In all our experiments, and have the same number of dimensions. Consider jointly Gaussian random variables and such that . Then we have
We use as an estimate of the mutual information leakage in the sequel for Gaussian synthetic data. We note that this underestimates the true mutual information leakage since
where is the linear MMSE estimate of as a function of . We use this estimate only for its simplicity. One could certainly use other non-parametric estimates of mutual information.
3.2.2 Rate Distortion
We first apply the PPAN framework to the problem of finding the minimum required code rate in order to describe a multivariate Gaussian source within a given value of mean squared error. This is a standard problem in information theory, for example, see [29, Chap. 10]. This problem can be viewed as a degenerate case of the PPAN framework with , i.e., the sensitive and useful attributes are the same and the observed dataset is the attribute. The release corresponds to an estimate with mean squared error less than a distortion level while retaining as much expected uncertainty about as possible.
We choose the attribute model . For the multiplier of the distortion term in the penalty formulation, we use the value . We run the experiment for different values of the target distortion, linearly spaced between 0 to 2.5. The inputs to the adversarial network are realizations of the attributes and seed noise. The seed noise is chosen to be a random vector of length 8 with each component i.i.d. Uniform. The testing procedure is as follows. We evaluate the mechanism network for all in the test set. Here, are the learned parameters and consists of independent seed noise samples. Since , the utility loss is quantified by the empirical average of the MSE over all test samples. The privacy loss is quantified by the estimate as described in Section 3.2.1.
The optimal privacy-utility tradeoff (or, rate-distortion) curve is given as , where are the true parameters of the attribute distribution and is the allowed squared error distortion in the th component. We plot the (privacy-leakage, utility loss) pairs returned by the PPAN mechanism along with the optimal tradeoff curve in Figure 4. One can see that the operating points attained by the PPAN mechanism are very close to the theoretical optimum tradeoff for a wide range of target distortion values.
3.2.3 Scalar Attribute: Useful Data Only and Full Data
Here we consider jointly Gaussian sensitive and useful attributes such that . We consider two different observation models here: , called useful data only (UD) and , called full data (FD). For the useful data only observation model, the input to the adversarial network is the useful attribute and seed noise , while for the full data observation model, the input is the pair of attributes and seed noise . In both cases, is a scalar random variable following Uniform. The values of the multipliers chosen are: and . In each case, we run experiments for 20 different values of the target distortion with and . The output of the mechanism in the testing phase can be denoted as
where are the learned parameters in the two cases and are independent samples of the seed noise. The utility loss is given in both cases by the empirical average of the MSE over all test samples.
The privacy loss is computed following the procedure described in Section 3.2.1. The (privacy-leakage, distortion) pairs returned by PPAN are plotted along with the optimal tradeoff curves (from Propositions 1 and 3) in Figure 5. In both the observation models, we observe that the PPAN mechanism generates releases that have nearly optimal privacy-leakage over a range of distortion values.
3.2.4 Vector Attribute: Useful Data Only
Here we consider multivariate jointly Gaussian sensitive and useful attributes where both and . We choose the multiplier in this case. The value of the target distortion in the penalty formulation is linearly varied in the range . For each value of , we sample an independent dataset realization which is used to train and test the adversarial networks. The seed noise is a vector random variable of length 8, each component of it being i.i.d. Uniform. As the observation model is useful data only, we have that . The utility loss is measured by mean squared error between and and the privacy-leakage is measured using the procedure in Section 3.2.1. We plot the (privacy-leakage, distortion) pairs returned by the PPAN mechanism along with the optimal tradeoff curve (from Proposition 2) in Figure 6. We see that the operating points of the PPAN mechanism are very close to the theoretically optimum tradeoff curve over a wide range of target distortion values.
3.3 MNIST Handwritten Digits
The MNIST dataset consists of 70 thousand labeled images of handwritten digits split into training and test sets of 60K and 10K images, respectively. Each image consists of grayscale pixels, which we handle as normalized vectors in .
In this experiment, we consider the image to be both the useful and observed data, i.e., , the digit label to be the sensitive attribute , and the mechanism releases an image . We measure the distortion between the original and released images with
which, for a fixed
, corresponds to minimizing the average KL-divergence between corresponding pixels that are each treated as a Bernoulli distribution. Thus, the privacy objective is to conceal the digit, while the utility objective is to minimize image distortion.
The mechanism and adversary networks both use two hidden layers with 1000 nodes each and fully-connected links between all layers. The hidden layers use as the activation function. The mechanism input layer uses nodes for the image concatenated with 20 random Uniform seed noise values. The mechanism output layer uses 784 nodes with the sigmoid activation function to directly produce an image in . Note that the mechanism network is an example of the universal approximator architecture mentioned in Section 2.3.3. The attacker input layer uses 784 nodes to receive the image produced by the mechanism. The attacker output layer uses 10 nodes normalized with a softmax activation function to produce a distribution over the digit labels .
For some experiments, we also employ the standard GAN approach by also adding a discriminator network to further encourage the mechanism toward producing output images that resemble realistic digits. The discriminator network architecture uses a single hidden layer with 500 nodes, and has an output layer with one node that uses the sigmoid activation function. The discriminator network, denoted by with parameters , attempts to distinguish the outputs of the mechanism network from the original training images. Its contribution to the overall loss is controlled by a parameter (with zero indicating its absence). Incorporating this additional network, the training loss terms are given by
where is generated from the input image by the mechanism network controlled by the parameters . The overall adversarial optimization objective with both the privacy adversary and the discriminator is given by
Figures 7 and 8 show example results from applying trained privacy mechanisms to MNIST test set examples. The first, Figure 7, shows the results with the standard PPAN formulation, trained via (10) with . The second, Figure 8, shows the results when the additional discriminator network is introduced, which is jointly trained via (10) with . The first row of each figure depicts the original test set examples input to the mechanism, while the remaining rows each depict the corresponding outputs from a mechanism trained with different values for . From the second to last rows of the figures, the value of is decreased, reducing the emphasis on minimizing distortion. We see in both figures that the outputs start from accurate reconstructions and become progressively more distorted while the digit becomes more difficult to correctly recognize as decreases. In Figure 7, we see that mechanism seems to learn to minimize distortion while rendering the digit unrecognizable, which in some cases results in an output that resembles a different digit. In Figure 8, we see that the additonal discriminator network encourages outputs that more cleanly resemble actual digits, which required lower values for to generate distorted images and also led to a more abrupt shift toward rendering a different digit. For both sets of experiments, the networks were each alternatingly updated once per batch (of 100 images) over 50 epochs of the 60K MNIST training set images. We used the 10K test images to objectively evaluate the performance of the trained mechanisms for Figure 9, which depicts image distortion versus privacy measured by the accuracy of the adversary in recognizing the original digit and the variational lower bound for mutual information.
4 Optimum Privacy Utility Tradeoff for Gaussian Attributes
In Section 3 we compare the (privacy, distortion) pairs achieved by the model-agnostic PPAN mechanism with the optimal model-aware privacy-utility tradeoff curve. For jointly Gaussian attributes and mean squared error distortion, we can obtain, in some cases, analytical expressions for the optimal tradeoff curve as described below. Some of the steps in the proofs use bounding techniques from rate-distortion theory, which is to be expected given the tractability of the Gaussian model and the choice of mutual information and mean squared error as the privacy and utility metrics respectively.
(Useful Data only: Scalar Gaussian with mean squared error) In problem (1), let be jointly Gaussian scalars with zero means , variances respectively, and correlation coefficient . Let mean squared error be the distortion measure. If the observation (Useful Data only observation model), then the optimal release corresponding to
is given by
where and . The mutual information leakage caused by releasing is
The result of Proposition 1 is known in the existing literature, e.g., see [8, eq. 8] and [10, example 2]. For completeness, we present the proof of this result in Appendix 6.1. The theoretical tradeoff curve in Figure 5 was obtained using the expressions in Proposition 1.
The case of Useful Data only observation model for jointly Gaussian vector attributes and mean squared error is also considered in , where they provide a numerical procedure to evaluate the tradeoff curve. Here, we focus on a special case where we can compute the solution analytically.
Consider the generalization to vector variables of problem (11)
Let be jointly Gaussian vectors of dimensions and respectively. We assume that have zero means and non-singular covariance matrices . Let denote the cross-covariance matrix and
the normalized cross-covariance matrix with singular value decomposition. We assume that all singular values of , denoted by , are strictly positive. If
denote reparameterized variables, then are zero-mean, jointly Gaussian, with identity covariance matrices respectively and diagonal cross-covariance matrix . Since the transformation from to is invertible, . The mean squared error between and :
For the special case when for some , the vector problem (12) reduces to the following problem
If , then the minimizer of (13) is given by
where and for all , , , where denotes the -th main diagonal entry of , and the value of parameter can be found by the equation . The mutual information between the release and the sensitive attribute is .
For the case of full data observation, we have the following result.
(Full Data: Scalar Gaussian with mean squared error) In problem (1), let be jointly Gaussian scalars with zero means, unit variances, and correlation coefficient . Let mean squared error be the distortion measure. If the observation (full data observation model), then the optimal release corresponding to
is given by
The mutual information leakage caused by this release is
In this paper, we developed a data-driven framework for optimizing privacy-preserving data release mechanisms. The key to this approach is the application of adversarially-trained neural networks, where the mechanism is realized as a randomized network, and a second network acts as a privacy adversary that attempts to recover sensitive information. By estimating the posterior distribution of the sensitive variable given the released data, the adversarial network enables a variational approximation of mutual information. This allows our framework to approach the information-theoretically optimal privacy-utility tradeoffs, which we demonstrate in experiments with discrete and continuous synthetic data. We also conducted experiments with the MNIST handwritten digits dataset, where we trained a mechanism that trades off between minimizing the pixel-level image distortion and concealing the digit. While we focused on expected distortion to measure (dis)utility, our framework can be adapted to other general utility measures. For example, in the following subsection, we outline an adaptation to utility measured by the mutual information between the useful information and the released data.
5.1 Mutual Information Utility
The conditional entropy is an alternative measure for distortion, which corresponds to the utility objective of maximizing the mutual information , since is fixed. When is used as the distortion measure in a scenario where the observation , the privacy-utility tradeoff optimization problem, as described in Section 2.1, becomes equivalent to the Information Bottleneck problem considered in . In other scenarios where the observation , this problem becomes the Privacy Funnel problem introduced by . The formulation of (3) can be modified to address conditional entropy distortion by introducing another variational posterior and using the following optimization, which applies a second variational approximation of mutual information,
where the expectations are with respect to , and the parameter can be adjusted to obtain various points along the optimal tradeoff curve. In a similar fashion to the approach in Section 2.2, this optimization problem can be practically addressed via the training of three neural networks, which respectively parameterize the mechanism and the two variational posteriors and .
-  L. Sweeney, “Simple demographics often identify people uniquely,” Carnegie Mellon University, Data Privacy Working Paper, 2000.
-  A. Narayanan and V. Shmatikov, “Robust de-anonymization of large sparse datasets,” in IEEE Symp. on Security and Privacy. IEEE, 2008, pp. 111–125.
-  L. Sweeney, “k-anonymity: A model for protecting privacy,” Intl. Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 10, no. 5, pp. 557–570, 2002.
-  A. Machanavajjhala, D. Kifer, J. Gehrke, and M. Venkitasubramaniam, “l-diversity: Privacy beyond k-anonymity,” ACM Trans. on Knowledge Discovery from Data, vol. 1, no. 1, p. 3, 2007.
-  N. Li, T. Li, and S. Venkatasubramanian, “t-closeness: Privacy beyond k-anonymity and l-diversity,” in IEEE Intl. Conf. on Data Eng. IEEE, 2007, pp. 106–115.
-  C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography. Springer, 2006, pp. 265–284.
-  H. Yamamoto, “A source coding problem for sources with additional outputs to keep secret from the receiver or wiretappers,” IEEE Trans. on Information Theory, vol. 29, no. 6, pp. 918–923, 1983.
-  D. Rebollo-Monedero, J. Forné, and J. Domingo-Ferrer, “From t-closeness-like privacy to postrandomization via information theory,” IEEE Trans. Knowl. Data Eng., vol. 22, no. 11, pp. 1623–1636, 2010.
-  F. du Pin Calmon and N. Fawaz, “Privacy against statistical inference,” in Allerton Conf. on Comm., Ctrl., and Comp., 2012, pp. 1401–1408.
-  L. Sankar, S. R. Rajagopalan, and H. V. Poor, “Utility-privacy tradeoffs in databases: An information-theoretic approach,” IEEE Trans. on Information Forensics and Security, vol. 8, no. 6, pp. 838–852, 2013.
-  Y. O. Basciftci, Y. Wang, and P. Ishwar, “On privacy-utility tradeoffs for constrained data release mechanisms,” in Information Theory and Applications Workshop, Feb. 2016.
-  D. Kifer and A. Machanavajjhala, “No free lunch in data privacy,” in Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. ACM, 2011, pp. 193–204.
-  C. Liu, S. Chakraborty, and P. Mittal, “Dependence makes you vulnberable: Differential privacy under dependent tuples,” in Network and Distributed System Security Symposium, 2016.
-  Y. Wang, Y. O. Basciftci, and P. Ishwar, “Privacy-utility tradeoffs under constrained data release mechanisms,” arXiv preprint arXiv:1710.09295, 2017. [Online]. Available: https://arxiv.org/abs/1710.09295
-  D. Barber and F. V. Agakov, “The im algorithm: A variational approach to information maximization,” in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2003, p. None. [Online]. Available: http://books.nips.cc/papers/files/nips16/NIPS2003_AA26.pdf
-  I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, 2014, pp. 2672–2680.
-  A. Hindupur, “The gan zoo,” https://deephunt.in/the-gan-zoo-79597dc8c347, 2017.
-  H. Edwards and A. J. Storkey, “Censoring representations with an adversary,” CoRR, vol. abs/1511.05897, 2015. [Online]. Available: http://arxiv.org/abs/1511.05897
-  J. Hamm, “Enhancing utility and privacy with noisy minimax filters,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March 2017, pp. 6389–6393.
-  C. Huang, P. Kairouz, X. Chen, L. Sankar, and R. Rajagopal, “Context-aware generative adversarial privacy,” arXiv preprint arXiv:1710.09549, 2017.
-  X. Chen, X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Interpretable representation learning by information maximizing generative adversarial nets,” in Advances in Neural Information Processing Systems 29, D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, Eds. Curran Associates, Inc., 2016, pp. 2172–2180. [Online]. Available: http://papers.nips.cc/paper/6399-infogan-interpretable-representation-learning-by-information-maximizing-generative-adversarial-nets.pdf
-  C. J. Maddison, A. Mnih, and Y. W. Teh, “The concrete distribution: A continuous relaxation of discrete random variables,” arXiv preprint arXiv:1611.00712, 2016.
-  E. Jang, S. Gu, and B. Poole, “Categorical reparameterization with gumbel-softmax,” arXiv preprint arXiv:1611.01144, 2016.
-  D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
-  A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, “Adversarial autoencoders,” arXiv preprint arXiv:1511.05644, 2015.
-  A. Makhdoumi and N. Fawaz, “Privacy-utility tradeoff under statistical uncertainty,” in Allerton Conf. on Comm., Ctrl., and Comp., 2013, pp. 1627–1634.
S. Tokui, K. Oono, S. Hido, and J. Clayton, “Chainer: a next-generation open
source framework for deep learning,” in
Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS), 2015. [Online]. Available: http://learningsys.org/papers/LearningSys_2015_paper_33.pdf
-  D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
-  T. M. Cover and J. A. Thomas, Elements of information theory, 2nd ed. John Wiley & Sons, 2012.
-  N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” in Allerton Conf. on Comm., Ctrl., and Comp., 1999, pp. 368––377.
-  A. Makhdoumi, S. Salamatian, N. Fawaz, and M. Médard, “From the information bottleneck to the privacy funnel,” in IEEE Information Theory Workshop, 2014, pp. 501–505.
6.1 Proof of Proposition 1
We can expand the mutual information term as follows,
) is true since the zero-mean normal distribution has the maximum entropy for a given value of the second moment. Let, then is jointly Gaussian and we have that
Hence, is independent of . Since also forms a Markov chain, we have that is conditionally independent of given . Due to the distortion constraint, we can upper bound in the following manner.
Inequality (18) is true because , and equation (