1 Introduction
Generative adversarial networks (GANs) have been applied successfully in various research fields such as natural image modeling Radford2015Unsupervised , image translation Isola2016Image ; Zhu2017Unpaired , crossmodal image generation Dash2017TAC
, image superresolution
Ledig2016Photo Odena2016Semi and sequential data modeling Mogren2016C ; Yu2016SeqGAN. Different from explicit density estimation based models
kingma2014semi ; Oord2016Pixel ; hinton2012a, GANs are implicit generative models with two neural networks playing minmax game to find a map from random noise to target distribution, in which the generator tries to generate fake samples to fool discriminator and the discriminator tries to distinguish them from real samples
Goodfellow2014Generative . In original GANs formula, optimal discriminator measures the JensenShannon divergence between real data distribution and generated distribution. The discrepancy measure can be generalized to fdivergence nowozin2016f or replaced by earthmover distance arjovsky2017wasserstein . Despite the success, GANs are notoriously difficult to trainkodali2018on ; arjovsky2017towards , which are very sensitive to hyperparameters. When the support of these two distributions are approximately disjoint, gradient given by discriminator with standard objective may vanish, and training becomes unstable arjovsky2017wasserstein . More seriously, generated distribution can fail to cover the whole data distribution and collapse to a single mode in some cases dumoulin2017adversarially ; che2016mode .The condition of discriminator determines the training stability and performance to a great extent. From a practical standpoint, representation capacity of discriminator realized by a neural network is not infinite. Meanwhile, the discriminator is usually not optimal to measure true discrepancy when trained in an alternative manner. On the other hand, discriminator as a binary classifier is also vulnerable to adversarial samples
szegedy2014intriguing ; goodfellow2015explaining , which can be validated by the experiments shown in Figure 1. Real samples added by imperceptible perturbation, eg, with a norm of 1/255, can mislead classifier to give wrong prediction. Adversarial samples can be easily crafted by gradientbased method such as Fast Gradient Sign Method (FGSM) goodfellow2015explaining and Basic Iterative Method (BIM) kurakin2017adversarial . It should be noted that the gradient given by discriminator that guides update of the generator is exactly the same as gradient used to craft adversarial samples of the discriminator. In other words, the gradient contains uninformative adversarial noise which is imperceptible but can mislead the generator.However, in vanilla training procedure of GANs, generator can still succeed to generate meaningful samples instead of adversarial noise. This is because discriminator is adversarially trained with diverse generated fake samples, which can partly alleviate misleading effect of adversarial noise although training is usually unstable. Nevertheless, from the perspective of symmetry, adversarial training on real samples, ie, training with adversarial samples of real data, does not exist. Intuitively, we can improve GAN with adversarial training on real samples to further reduce adversarial noise. In other words, we propose to augment training set of discriminator with adversarial samples of real data. As shown in Figure 2, we visualize the gradient of standard discriminator and the discriminator further adversarially trained on real samples. The gradient of standard discriminator seems like uninformative noise but the gradient of further adversarially trained discriminator contains more semantic information, eg, profile of face. Therefore, the proposed method can indeed eliminate adversarial noise contained in gradient of discriminator.
What’s more, we validate the proposed method on image generation with widely adopted DCGAN Radford2015Unsupervised and ResNet he2015deep ; gulrajani2017improved architecture, which shows consistent improvement of training stability and acceleration of convergence. More importantly, FID scores of generated samples are improved by relative to the baseline on CIFAR10, CelebA, and LSUN. The computation overhead of additional adversarial training is about . We term the proposed method as adversarial symmetric GAN (ASGAN), for the key point of the method is adversarial training both on fake samples and real samples.
2 Related Work
There is a large body of work on how to stabilize GANs training and alleviate mode collapse. arjovsky2017towards
proved that the widely adopted nonsaturating loss function for the generator can be decomposed into Kullback–Leibler divergence minus two JensenShannon divergence when discriminator trained to be optimal, which accounts for training instability and mode dropping during GANs training.
metz2016unrolled proposed to unroll the optimization of discriminator as surrogate objective to guide update of the generator, which shows improvement of training stability with relatively large computation overhead. kodali2018on claimed that the existence of undesirable local equilibria is responsible for mode collapse and proposed to regularize the discriminator around real data samples with gradient penalty.Integral probability metric (IPM) based GANs such as Wasserstein GAN
arjovsky2017wasserstein and its variants gulrajani2017improved ; wu2018wasserstein can solve gradient vanishing in GANs training theoretically but it is not simple to make discriminator 1Lipschitz required by the duality conversion practically. Wasserstein GAN arjovsky2017wasserstein suggests earthmover (EM) distance as a measure of discrepancy between two distributions and adopts weight clip to make the discriminator 1Lipschitz constrained. WGANGP gulrajani2017improved adopts gradient penalty to regularize discriminator in a less rigorous way, but it requires calculation of the second order derivative with remarkable computation overhead. Spectral normalization on the weight of discriminator proposed by Miyato2018Spectral can make discriminator 1Lipschitz constrained efficiently, but capacity of discriminator is significantly constrained.Adversarial vulnerability is an intriguing property of neural networkbased classifier szegedy2014intriguing . A welltrained neural network can give totally wrong prediction to adversarial samples that human can recognize accurately. Smallmagnitude adversarial perturbation added to benign data can be easily calculated based on gradient goodfellow2015explaining ; carlini2017towards ; dong2018boosting . goodfellow2015explaining proposed to augment training data with adversarial samples to improve the robustness of neural networks, which can smooth the decision boundary of classifier around training samples. Gradient of adversarially trained classifier contains more semantic information and less adversarial noise tsipras2018robustness ; Kim2019Bridging .
Some work tried to craft or defense against adversarial samples using GANs. Xiao2018Generating proposed to generate adversarial samples efficiently with GANs, in which a generator is used to generate adversarial perturbation for target classifier given original samples. Shen2017AE proposed AEGAN to eliminate adversarial perturbation in an adversarial training manner, which can generate images with better perceptual quality. Different from their motivations, our work aims at improving the robustness of discriminator by introducing adversarial training on real samples, which does not exist in classic GANs. zhou2018dont proposed to perform additional adversarial training on fake samples to robustify discriminator. However, our work clarifies that standard GAN training is approximately equivalent to adversarial training on fake samples.
3 Method
3.1 Vanilla GAN
In GAN proposed by Goodfellow2014Generative , the generator parameterized by tries to generate fake samples to fool discriminator and the discriminator parameterized by tries to distinguish between generated samples and real samples. The formulation using minmax optimization is as follows:
(1) 
where is the objective function. Equation 1 can be formulated as a binary classification problem with cross entropy loss:
(2) 
where is real data distribution and noise
obeys standard Gaussian distribution. When discriminator is trained to be optimal, the training objective for the generator can be reformulated as JensenShannon divergence, which can measure dissimilarity between two distributions.
In practice, we use minibatch gradient descent to optimize generator and discriminator alternatively. At each iteration, update rule can be derived as follows:
(3) 
where and are the learning rate of discriminator and generator, respectively. denotes the objective function of minibatch with real samples and fake samples, which is:
(4) 
After updating parameters of networks, fake samples generated by
are adjusted as following equation according to chain rule:
(5) 
where is a Jacobian matrix. The updated can be seen as adversarial samples of the discriminator at this iteration because is usually small. These samples will be fed into the discriminator at future iteration to perform adversarial training. From this point of view, vanilla GANs mainly include adversarial training on fake samples, which is illustrated as the first pass in Figure 3. Nevertheless, adversarial training of discriminator on real samples does not exist in this framework, which makes adversarial training unsymmetric and unbalanced. Adversarial noise contained in the gradient of nonrobust discriminator can make training unstable because of the unsmoothed decision boundary of discriminator.
3.2 Adversarial training on real samples
To further robustify discriminator, we incorporate adversarial training on real samples into vanilla GANs. Specifically, we perform adversarial training after Equation 3 at each iteration as the following equation:
(6) 
where is an adversarial sample of discriminator, perturbation of which is the gradient of discriminator with respect to . The adversarial sample can be calculated with constant as follows:
(7) 
This adversarial training formulation is adopted from goodfellow2015explaining , which calculates norm constrained perturbation by linearizing objective function. Adversarial training on real samples of discriminator is illustrated as the second pass in Figure 3, where denotes the minus of . The reason why we adopt this training scheme is two fold: First, gradient used to craft perturbation can be obtained from the first backward pass conveniently and the overall computation overhead of additional training is relatively low, ie, about 25%. Second, the simple scheme can already provide significant improvement. We do not adopt other more complicated adversarial training schemes such as Projected Gradient Descent (PGD) MadryTowards because further improvement is marginal. Actually, we can feed both real samples and adversarial samples at one pass but this need additional pass to calculate adversarial perturbation, which is computationalinefficient. Therefore, we train discriminator with real samples and adversarial samples in separate passes, respectively. Please refer to Algorithm 1 for more details about symmetric adversarial training.
(8) 
(9) 
(10) 
(11) 
Minibatch stochastic gradient descent training of ASGAN. We set perturbation
to as default for image generation tasks.3.3 Robust optimization
By introducing adversarial training on real samples, we generalize the original objective of GAN to the following one that forces discriminator to be robust:
(12) 
In fact, when training data of real samples is infinite and the distribution of real data is continuous, the above objective is approximately equivalent to the original. However, in practice, contrary to the fake data, number of real samples is always limited, which partly accounts for the existence of adversarial samples. With the proposed objective, discriminator is not only required to classify real data correctly but also should be not vulnerable to small perturbation. In a sense, adversarial training on real samples regularizes discriminator by augmenting training data, which can smooth the decision boundary of discriminator. Moreover, the capability of two networks become more balanced. This can prevent discriminator from being too strong to alleviate training collapse.
3.4 Effective perturbation level
It is crucial to set an appropriate perturbation level to make adversarial training effective. When perturbation set to zero, the proposed method degrades to updating discriminator twice on the same real data. When perturbation set to a too large value, real data will be drastically perturbed. Semantic information and quality will be changed, which can mislead discriminator to recognize degraded samples as real data incorrectly. In addition, we suggest to set perturbation to zero at the beginning of training in case discriminator is too weak. We do extensive experiments on how perturbation affects training in the next section.
4 Experiments
For the purpose of evaluating our method and investigating the reason behind its efficacy, we test our adversarial training method on image generation on CIFAR10, CelebA, and LSUN with DCGAN and ResNet architecture. CIFAR10 is a wellstudied dataset of 32 x 32 natural images, containing 10 categories of objects and 60k samples. CelebA is a face attributes dataset with more than 200k images. LSUN is a large scale scene understanding dataset with 10 categories and we choose 3000k images labeled as bedroom for training. For fast validation, We resize images in CelebA and LSUN to 64 x 64. Before fed into discriminator, images are rescaled within
. Dimension of the latent vector is set to 100 for all implementations.
In ResNet architecture, the residual block is organized as BatchNormReLUResampleConvBatchNormReLUConv with skip connection. We use bilinear interpolation for upsampling and average pooling for downsampling. Batch normalization
Ioffe2015Batch is used for both generator and discriminator. Parameters of network are initialized by Xavier method glorot2010understanding .In DCGAN architecture, the basic block is organized as ConvBatchNormLeakyReLU for discriminator and ConvTransposedBatchNormReLU for generator. The weights of convolution are initialized by normal distribution with zero mean and 0.02 standard deviation. We do not use bias for convolution.
We implement models by Pytorch with acceleration of RTX 2080ti GPUs. We train networks with Adam optimizer
kingma2014adam with learning rate 2e4. is set to 0.5 and is set to 0.999. Because training standard GAN on CelebA is unstable, we decrease learning rate of discriminator to 5e5 to balance training as TTUR training strategy Heusel2017GANs. We train models on CelebA for 100 epochs, CIFAR10 for 200 epochs, LSUN for 10 epochs.
In this paper, we use Fréchet inception distance (FID) Heusel2017GANs and inception score Salimans2016Improved to measure model performance, both of which are wellstudied metric of image modeling. Our source code is available on Github https://github.com/CirclePassFilter/AS_GAN.
4.1 Evaluation with different perturbation level
In this section, we conduct experiments on how perturbation level affects performance improvement. Specifically, we do unsupervised image generation on CIFAR10 with DCGAN architecture in different settings of perturbation level. Due to the large searching space, we select several typical values for experiments such as {0,1,2,3,4}/255. All experiments are run three times independently to reduce effect of randomness.
Shown as Figure 4, our method performs better than the baseline when lies in interval . FID score can be improved by with perturbation of . However, when the perturbation level is too tiny (), the method improves original model marginally. On this condition, the effect of adversarial training is limited. Remarkably, when the imposed perturbation is too strong, the model performs even worse than the baseline. This is because discriminator recognizes degraded samples as real data and can not provide correct gradient to update generator.
Above all, with appropriate perturbation, discriminator can be regularized to be more robust, facilitating itself to produce more accurate and informative gradient. In this way, generator obtains more reliable gradient, which can alleviate collapse of training and improve fidelity of generated samples.
4.2 Ablation study
In addition, we plot the training curve of FID on CIFAR10 as shown in Figure 5. In particular, when is zero, the proposed method degrades to updating discriminator twice on the same real data, result of which is comparable to the baseline and much worse than the setting with appropriate perturbation. This indicates that performance improvement provided by the proposed method does not attribute to additional update of discriminator. Furthermore, we do another experiment in which gradient used to craft perturbation is replaced by Gaussian noise. FID score of this setting is slightly worse than the baseline, which indicates that perturbation of gradient direction instead of random direction is a key factor that makes the proposed method effective.
4.3 Evaluation with different architectures
To explore the compatibility of the proposed method, we test it with widely adopted DCGAN and ResNet architecture on CIFAR10 and CelebA. Figure 6 plots the comparison results. With the proposed method, FID scores on CelebA is improved significantly and convergence is also accelerated. Even with the setting in which vanilla GANs collapse, our model can still converge stablely.
In addition, we further test the proposed method with spectral normalization. Results show that the proposed method can work well with spectral normalization and achieve a FID of 5.88 on CelebA, exceeding the FID of 11.71 achieved by the proposed method alone. Therefore, we can combine these two schemes to obtain better performance in practice.
As shown in Table 1, through unsupervised training on CIFAR10, CelebA and bedroom in LSUN, the proposed method with spectral normalization can achieve comparable performance to state of the art. It should be noted that the six rows at the bottom show the results of our implementation. Generated samples are shown in Figure 7.
Method  Inception score  FID  

CIFAR10  CIFAR10  CelebA  LSUN  
(Standard CNN)  
WGANGP  6.68.06  40.2  21.2  
SNGAN  7.58.12  25.5  
WGANGP(ResNet)  7.86.07  18.8  18.4  26.8 
WGANdiv(ResNet)  18.1  15.2  15.9  
DCGAN  7.05.14  28.05  20.45  25.36 
ASDCGAN  7.21.02  25.50  10.90  18.08 
ASSNDCGAN  7.24.14  24.50  10.60  21.84 
ResNet  7.35.16  22.92  25.72  175.70 
ASGAN(ResNet)  7.65.15  21.84  11.71  45.96 
ASSNGAN(ResNet)  7.84.17  22.26  5.88  8.00 
5 Analysis
5.1 Computation overhead
The training computation overhead of the proposed method is about 25% relative to the baseline, which is comparable to that of spectral normalization and much smaller than that of gradient penalty. Comparison of average training time of one epoch of different methods is shown in Table 2.
Setting  DCGAN  (ours)ASDCGAN  SNDCGAN  DCGANGP 
Training time  19.83s  26.40s  24.50s  31.57s 
5.2 Gradient visualization
Gradient given by discriminator is the key to update generator. Hence, we visualize the gradient in Figure 2, which shows that the gradient of adversarially trained discriminator contains more semantic information, eg, profile of face, but the gradient of standard discriminator looks like uninformative noise. We further show the histogram of the gradient of discriminator with respect to real samples as Figure 9a. Our method can obtain more sparse gradient and lower L1 norm (Figure 9b) as training iteration increases, which means adversarial noise in gradient is partly eliminated. What’s more, the proposed method can augment training data and smooth the decision boundary of discriminator, which can alleviate mode collapse to a large extent shown as Figure 8. By means of adversarial training evenly on both real and fake samples, we make the training scheme symmetric and stable.
5.3 Training loss
With the proposed method, the confidence and the loss of discriminator become more smooth and stable during training, which are depicted as Figure 10. Moreover, when adversarial perturbation is appropriate for training, the confidence about adversarial samples will be lower than by a large margin because discriminator is sensitive to adversarial samples in the beginning. With more iterations, discriminator will become robust to adversarial samples. As shown in Table 3, accuracy on real samples under FGSM attack with perturbation of 1/255 is improved significantly compared to the baseline. Similarly, the loss is also stabilized with our algorithm (Figure 10b).
Model  standard accuracy  adversarial accuracy 

GAN(ResNet)  0.98  0.50 
ASGAN(ResNet)  0.99  0.93 
6 Conclusion
The relationship between GANs and adversarial samples has been a open question since both models emerged. In this paper, we analyze that adversarial training on fake samples is already taken into account in standard GAN training, but adversarial training on real samples does not exist, which can make training unbalanced and unstable. This is because gradient given by nonrobust discriminator contains more adversarial noise, which can mislead update of the generator. In order to make training scheme symmetric and make discriminator more robust, we introduce adversarial training on real samples. We validate the proposed method on image generation on CIFAR10, CelebA, and LSUN with varied network architectures. Experiments show that the gradient of adversarially trained discriminator contains less adversarial noise. Convergence speed and performance are improved with additional computation overhead. Moreover, mode collapse is alleviated. The proposed method with spectral normalization can achieve comparable FID to state of the art on these datasets.
References
 (1) A. Radford, L. Metz, S. Chintala, Unsupervised representation learning with deep convolutional generative adversarial networks, Computer Science.

(2)
P. Isola, J. Y. Zhu, T. Zhou, A. A. Efros, Imagetoimage translation with conditional adversarial networks (2017) 5967–5976.

(3)
J. Y. Zhu, T. Park, P. Isola, A. A. Efros, Unpaired imagetoimage translation using cycleconsistent adversarial networks, in: IEEE International Conference on Computer Vision, 2017.
 (4) A. Dash, J. C. B. Gamboa, S. Ahmed, M. Z. Afzal, M. Liwicki, Tacgan  text conditioned auxiliary classifier generative adversarial network.
 (5) C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Aitken, A. Tejani, J. Totz, Z. Wang, W. Shi, Photorealistic single image superresolution using a generative adversarial network (2017) 105–114.
 (6) A. Odena, Semisupervised learning with generative adversarial networks, arXiv preprint arXiv:1606.01583.
 (7) O. Mogren, Crnngan: Continuous recurrent neural networks with adversarial training, arXiv preprint arXiv:1611.09904.
 (8) L. Yu, W. Zhang, J. Wang, Y. Yu, Seqgan: Sequence generative adversarial nets with policy gradient.
 (9) D. P. Kingma, D. J. Rezende, S. Mohamed, M. Welling, Semisupervised learning with deep generative models (2014) 3581–3589.

(10)
A. V. D. Oord, N. Kalchbrenner, K. Kavukcuoglu, Pixel recurrent neural networks (2016) 1747–1756.

(11)
G. E. Hinton, A practical guide to training restricted boltzmann machines, Neural Networks: Tricks of the Trade (2nd ed.) (2012) 599–619.
 (12) I. J. Goodfellow, J. PougetAbadie, M. Mirza, X. Bing, D. WardeFarley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: International Conference on Neural Information Processing Systems, 2014.
 (13) S. Nowozin, B. Cseke, R. Tomioka, f gan: training generative neural samplers using variational divergence minimization, in: NIPS’16 Proceedings of the 30th International Conference on Neural Information Processing Systems, 2016, pp. 271–279.
 (14) M. Arjovsky, S. Chintala, L. Bottou, Wasserstein gan, stat 1050 (2017) 9.
 (15) N. Kodali, J. Hays, J. Abernethy, Z. Kira, On convergence and stability of gans, arXiv preprint arXiv:1705.07215.
 (16) M. Arjovsky, L. Bottou, Towards principled methods for training generative adversarial networks, arXiv preprint arXiv:1701.04862.
 (17) V. Dumoulin, I. Belghazi, B. Poole, A. Lamb, M. Arjovsky, O. Mastropietro, A. C. Courville, Adversarially learned inference, in: ICLR 2017 : International Conference on Learning Representations 2017, 2017.
 (18) T. Che, Y. Li, A. P. Jacob, Y. Bengio, W. Li, Mode regularized generative adversarial networks, arXiv preprint arXiv:1612.02136.
 (19) C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, R. Fergus, Intriguing properties of neural networks, international conference on learning representations.
 (20) I. J. Goodfellow, J. Shlens, C. Szegedy, Explaining and harnessing adversarial examples, international conference on learning representations.
 (21) A. Kurakin, I. J. Goodfellow, S. Bengio, Adversarial examples in the physical world, international conference on learning representations.
 (22) K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition (2016) 770–778.
 (23) I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, A. C. Courville, Improved training of wasserstein gans, in: Advances in Neural Information Processing Systems, 2017, pp. 5767–5777.
 (24) L. Metz, B. Poole, D. Pfau, J. SohlDickstein, Unrolled generative adversarial networks, arXiv preprint arXiv:1611.02163.
 (25) J. Wu, Z. Huang, J. Thoma, D. Acharya, L. V. Gool, Wasserstein divergence for gans, in: Computer Vision – ECCV 2018. 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part V, Vol. 11209, 2018, pp. 673–688.
 (26) T. Miyato, T. Kataoka, M. Koyama, Y. Yoshida, Spectral normalization for generative adversarial networks.
 (27) N. Carlini, D. A. Wagner, Towards evaluating the robustness of neural networks, in: 2017 IEEE Symposium on Security and Privacy (SP), 2017, pp. 39–57.

(28)
Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, J. Li, Boosting adversarial attacks with momentum, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 9185–9193.
 (29) D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, A. Madry, Robustness may be at odds with accuracy.
 (30) B. Kim, J. Seo, T. Jeon, Bridging adversarial robustness and gradient interpretability, arXiv preprint arXiv:1903.11626.
 (31) C. Xiao, L. Bo, J. Y. Zhu, W. He, D. Song, Generating adversarial examples with adversarial networks (2018) 3905–3911.
 (32) S. Shen, G. Jin, G. Ke, Y. Zhang, Aegan: adversarial eliminating with gan, arXiv preprint arXiv:1707.05474 2.
 (33) B. Zhou, P. Krähenbühl, Don’t let your discriminator be fooled, in: International Conference on Learning Representations, 2019.

(34)
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, A. Vladu, Towards deep learning models resistant to adversarial attacks, stat 1050 (2017) 9.
 (35) S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015) 448–456.

(36)
X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the thirteenth international conference on artificial intelligence and statistics, 2010, pp. 249–256.
 (37) D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980.
 (38) M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, G. Klambauer, S. Hochreiter, Gans trained by a two timescale update rule converge to a nash equilibrium (2017) 6626–6637.
 (39) T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, C. Xi, Improved techniques for training gans (2016) 2234–2242.
 (40) J. Wu, Z. Huang, J. Thoma, D. Acharya, L. Van Gool, Wasserstein divergence for gans (2018) 653–668.
Comments
There are no comments yet.