The hallmark of learning new concepts from very few examples characterizes human intelligence. Though constantly pushing limits forward in various visual tasks, current deep learning approaches struggle in cases when abundant training data is impractical to gather. A straightforward idea to learn new concepts is to fine-tune a model pre-trained onbase categories, using limited data from another set of novel categories. However, this usually leads to catastrophic forgetting goodfellow2013empirical , i.e., fine-tuning makes the model over-fitting on novel classes, and agnostic to the majority of base classes kirkpatrick2017overcoming ; shmelkov2017incremental , deteriorating overall performance.
One way to address this problem is to augment data for novel classes. Since generating images could be both unnecessary xian2017feature and impractical salimans2016improved on large datasets, feature augmentation arandjelovic2012three ; chen2018person is more preferable in this scenario. Building upon learned representations vinyals2016matching ; snell2017prototypical ; sung2017learning , recently two variants of generative models show promising capability of learning variation modes from base classes to imagine the missing pattern of novel classes. Bharath et al. proposed Feature Hallucination (FH) hariharan2016low , which can learn a finite set of transformation mappings between examples in each base category and directly apply them to seed novel points for extra data. However, since mappings are enumerable (even in large amount), this model suffers from poor generalization. To address this issue, Wang et al. wang2018low proposed Feature Imagination (FI), a meta-learning based generation framework that can train an agent to synthesize extra data given a specific task. They circumvented the demand for latent distribution of novel classes by end-to-end optimization. But the generation results usually collapse into certain modes. Finally, it should be noted that both works erroneously assume that intra-class variances of base classes are sharable with any novel classes. For example, the visual variability of the concept lemon cannot be generalized to other irrelevant categories such as raccoon.
In this work, we propose a new approach to addressing the problem of low-shot learning by enabling better feature augmentation beyond current limits. Our approaches are novel in two aspects: modeling and training strategy. We propose Covariance-Preserving Adversarial Augmentation Networks (CP-AAN), a new class of Generative Adversarial Networks (GAN) goodfellow2014generative ; mirza2014conditional
for feature augmentation. We take inspiration from unpaired image-to-image translationzhu2017unpaired ; zhu2017toward and formulate our feature augmentation problem as an imbalanced set-to-set translation problem where the conditional distribution of examples of each novel class can be conceptually expressed as a mixture of related base classes. We first extract all related base-novel class pairs by an intuitive yet effective approach called Neighborhood Batch Sampling. Then, our model aims to learn the latent distribution of each novel class given its base counterparts. Since the direct estimation on novel classes can be inductively biased during this process, we explicitly preserve the covariance base examples during the generation process.
We systematically evaluate our approach by considering a series of objective functions. Our model achieves the state-of-the-art performance over the challenging ImageNet benchmark deng2009imagenet . With ablation studies, we also demonstrate the effectiveness of each component in our method.
2 Related Works
Low-shot Learning For quick adaptation when very few novel examples are available, the community has often used a meta-agent lei1996dynamic
to further tune base classifiersvinyals2016matching ; snell2017prototypical ; sung2017learning . Intuitive yet often ignored, feature augmentation was recently brought into the field by Bharath et al. hariharan2016low to ease the data scarce scenario. Compared to traditional meta-learning based approaches, they have reported noticeable improvement on not only the conventional setting (i.e., to test on novel examples only), but also the more challenging generalized setting (i.e., to test on all classes). Yet the drawback is that both the original work and its variants wang2018low fail to synthesize diverse examples because of ill-constrained generation processes. Our approach falls in this line of research while seeking for more principal guidance from base examples in a selective, class-specific manner.
Generative Adversarial Network for Set-to-set Translation GANs goodfellow2014generative map each latent code from an easily sampled prior to a realistic sample of a complex target distribution. Zhu et al. zhu2017unpaired have achieved astounding results on image-to-image translation without any paired training samples. In our case, diverse feature augmentation is feasible through conditional translation given a pair of related novel and base classes. Yet two main challenges remain: practically, not all examples are semantically translatable. Second, given extremely scarce data for novel classes, we are unable to estimate their latent distributions (see Figure 4). In this work, we thoroughly investigate conditional GAN variants inspired by previous works salimans2016improved ; mirza2014conditional ; zhu2017toward ; odena2016conditional to enable low-shot generation. Furthermore, we introduce a novel batch sampling technique for learning salient set-to-set mappings using unpaired data with categorical conditions.
Generation from Limited Observations Estimation of latent distribution from a handful of observations is biased and inaccurate tobin1958estimation ; candes2007dantzig . The Bayesian approaches aim to model latent distributions of a variety of classes as hierarchical Gaussian mixture salakhutdinov2012one , or alternatively model generation as a sequential decision making process rezende2016one . For GANs, Gaussian mixture noise has also been incorporated for latent code sampling gurumurthy2017deligan . Recent works mroueh2017mcgan ; mroueh2017fisher
on integral probability metrics provide theoretical guidance towards the high order feature matching. In this paper, building upon the assumption that related classes should have similar intra-class variance, we introduce a new loss term for preserving covariance during the translation process.
3 Imbalanced Set-to-set Translation
In this section, we formulate our low-shot feature augmentation problem under an imbalanced set-to-set translation framework. Concretely, we are given two labeled datasets represented in the same -dimensional semantic space: (1) a base set consisting of abundant samples and (2) a novel set with only a handful of observations. Their discrete label spaces are assumed to be non-overlapping, i.e., . Our goal is to learn a mapping function in order to translate examples of the base classes into novel categories. After the generation process, a final classifier is trained using both original examples of the base classes and all (mostly synthesized) examples of the novel classes.
Existing works hariharan2016low ; wang2018low suffer from the use of arbitrary, and thus possibly unrelated, base classes for feature augmentation. Moreover, their performances are degraded by naive generation methods without modeling the latent distribution of each novel class. Our insight, conversely, is to sample extra features from continuous latent distributions rather than certain modes from enumerations, by learning a GAN model (see Figure 2).
Specifically, we address two challenges that impede good translation under imbalanced scenarios: (1) through which base-novel class pairs we can translate; and more fundamentally, (2) through what objectives for GAN training we can estimate latent distribution of novel classes with limited observations. We here start by proposing a straightforward batch sampling technique to address the first problem. Then we suggest a simple extension of existing methods and study its weakness, which motivates the development of our final approach. For clarity, we introduce a toy dataset for imbalanced set-to-set translation in Figure 3 as a conceptual demonstration of the proposed method compared to baselines.
3.1 Neighborhood Batch Sampling
It is widely acknowledged goldberger2005neighbourhood ; vinyals2016matching ; snell2017prototypical that a metric-learned high dimensional space encodes relational semantics between examples. Therefore, to define which base classes are translatable to a novel class, we can rank them by their distance in a semantic space. For simplicity, we formulate our approach on top of Prototypical Networks snell2017prototypical , learned by a nearest neighbor classifier on the semantic space measured by the Euclidean distance. We represent each class as a cluster and encode its categorical information by the cluster prototype :
It should be noted that by “prototype” we mean the centroid of examples of a class. It should not be confused with the centroid of randomly sampled examples that is computed in each episode to train original Prototypical Networks.
We introduce translation mapping where is the powerset of the collection of all base classes. This defines a many-to-many relationship between novel and base classes, and is used to translate data from selected base classes to each novel class. To this end, given a novel class , we compute its similarity scores with all base classes using softmax over Euclidean distances between prototypes,
This results in a soft mapping (NBS-S) between base and novel classes, in which each novel class is paired with all base classes with soft scores. In practice, translating from all base classes is unnecessary, and computationally expensive. Alternatively, we consider a hard version of based on -nearest neighbor search, where the top base classes are selected and treated as equal (). This hard mapping (NBS-H) saves memory, but introduces an extra hyper-parameter.
3.2 Adversarial Objective
After constraining our translation process to selected class pairs, we develop a baseline based on Conditional GAN (c-GAN) mirza2014conditional . To this end, a discriminator is trained to classify real examples as the corresponding novel classes, and classify synthesized examples as an auxiliary “fake” class. salimans2016improved . The generator takes an example from base classes that are paired with via NBS, and aims to fool the discriminator into classifying the generated example as instead of the “fake”. More specifically, the adversarial objective can be written as:
where consists of all novel examples labeled with in while consists all base examples labeled by one of the classes in .
We train c-GAN by solving the minimax game of the adversarial loss. In this scenario, there is no explicit way to incorporate base classes intra-class variance into the generation of new novel examples. Also, any mappings that collapse synthesized features into existing observations yield the optimal solution goodfellow2014generative . These facts lead to unfavorable generation results as shown in Figure 2(b). We next explore different ways to explicitly force the generator to learn the latent conditional distributions.
3.3 Cycle-consistency Objective
A natural idea for preventing modes from getting dropped is to apply the cycle-consistency constraint whose effectiveness has been proven over image-to-image translation tasks zhu2017unpaired . Besides extra supervision, it eliminates the demand for paired data, which is impossible to acquire for the low-shot learning setting. We extend this method for our conditional scenario and derive cCyc-GAN. Specifically, we learn two generators: , which is our main target, and as an auxiliary mapping that reinforces . We train the generators such that, the translation cycle recovers the original embedding in either a forward cycle or a backward cycle . Our cycle-consistency objective could then be derived as,
-dimensional noise vector sampled from a distributionis injected into ’s input since novel examples lack variability given the very limited amount of data.
is a normal distributionfor our cCyc-GAN model.
While is hard to train due to the extremely small data volume; has more to learn from, and can thus indirectly guide through its gradient. During our experiments, we found that cycle-consistency is indispensable for stabilizing the training procedure. Swaminathan et al. gurumurthy2017deligan
observe that incorporating extra noise from a mixture of Gaussian distributions could result in more diverse results. Hence, we also report a variant calledcDeLi-GAN which uses the same objective as cCyc-GAN, but sample the noise vector from a mixture of different Gaussian distributions,
We follow the initialization setup in the previous work gurumurthy2017deligan . For each
, we sample from a uniform distribution. And for each , we first sample a vector from a Gaussian distribution , then we simply set
3.4 Covariance-preserving Objective
While cycle-consistency alone can transfer certain degrees of intra-class variance from base classes, we find it rather weak and unreliable since there are still infinite candidate distributions that cannot be discriminated based on limited observations (See Figure 4).
Building upon the assumption that similar classes share similar intra-class variance, one straightforward idea is to penalize the change of “variability” during translation. Hierarchical Bayesian models salakhutdinov2012one , prescribe each class as a multivariate Gaussian, where intra-class variability is embedded in a covariance matrix. We generalize this idea and try to maintain covariance in the translation process, although we model the class distribution by GAN instead of any prescribed distributions.
To compute the difference between two covariance matrices mroueh2017mcgan , one typical way is to measure the worst case distance between them using Ky Fan -norm, i.e.
, the sum of singular values of-truncated SVD, which we denote as . To this end, we define the pseudo-prototype of each novel class as the centroid of all synthetic samples translated from related base classes. The covariance distance between a base-novel class pair can then be formulated as,
Consequently, our covariance-preserving objective can be written as the expectation of the weighted covariance distance using NBS-S,
Note that, for a matrix , is non-differentiable with respect to itself, thus in practice, we calculate its subgradient instead. Specifically, we first compute the unitary matrices , by -truncated SVD xu1998truncated , and then back-propagate for sequential parameter updates. Proof of the correctness is provided in the supplementary material.
Finally, we propose our covariance-preserving conditional cycle-GAN, cCov-GAN, as:
As illustrated in Figure 2(e), preserving covariance information from relevant base classes to a novel class can improve low-shot generation quality. We attribute this empirical result to the interplay of adversarial learning, cycle consistency, and covariance preservation, that respectively lead to realistic generation, semantic consistency, and diversity.
Following recent works on meta-learning santoro2016one ; hariharan2016low ; wang2018low , we design a two-stage training procedure. During the “meta-training” phase, we train our generative model with base examples only, by mimicking the low-shot scenario it would encounter later. After that, in the “meta-testing” phase, we are given novel classes as well as their low-shot examples. We use the trained to augment each class until it has the average capacity of the base classes. Then we train a classifier as one would normally do in a supervised setting using both real and synthesized data. For the choice of this final classifier, we apply the same one as in the original representation learning stage. For examples, we use a nearest neighbor classifier for embeddings from Prototypical Networks, and a normal linear classifier for those from ResNets.
We follow the episodic procedure used by wang2018low during meta-training. In each episode, we sample “meta-novel” classes from , and use the rest of as “meta-base” classes. Then we sample examples from each meta-novel class as meta-novel examples. We compute prototypes of each class and similarity scores between each “meta-novel” and “meta-base” class. To sample a batch of size , we first include all “meta-novel” examples, and sample examples uniformly from the “meta-base” classes retrieved by translation mapping . Next, we push our samples through generators and discriminators to compute the loss. Finally, we update their weights for the current episode and start the next one.
This section is organized as follows. In Section 4.1, we conduct low-shot learning experiments on the challenging ImageNet benchmark. In Section 4.2, we further discuss with ablation, both quantitatively and qualitatively, to better understand the performance gain. We demonstrate our model’s capacity to generate diverse and reliable examples and its effectiveness in low-shot classification.
Dataset We evaluate our method on the real-world benchmark proposed by Bharath et al. hariharan2016low . This is a challenging task because it requires us to learn a large variety of ImageNet deng2009imagenet given a few exemplars for each novel classes. To this end, our model must be able to model the visual diversity of a wide range of categories and transfer knowledge between them without confusing unrelated classes. Following hariharan2016low , we split the 1000 ImageNet classes into four disjoint class sets , which consist of 193, 300, 196, 311 classes respectively. All of our parameter tuning is done on validation splits, while final results are reported using held-out test splits.
Evaluation We repeat sampling novel examples five times for held-out novel sets and report results of mean top-5 accuracy in both conventional low-shot learning (LSL, to test on novel classes only) and its generalized setting (GLSL, to test on all categories including base classes).
Baselines We compare our results to the exact numbers reported by Feature Hallucination hariharan2016low and Feature Imagination wang2018low . We also compared to other non-generative methods including classical Siamese Networks koch2015siamese , Prototypical Networks snell2017prototypical , Matching Networks vinyals2016matching , and MAML finn2017model as well as more recent Prototypical Matching Networks wang2018low and Attentive Weight Generators gidaris2018dynamic . For stricter comparison, we provide two extra baselines to exclude the bias induced by different embedding methods: P-FH builds on Feature Hallucinating by substituting their non-episodic representation with learned prototypical features. Another baseline (first row in Table 1), on the contrary, replaces prototypical features with raw ResNet-10 embeddings. The results for MAML and SN are reported using their published codebases online.
Our implementation is based on PyTorchpaszke2017automatic . Since deeper networks would unsurprisingly result in better performance, we confine all experiments in a ResNet-10 backbone111Released on https://github.com/facebookresearch/low-shot-shrink-hallucinate with a 512-d output layer. We fine-tune the backbone following the procedure described in hariharan2016low
. For all generators, we use three-layer MLPs with all hidden layers’ dimensions fixed at 512 as well as their output for synthesized features. Our discriminators are accordingly designed as three-layer MLPs to predict probabilities over target classes plus an extra fake category. We use leaky ReLU of slope 0.1 without batch normalization. Our GAN models are trained for 100000 episodes by ADAMkingma2014adam with initial learning rate fixed at 0.0001 which anneals by 0.5 every 20000 episodes. We fix the hyper-parameter for computing truncated SVD. For loss term contributions, we set and for all final objectives. We choose as the dimension of noise vectors for ’s input, and for the Gaussian mixture. We inject prototype embeddings instead of one-hot vectors as categorical information for all networks (prototypes for novel classes are computed using the low-shot examples only). We empirically set batch size , and and for all training, no matter what would the number of shots be in the test. This is more efficient but possibly less accurate than snell2017prototypical who trained separate models for each testing scenario, so the number of shots in train and test always match. All hyper-parameters are cross-validated on the validation set using a coarse grid search.
4.1 Main Results
|FH hariharan2016low||ResNet-10||LR w/ A.||40.7||50.8||62.0||69.3||76.4||52.2||59.7||68.6||73.3||76.9|
|P-FH||PN||LR w/ A.||41.5||52.2||63.5||71.8||76.4||53.6||61.7||69.0||73.5||75.9|
|FI wang2018low||PN||meta-learned LR||45.0||55.9||67.3||73.0||76.5||56.9||63.2||70.6||74.5||76.5|
LR w/ A.: Logistic Regressor with Analogies.
of all comparing methods under LSL and GLSL settings on ImageNet dataset. All results are averaged over five trials separately, and omit standard deviation for all numbers are of the order of. The best and second best methods under each setting are marked in according formats.
For comparisons, we include numbers reported in previous works under same experimental settings. Note that the results for MAML and SN are reported using their published codebases online. We decompose each method into stage-wise operations for breaking performance gain down to detailed choices made in each stage.
We provide four models constructed with different GAN choices as justified in Section 3. All of our introduced CP-AAN approaches are trained with NBS-S which would be further investigated with ablation in next subsection. Results are shown in Table 1. Our best method consistently achieves significant improvement over previous augmentation-based approaches for different values of under both LSL and GLSL settings, achieving almost 2 performance gain compared to baselines. We also notice that apart from overall improvement, our best model achieves largest boost (~) at the lowest shot over naive baseline and over Feature Imagination (FI) wang2018low under the LSL setting, even though we use a simpler embedding technique (PN compared to their PMN). We believe such performance gain can be attributed to our advanced generation methods since at low shots, FI applies discrete transformations that its generator has previously learned while we can now sample through a smooth distribution combining all related base classes’ covariance information.
Note that in the LSL setting, all generative methods assume we still have access to original base examples when learning final classifiers while non-generative baselines usually don’t have this constraint.
In this subsection, we carefully examine our design choices for the final version of our CP-AAN. We start by unpacking performance gain over the standard batch sampling procedure and proceed by showing both quantitative and qualitative evaluations on generation quality.
Ablation on NBS To validate the effectiveness of the NBS strategy over standard batch sampling for feature augmentation, we conduct an ablation study to show our absolute performance gain in Figure 4(a). In general, we empirically demonstrate that applying NBS improves the performance of low-shot recognition. We also show that the performance of NBS-H is sensitive to the hyper-parameter in the -nearest neighbor search. Therefore, soft assignment is preferable if computational resources allow.
Quantitative Generation Quality We next quantitatively evaluate the generation quality of the variants introduced in Section 3 and previous works as shown in Figure 4(b). Note that for FH, we used their published codebase online; for FI, we implemented the network and train with the procedure described in the original paper. We measure the diversity of generation via the mean average pairwise Euclidean distance of generated examples within each novel class. We adopt same augmentation strategies as used for ImageNet experiments. For reference, the mean average Euclidean distance over real examples is . In summary, the results are consistent with our expectation and support our design choices. Feature Hallucination and Imagination show less diversity compared to real data. Naive c-GAN even under-performs those baselines due to the mode collapse. Cycle-consistency and Gaussian mixture noise do help generation in both accuracy and diversity. However, they either under- or over-estimate the diversity. Our covariance-preserving objective leads to the best hallucination quality, since the generated distribution more closely resembles the real data diversity. Another insight from Figure 4(b) is that not surprisingly, under-estimating data diversity is more detrimental to classification accuracy than over-estimating.
Qualitative Generation Quality Figure 4(c), 4(d), 4(e) show t-SNE maaten2008visualizing visualizations of the data generated by Feature Hallucination, Feature Imagination and our best model in the prototypical feature space. We fix the number of examples per novel class in all cases and plot their real distribution with translucent point clouds. The 5 real examples are plotted in crosses and synthesized examples are denoted by stars. Evidently, naive generators could only synthesize novel examples that are largely pulled together. Although t-SNE might visually drag similar high dimensional points towards one mode, our model shows more diverse generation that is better aligned with the latent distribution, improving overall recognition performance by spreading seed examples in meaningful directions.
In this paper, we have presented a novel approach to low-shot learning that augments data for novel classes by training a cyclic GAN model, while shaping intra-class variability through similar base classes. We introduced and compared several GAN variants in a logical process and demonstrated the increasing performance of each model variant. Our proposed model significantly outperforms the state of the art on the challenging ImageNet benchmark in various settings. Quantitative and qualitative evaluations show the effectiveness of our method in generating realistic and diverse data for low-shot learning, given very few examples.
Acknowledgments This work was supported by the U.S. DARPA AIDA Program No. FA8750-18-2-0014. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.
-  Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013.
-  James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017.
-  Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. Incremental learning of object detectors without catastrophic forgetting. arXiv preprint arXiv:1708.06977, 2017.
-  Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. arXiv preprint arXiv:1712.00981, 2017.
-  Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pages 2234–2242, 2016.
-  Relja Arandjelović and Andrew Zisserman. Three things everyone should know to improve object retrieval. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2911–2918. IEEE, 2012.
-  Ying-Cong Chen, Xiatian Zhu, Wei-Shi Zheng, and Jian-Huang Lai. Person re-identification by camera correlation aware feature augmentation. IEEE transactions on pattern analysis and machine intelligence, 40(2):392–408, 2018.
-  Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, pages 3630–3638, 2016.
-  Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4080–4090, 2017.
-  Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. arXiv preprint arXiv:1711.06025, 2017.
-  Bharath Hariharan and Ross Girshick. Low-shot visual recognition by shrinking and hallucinating features. arXiv preprint arXiv:1606.02819, 2016.
-  Yu-Xiong Wang, Ross Girshick, Martial Hebert, and Bharath Hariharan. Low-shot learning from imaginary data. arXiv preprint arXiv:1801.05401, 2018.
-  Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image translation networks. arXiv preprint arXiv:1804.04732, 2018.
-  Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
-  Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
-  Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593, 2017.
-  Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems, pages 465–476, 2017.
-  Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248–255. IEEE, 2009.
-  David Lei, Michael A Hitt, and Richard Bettis. Dynamic core competences through meta-learning and strategic context. Journal of management, 22(4):549–569, 1996.
-  Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. arXiv preprint arXiv:1610.09585, 2016.
-  James Tobin. Estimation of relationships for limited dependent variables. Econometrica: journal of the Econometric Society, pages 24–36, 1958.
-  Emmanuel Candes, Terence Tao, et al. The dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics, 35(6):2313–2351, 2007.
Ruslan Salakhutdinov, Joshua Tenenbaum, and Antonio Torralba.
One-shot learning with a hierarchical nonparametric bayesian model.
Proceedings of ICML Workshop on Unsupervised and Transfer Learning, pages 195–206, 2012.
-  Danilo Jimenez Rezende, Shakir Mohamed, Ivo Danihelka, Karol Gregor, and Daan Wierstra. One-shot generalization in deep generative models. arXiv preprint arXiv:1603.05106, 2016.
-  Swaminathan Gurumurthy, Ravi Kiran Sarvadevabhatla, and V Babu Radhakrishnan. Deligan: Generative adversarial networks for diverse and limited data. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, 2017.
-  Youssef Mroueh, Tom Sercu, and Vaibhava Goel. Mcgan: Mean and covariance feature matching gan. arXiv preprint arXiv:1702.08398, 2017.
-  Youssef Mroueh and Tom Sercu. Fisher gan. In Advances in Neural Information Processing Systems, pages 2510–2520, 2017.
-  Jacob Goldberger, Geoffrey E Hinton, Sam T Roweis, and Ruslan R Salakhutdinov. Neighbourhood components analysis. In Advances in neural information processing systems, pages 513–520, 2005.
-  Peiliang Xu. Truncated svd methods for discrete linear ill-posed problems. Geophysical Journal International, 135(2):505–514, 1998.
-  Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. One-shot learning with memory-augmented neural networks. arXiv preprint arXiv:1605.06065, 2016.
-  Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2, 2015.
-  Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. arXiv preprint arXiv:1703.03400, 2017.
-  Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4367–4375, 2018.
-  Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
-  Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
-  Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
Laurens van der Maaten and Geoffrey Hinton.
Visualizing data using t-sne.
Journal of machine learning research, 9(Nov):2579–2605, 2008.