1 Introduction
Active learning (AL) is a machine learning paradigm for efficiently acquiring data for annotation from a (typically large) pool of unlabeled data
Lewis1994mj; Cohn:1996:ALS:1622737.1622744; settles2009active. Its goal is to concentrate the human labeling effort on the most informative data points that will benefit model performance the most and thus reducing data annotation cost.The most widely used approaches to acquiring data for AL are based on uncertainty and diversity, often described as the “two faces of AL” DASGUPTA20111767. While uncertaintybased methods leverage the model predictive confidence to select difficult examples for annotation Lewis:1994:SAT:188490.188495; Cohn:1996:ALS:1622737.1622744, diversity sampling exploits heterogeneity in the feature space by typically performing clustering Brinker03incorporatingdiversity; pmlrv16bodo11a. Still, both approaches have core limitations that may lead to acquiring redundant data points. Algorithms based on uncertainty may end up choosing uncertain yet uninformative repetitive data, while diversitybased methods may tend to select diverse yet easy examples for the model 10.5555/645530.655646. The two approaches are orthogonal to each other, since uncertainty sampling is usually based on the model’s output, while diversity exploits information from the input (i.e. feature) space. Hybrid data acquisition functions that combine uncertainty and diversity sampling have also been proposed shenetal2004multi; zhuetal2008active; Ducoffe2018sq; Ash2020Deep; yuanetal2020cold; ruetal2020active.
In this work, we aim to leverage characteristics from hybrid data acquisition. We hypothesize that data points that are close in the model feature space (i.e. share similar or related vocabulary, or similar model encodings) but the model produces different predictive likelihoods, should be good candidates for data acquisition. We define such examples as contrastive (see example in Figure 1). For that purpose, we propose a new acquisition function that searches for contrastive examples in the pool of unlabeled data. Specifically, our method, Contrastive Active Learning (Cal) selects unlabeled data points from the pool, whose predictive likelihoods diverge the most from their neighbors in the training set. This way, Cal shares similarities with diversity sampling, but instead of performing clustering it uses the feature space to create neighborhoods. Cal also leverages uncertainty, by using predictive likelihoods to rank the unlabeled data.
We evaluate our approach in seven datasets from four tasks including sentiment analysis, topic classification, natural language inference and paraphrase detection. We compare
Cal against a full suite of baseline acquisition functions that are based on uncertainty, diversity or both. We also examine robustness by evaluating on outofdomain data, apart from indomain heldout sets. Our contributions are the following:
We propose Cal, a new acquisition function for active learning that acquires contrastive examples from the pool of unlabeled data (§2);

We show that Cal performs consistently better or equal compared to all baselines in all tasks when evaluated on indomain and outofdomain settings (§4);

We conduct a thorough analysis of our method showing that Cal achieves a better tradeoff between diversity and uncertainty compared to the baselines (§6).
We release our code online ^{1}^{1}1https://github.com/mourga/contrastiveactivelearning.
2 Contrastive Active Learning
In this section we present in detail our proposed method, Cal: Contrastive Active Learning. First, we provide a definition for contrastive examples and how they are related to finding data points that are close to the decision boundary of the model (§2.1). We next describe an active learning loop using our proposed acquisition function (§2.2).
2.1 Contrastive Examples
In the context of active learning, we aim to formulate an acquisition function that selects contrastive examples from a pool of unlabeled data for annotation. We draw inspiration from the contrastive learning framework, that leverages the similarity between data points to push those from the same class closer together and examples from different classes further apart during training 10.5555/2999792.2999959; NIPS2016_6b180037; oord2019representation; pmlrv119chen20j; gunel2021supervised.
In this work, we define as contrastive examples two data points if their model encodings are similar, but their model predictions are very different (maximally disagreeing predictive likelihoods).
Formally, data points and should first satisfy a similarity criterion:
(1) 
where is an encoder that maps in a shared feature space, is a distance metric and is a small distance value.
A second criterion, based on model uncertainty, is to evaluate that the predictive probability distributions of the model
and for the inputs and should maximally diverge:(2) 
where KL is the KullbackLeibler divergence between two probability distributions
^{2}^{2}2KL divergence is not a symmetric metric, . We use as input the output probability distribution of an unlabeled example from the pool and as target the output probability distribution of an example from the train set (See §2.2 and algorithm 1)..For example, in a binary classification problem, given a reference example with output probability distribution () ^{3}^{3}3A predictive distribution () here denotes that the model is confident that belongs to the first class and to the second. and similar candidate examples with () and with (), we would consider as contrastive examples the pair . However, if another example (similar to in the model feature space) had a probability distribution (), then the most contrastive pair would be (, ).
Figure 1 provides an illustration of contrastive examples for a binary classification case. All data points inside the circle (dotted line) are similar in the model feature space, satisfying Eq. 1. Intuitively, if the divergence of the output probabilities of the model for the gray and blue shaded data points is high, then Eq. 2 should also hold and we should consider them as contrastive.
From a different perspective, data points with similar model encodings (Eq. 1) and dissimilar model outputs (Eq. 2), should be close to the model’s decision boundary (Figure 1). Hence, we hypothesize that our proposed approach to select contrastive examples is related to acquiring difficult examples near the decision boundary of the model. Under this formulation, Cal
does not guarantee that the contrastive examples lie near the model’s decision boundary, because our definition is not strict. In order to ensure that a pair of contrastive examples lie on the boundary, the second criterion should require that the model classifies the two examples in different classes (i.e. different predictions). However, calculating the distance between an example and the model decision boundary is intractable and approximations that use adversarial examples are computationally expensive
Ducoffe2018sq.2.2 Active Learning Loop
Assuming a multiclass classification problem with classes, labeled data for training and a pool of unlabeled data , we perform AL for iterations. At each iteration, we train a model on and then use our proposed acquisition function, Cal (Algorithm 1), to acquire a batch consisting of examples from . The acquired examples are then labeled^{4}^{4}4We simulate AL, so we already have the labels of the examples of (but still treat it as an unlabeled dataset)., they are removed from the pool and added to the labeled dataset , which will serve as the training set for training a model in the next AL iteration. In our experiments, we use a pretrained Bert model Devlin2019ou, which we finetune at each AL iteration using the current . We begin the AL loop by training a model using an initial labeled dataset ^{5}^{5}5We acquire the first examples that form the initial training set by applying random stratified sampling (i.e. keeping the initial label distribution)..
Find Nearest Neighbors for Unlabeled Candidates
The first step of our contrastive acquisition function (cf. line 2) is to find examples that are similar in the model feature space (Eq. 1). Specifically, we use the [CLS] token embedding of Bert as our encoder to represent all data points in and
. We use a KNearestNeighbors (KNN) implementation using the labeled data
, in order to query similar examples for each candidate . Our distance metric is Euclidean distance. To find the most similar data points in for each , we select the top instead of selecting a predefined threshold (Eq. 1) ^{6}^{6}6We leave further modifications of our scoring function as future work. One approach would be to add the average distance from the neighbors (cf. line 6) in order to alleviate the possible problem of selecting outliers.
. This way, we create a neighborhood that consists of the unlabeled data point and its closest examples in (Figure 1).Compute Contrastive Score between Unlabeled Candidates and Neighbors
In the second step, we compute the divergence in the model predictive probabilities for the members of the neighborhood (Eq. 2). Using the current trained model to obtain the output probabilities for all data points in (cf. lines 34), we then compute the Kullback–Leibler divergence (KL) between the output probabilities of and all (cf. line 5). To obtain a score for a candidate , we take the average of all divergence scores (cf. line 6).
Rank Unlabeled Candidates and Select Batch
We apply these steps to all candidate examples and obtain a score for each. With our scoring function we define as contrastive examples the unlabeled data that have the highest score . A high score indicates that the unlabeled data point has a high divergence in model predicted probabilities compared to its neighbors in the training set (Eq. 1, 2), suggesting that it may lie near the model’s decision boundary. To this end, our acquisition function selects the top examples from the pool that have the highest score (cf. line 8), that form the acquired batch .
3 Experimental Setup
3.1 Tasks & Datasets
We conduct experiments on sentiment analysis, topic classification, natural language inference and paraphrase detection tasks. We provide details for the datasets in Table 1. We follow yuanetal2020cold and use imdb maasetal2011learning, sst socheretal2013recursive, pubmed dernoncourtlee2017pubmed and agnews from NIPS2015_250cf8b5 where we also acquired dbpedia. We experiment with tasks requiring pairs of input sequences, using qqp and qnli from glue wang2018glue. To evaluate robustness on outofdistribution (OOD) data, we follow hendrycksetal2020pretrained and use sst as OOD dataset for imdb and vice versa. We finally use twitterppdb lanetal2017continuously as OOD data for qqp as in Desai2020ys.


dataset  task  domain  ood dataset  train  val  test  classes 
imdb  Sentiment Analysis  Movie Reviews  sst2  K  K  K  
sst2  Sentiment Analysis  Movie Reviews  imdb  6K  K  
agnews  Topic Classification  News    K  K  K  
dbpedia  Topic Classification  News    K  K  K  
pubmed  Topic Classification  Medical    K  K  K  
qnli  Natural Language Inference  Wikipedia    K  K  K  
qqp  Paraphrase Detection  Social QA Questions  twitterppdb  K  K  K  

3.2 Baselines
We compare Cal against five baseline acquisition functions. The first method, Entropy is the most commonly used uncertaintybased baseline that acquires data points for which the model has the highest predictive entropy. As a diversitybased baseline, following yuanetal2020cold, we use BertKM
that applies kmeans clustering using the
normalized Bert output embeddings of the finetuned model to select data points. We compare against Badge Ash2020Deep, an acquisition function that aims to combine diversity and uncertainty sampling, by computing gradient embeddings for every candidate data point in and then using clustering to select a batch. Each is computed as the gradient of the crossentropy loss with respect to the parameters of the model’s last layer, aiming to be the component that incorporates uncertainty in the acquisition function ^{7}^{7}7We note that BertKM and Badgeare computationally heavy approaches that require clustering of vectors with high dimensionality, while their complexity grows exponentially with the acquisition size. We thus do not apply them to the datasets that have a large
. More details can be found in the Appendix A.2. We also evaluate a recently introduced coldstart acquisition function called Alps yuanetal2020cold that uses the masked language model (MLM) loss of Bert as a proxy for model uncertainty in the downstream classification task. Specifically, aiming to leverage both uncertainty and diversity, Alps forms a surprisal embedding for each , by passing the unmasked input through the Bert MLM head to compute the crossentropy loss for a random 15% subsample of tokens against the target labels. Alps clusters these embeddings to sample sentences for each AL iteration. Lastly, we include Random, that samples data from the pool from a uniform distribution.
3.3 Implementation Details
We use BERTbase Devlin2019ou adding a taskspecific classification layer using the implementation from the HuggingFace library wolfetal2020transformers. We evaluate the model
times per epoch on the development set following
Dodge2020FineTuningPL and keep the one with the lowest validation loss. We use the standard splits provided for all datasets, if available, otherwise we randomly sample a validation set from the training set. We test all models on a heldout test set. We repeat all experiments with five different random seeds resulting into different initializations of the parameters of the model’s extra taskspecific output feedforward layer and the initial . For all datasets we use as budget the of , initial training set and acquisition size . Each experiment is run on a single Nvidia Tesla V100 GPU. More details are provided in the Appendix A.1.4 Results
4.1 Indomain Performance
We present results for indomain test accuracy across all datasets and acquisition functions in Figure 2. We observe that Cal is consistently the top performing method especially in dbpedia, pubmed and agnews datasets.
Cal performs slightly better than Entropy in imdb, qnli and qqp, while in sst most methods yield similar results. Entropy is the second best acquisition function overall, consistently performing better than diversitybased or hybrid baselines. This corroborates recent findings from Desai2020ys that Bert
is sufficiently calibrated (i.e. produces good uncertainty estimates), making it a tough baseline to beat in AL.
BertKM is a competitive baseline (e.g. sst, qnli) but always underperforms compared to Cal and Entropy
, suggesting that uncertainty is the most important signal in the data selection process. An interesting future direction would be to investigate in depth whether and which (i.e. which layer) representations of the current (pretrained language models) works best with similarity search algorithms and clustering.
Similarly, we can see that Badge, despite using both uncertainty and diversity, also achieves low performance, indicating that clustering the constructed gradient embeddings does not benefit data acquisition. Finally, we observe that Alps generally underperforms and is close to Random. We can conclude that this heterogeneous approach to uncertainty, i.e. using the pretrained language model as proxy for the downstream task, is beneficial only in the first few iterations, as shown in yuanetal2020cold.
Surprisingly, we observe that for the sst dataset Alps performs similarly with the highest performing acquisition functions, Cal and Entropy. We hypothesize that due to the informal textual style of the reviews of sst (noisy social media data), the pretrained Bert model can be used as a signal to query linguistically hard examples, that benefit the downstream sentiment analysis task. This is an interesting finding and a future research direction would be to investigate the correlation between the difficulty of an example in a downstream task with its perplexity (loss) of the pretrained language model.


train (id)  sst  imdb  qqp 
test (ood)  imdb  sst  twitterppdb 
Random  
BertKM    
Entropy  
Alps  
Badge    
Cal  

4.2 Outofdomain Performance
We also evaluate the outofdomain (OOD) robustness of the models trained with the actively acquired datasets of the last iteration (i.e. of or 100% of the AL budget) using different acquisition strategies. We present the OOD results for sst, imdb and qqp in Table 2. When we test the models trained with sst on imdb (first column) we observe that Cal achieves the highest performance compared to the other methods by a large margin, indicating that acquiring contrastive examples can improve OOD generalization. In the opposite scenario (second column), we find that the highest accuracy is obtained with Entropy. However, similarly to the ID results for sst (Figure 2), all models trained on different subsets of the imdb dataset result in comparable performance when tested on the small sst
test set (the mean accuracies lie inside the standard deviations across models). We hypothesize that this is because
sst is not a challenging OOD dataset for the different imdb models. This is also evident by the high OOD accuracy, 85% on average, which is close to the 91% sst ID accuracy of the full model (i.e. trained on 100% of the ID data). Finally, we observe that Cal obtains the highest OOD accuracy for qqp compared to Random, Entropy and Alps. Overall, our empirical results show that the models trained on the actively acquired dataset with Cal obtain consistently similar or better performance than all other approaches when tested on OOD data.5 Ablation Study
We conduct an extensive ablation study in order to provide insights for the behavior of every component of Cal. We present all AL experiments on the agnews dataset in Figure 3.
Decision Boundary
We first aim to evaluate our hypothesis that Cal acquires difficult examples that lie close to the model’s decision boundary. Specifically, to validate that the ranking of the constructed neighborhoods is meaningful, we run an experiment where we acquire candidate examples that have the minimum divergence from their neighbors opposite to Cal (i.e. we replace with in line 8 of Algorithm 1). We observe (Fig. 3  CAL opposite) that even after acquiring 15% of unlabeled data, the performance remains unchanged compared to the initial model (of the first iteration), even degrades. In effect, this finding denotes that Cal does select informative data points.
Neighborhood
Next, we experiment with changing the way we construct the neighborhoods, aiming to improve computational efficiency. We thus modify our algorithm to create a neighborhood for each labeled example (instead of unlabeled).^{8}^{8}8In this experiment, we essentially change the forloop of Algorithm 1 (cf. line 17) to iterate for each in (instead of each in ) and similarly find the nearest neighbors of each labeled example in the pool (KNN) As for the scoring (cf. line 6), if an unlabeled example was not picked (i.e. was not a neighbor to a labeled example), its score is zero. If it was picked multiple times we average its scores. We finally acquire the top unlabeled data with the highest scores. This formulation is more computationally efficient since usually .. This way we compute a divergence score only for the neighbors of the training data points. However, we find this approach to slightly underperform (Fig. 3  CAL per labeled example), possibly because only a small fraction of the pool is considered and thus the uncertainty of all the unlabeled data points is not taken into account.
Scoring function
We also experiment with several approaches for constructing our scoring function (cf. line 6 in Algorithm 1). Instead of computing the KL divergence between the predicted probabilities of each candidate example and its labeled neighbors (cf. line 5), we used cross entropy between the output probability distribution and the gold labels of the labeled data. The intuition is to evaluate whether information of the actual label is more useful than the model’s predictive probability distribution. We observe this scoring function to result in a slight drop in performance (Fig. 3  Cross Entropy). We also experimented with various pooling operations to aggregate the KL divergence scores for each candidate data point. We found maximum and median (Fig. 3  Max/Median) to perform similarly with the average (Fig. 3  Cal), which is the pooling operation we decided to keep in our proposed algorithm.
Feature Space
Since our approach is related to to acquiring data near the model’s decision boundary, this effectively translates into using the [CLS] output embedding of Bert. Still, we opted to cover several possible alternatives to the representations, i.e. feature space, that can be used to find the neighbors with KNN. We divide our exploration into two categories: intrinsic representations from the current finetuned model and extrinsic using different methods. For the first category, we examine representing each example with the mean embedding layer of Bert (Fig. 3  Mean embedding) or the mean output embedding (Fig. 3  Mean output). We find both alternatives to perform worse than using the [CLS] token (Fig. 3  Cal). The motivation for the second category is to evaluate whether acquiring contrastive examples in the input feature space, i.e. representing the raw text, is meaningful gardneretal2020evaluating ^{9}^{9}9This can be interpreted as comparing the effectiveness of selecting data near the model decision boundary vs. the task decision boundary, i.e. data that are similar for the task itself or for the humans (in terms of having the same raw input/vocabulary), but are from different classes.. We thus examine contextual representations from a pretrained Bert language model (Fig. 3  BERTpr [CLS]) (not finetuned in the task or domain) and noncontextualized tfidf vectors (Fig. 3  TFIDF). We find both approaches, along with Mean embedding, to largely underperform compared to our approach that acquires ambiguous data near the model decision boundary.
6 Analysis
Finally, we further investigate Cal and all acquisition functions considered (baselines), in terms of diversity, representativeness and uncertainty. Our aim is to provide insights on what data each method tends to select and what is the uncertaintydiversity tradeoff of each approach. Table 3 shows the results of our analysis averaged across datasets. We denote with the labeled set, the unlabeled pool and an acquired batch of data points from ^{10}^{10}10In the previous sections we used and to denote the labeled and unlabeled sets and we change the notation here to and , respectively, for simplicity..
6.1 Diversity & Uncertainty Metrics
Diversity in input space (Div.I)
We first evaluate the diversity of the actively acquired data in the input feature space, i.e. raw text, by measuring the overlap between tokens in the sampled sentences and tokens from the rest of the data pool . Following yuanetal2020cold, we compute Div.I as the Jaccard similarity between the set of tokens from the sampled sentences , , and the set of tokens from the unsampled sentences , , . A high Div.I value indicates high diversity because the sampled and unsampled sentences have many tokens in common.
Diversity in feature space (Div.F)
We next evaluate diversity in the (model) feature space, using the [CLS] representations of a trained Bert model ^{11}^{11}11To enable an appropriate comparison, this analysis is performed after the initial Bert model is trained with the initial training set and each AL strategy has selected examples equal to of the pool (first iteration). Correspondingly, all strategies select examples from the same unlabeled set while using outputs from the same Bert model.. Following Zhdanov2019mg and EinDor2020mm, we compute Div.F of a set as , where denotes the [CLS] output token of example obtained by the model which was trained using , and denotes the Euclidean distance between and in the feature space.
Uncertainty (Unc.)
To measure uncertainty, we use the model trained on the entire training dataset (Figure 2  Full supervision). As in yuanetal2020cold
, we use the logits from the fully trained model to estimate the uncertainty of an example, as it is a reliable estimate due to its high performance after training on many examples, while it offers a fair comparison across all acquisition strategies. First, we compute predictive entropy of an input
when evaluated by model and then we take the average over all sentences in a sampled batch . We use the average predictive entropy to estimate uncertainty of the acquired batch for each method . As a sampled batch we use the full actively acquired dataset after completing our AL iterations (with of the data).Representativeness (Repr.)
We finally analyze the representativeness of the acquired data as in EinDor2020mm. We aim to study whether AL strategies tend to select outlier examples that do not properly represent the overall data distribution. We rely on the KNNdensity measure proposed by zhuetal2008active, where the density of an example is quantified by one over the average distance between the example and its K most similar examples (i.e., K nearest neighbors) within , based on the [CLS] representations as in Div.F. An example with high density degree is less likely to be an outlier. We define the representativeness of a batch as one over the average KNNdensity of its instances using the Euclidean distance with K=.


Div.I  Div.F  Unc.  Repr.  
Random  0.766  0.356  0.132  1.848 
BertKM  0.717  0.363  0.145  2.062 
Entropy  0.754  0.323  0.240  2.442 
Alps  0.771  0.360  0.126  2.038 
Badge  0.655  0.339  0.123  2.013 
Cal  0.768  0.335  0.231  2.693 

6.2 Discussion
We first observe in Table 3 that Alps acquires the most diverse data across all approaches. This is intuitive since Alps is the most linguisticallyinformed method as it essentially acquires data that are difficult for the language modeling task, thus favoring data with a more diverse vocabulary. All other methods acquire similarly diverse data, except Badge that has the lowest score. Interestingly, we observe a different pattern when evaluating diversity in the model feature space (using the [CLS] representations). BertKM has the highest Div.F score, as expected, while Cal and Entropy have the lowest. This supports our hypothesis that uncertainty sampling tends to acquire uncertain but similar examples, while Cal by definition constrains its search in similar examples in the feature space that lie close to the decision boundary (contrastive examples). As for uncertainty, we observe that Entropy and Cal acquire the most uncertain examples, with average entropy almost twice as high as all other methods. Finally, regarding representativeness of the acquired batches, we see that Cal obtains the highest score, followed by Entropy, with the rest AL strategies to acquire less representative data.
Overall, our analysis validates assumptions on the properties of data expected to be selected by the various acquisition functions. Our findings show that diversity in the raw text does not necessarily correlate with diversity in the feature space. In other words, low Div.F does not translate to low diversity in the distribution of acquired tokens (Div.I), suggesting that Cal can acquire similar examples in the feature space that have sufficiently diverse inputs. Furthermore, combining the results of our AL experiments (Figure 2) and our analysis (Table 3) we conclude that the best performance of Cal, followed by Entropy, is due to acquiring uncertain data. We observe that the most notable difference, in terms of selected data, between the two approaches and the rest is uncertainty (Unc.), suggesting perhaps the superiority of uncertainty over diversity sampling. We show that Cal improves over Entropy because our algorithm “guides” the focus of uncertainty sampling by not considering redundant uncertain data that lie away from the decision boundary and thus improving representativeness. We finally find that Random is evidently the worst approach, as it selects the least diverse and uncertain data on average compared to all methods.
7 Related Work
Uncertainty Sampling
Uncertaintybased acquisition for AL focuses on selecting data points that the model predicts with low confidence. A simple uncertaintybased acquisition function is least confidence Lewis:1994:SAT:188490.188495 that sorts data in descending order from the pool by the probability of not predicting the most confident class. Another approach is to select samples that maximize the predictive entropy. Houlsby2011qz
propose Bayesian Active Learning by Disagreement (BALD), a method that chooses data points that maximize the mutual information between predictions and model’s posterior probabilities.
Gal2017gh applied BALD for deep neural models using Monte Carlo dropout Gal2016lf to acquire multiple uncertainty estimates for each candidate example. Least confidence, entropy and BALD acquisition functions have been applied in a variety of text classification and sequence labeling tasks, showing to substantially improve data efficiency Shen2017km; Siddhant2018lg; Lowell2019mf; Kirsch2019lk; shelmanovetal2021active; DBLP:journals/corr/abs210408320.Diversity Sampling
On the other hand, diversity or representative sampling is based on selecting batches of unlabeled examples that are representative of the unlabeled pool, based on the intuition that a representative set of examples once labeled, can act as a surrogate for the full data available. In the context of deep learning,
DBLP:journals/corr/abs171100941 and conf/iclr/SenerS18 select representative examples based on coreset construction, a fundamental problem in computational geometry. Inspired by generative adversarial learning, DBLP:journals/corr/abs190706347 define AL as a binary classification task with an adversarial classifier trained to not be able to discriminate data from the training set and the pool. Other approaches based on adversarial active learning, use outofthebox models to perform adversarial attacks on the training data, in order to approximate the distance from the decision boundary of the model Ducoffe2018sq; ruetal2020active.Hybrid
There are several existing approaches that combine representative and uncertainty sampling. Such approaches include active learning algorithms that use metalearning 10.5555/1005332.1005342; conf/aaai/HsuL15
fangetal2017learning; liuetal2018learningactively, aiming to learn a policy for switching between a diversitybased or an uncertaintybased criterion at each iteration. Recently, Ash2020Deep propose Batch Active learning by Diverse Gradient Embeddings (Badge) and yuanetal2020cold propose Active Learning by Processing Surprisal (Alps), a coldstart acquisition function specific for pretrained language models. Both methods construct representations for the unlabeled data based on uncertainty, and then use them for clustering; hence combining both uncertainty and diversity sampling. The effectiveness of AL in a variety of NLP tasks with pretrained language models, e.g. Bert Devlin2019ou, has empirically been recently evaluated by EinDor2020mm, showing substantial improvements over random sampling.8 Conclusion & Future Work
We present Cal, a novel acquisition function for AL that acquires contrastive examples; data points which are similar in the model feature space and yet the model outputs maximally different class probabilities. Our approach uses information from the feature space to create neighborhoods for each unlabeled example, and predictive likelihood for ranking the candidate examples. Empirical experiments on various indomain and outofdomain scenarios demonstrate that Cal performs better than other acquisition functions in the majority of cases. After analyzing the actively acquired datasets obtained with all methods considered, we conclude that entropy is the hardest baseline to beat, but our approach improves it by guiding uncertainty sampling in regions near the decision boundary with more informative data.
Still, our empirical results and analysis show that there is no single acquisition function to outperform all others consistently by a large margin. This demonstrates that there is still room for improvement in the AL field.
Furthermore, recent findings show that in specific tasks, as in Visual Question Answering (VQA), complex acquisition functions might not outperform random sampling because they tend to select collective outliers that hurt model performance karamchetietal2021mind. We believe that taking a step back and analyzing the behavior of standard acquisition functions, e.g. with Dataset Maps swayamdiptaetal2020dataset, might be beneficial. Especially, if similar behavior appears in other NLP tasks too.
Another interesting future direction for Cal, related to interpretability, would be to evaluate whether acquiring contrastive examples for the task Kaushik2020Learning; gardneretal2020evaluating is more beneficial than contrastive examples for the model, as we do in Cal.
Acknowledgments
KM and NA are supported by Amazon through the Alexa Fellowship scheme.
References
Appendix A Appendix
a.1 Data & Hyperparameters
In this section we provide details of all the datasets we used in this work and the hyperparparameters used for training the model. For qnli, imdb and sst we randomly sample 10% from the training set to serve as the validation set, while for agnews and qqp we sample 5%. For the dbpedia dataset we undersample both training and validation datasets (from the standard splits) to facilitate our AL simulation (i.e. the original dataset consists of 560K training and 28K validation data examples). For all datasets we use the standard test set, apart from sst, qnli and qqp datasets that are taken from the glue benchmark wang2018glue we use the development set as the heldout test set and subsample a development set from the training set.
For all datasets we train BERTbase Devlin2019ou from the HuggingFace library wolfetal2020transformers
in Pytorch
NEURIPS2019_9015. We train all models with batch size , learning rate , no weight decay, AdamW optimizer with epsilon . For all datasets we use maximum sequence length of , except for imdb that contain longer input texts, where we use . To ensure reproducibility and fair comparison between the various methods under evaluation, we run all experiments with the same five seeds that we randomly selected from the range . We evaluate the model 5 times per epoch on the development set following Dodge2020FineTuningPL and keep the one with the lowest validation loss. We use the code provided by yuanetal2020cold for Alps, Badge and BertKM.a.2 Efficiency
In this section we compare the efficiency of the acquisition functions considered in our experiments. We denote the number of labeled data in , the number of unlabeled data in , the number of classes in the downstream classification task, the dimension of embeddings, is fixed number of iterations for kMEANS, the maximum sequence length and the acquisition size. In our experiments, following yuanetal2020cold, , , , and ^{12}^{12}12Except for imdb where .. Alps requires considering that the surprisal embeddings are computed. BertKM and Badge, the most computationally heavy approaches, require and respectively, given that gradient embeddings are computed for Badge ^{13}^{13}13This information is taken from Section 6 of yuanetal2020cold.. On the other hand, Entropy only requires forward passes though the model, in order to obtain the logits for all the data in . Instead, our approach, Cal, first requires forward passes, in order to acquire the logits and the CLS representations of the the data (in and ) and then one iteration for all data in to obtain the scores.
We present the runtimes in detail for all datasets and acquisition functions in Tables 4 and 5. First, we define the total acquisition time as a sum of two types of times; inference and selection time. Inference time is the time that is required in order to pass all data from the model to acquire predictions or probability distributions or model encodings (representations). This is explicitly required for the uncertaintybased methods, like Entropy, and our method Cal. The remaining time is considered selection and essentially is the time for all necessary computations in order to rank and select the most important examples from .
We observe in Table 4 that the diversitybased functions do not require this explicit inference time, while for Entropy it is the only computation that is needed (taking the argmax of a list of uncertainty scores is negligible). Cal requires both inference and selection time. We can see that inference time of Cal is a bit higher than Entropy because we do forward passes instead of , that is equivalent to both and instead of only . The selection time for Cal is the forloop as presented in our Algorithm 1. We observe that it is often less computationally expensive than the inference step (which is a simple forward pass through the model). Still, there is room for improvement in order to reduce the time complexity of this step.
In Table 5 we present the total time for all datasets (ordered with increasing size) and the average time for each acquisition function, as a means to rank their efficiency. Because we do not apply all acquisition functions to all datasets we compute three different average scores in order to ensure fair comparison. avg.all is the average time across all datasets and is used to compare Random, Alps, Entropy and Cal. avg.3 is the average time across the first datasets (imdb, sst and dbpedia) and is used to compare all acquisition functions. Finally, avg.6 is the average time across all datasets apart from qqp and is used to compare Random, Alps, BertKM, Entropy and Cal.
We first observe that Entropy is overall the most efficient acquisition function. According to the avg.all column, we observe that Cal is the second most efficient function, followed by Alps. According to the avg.6 we observe the same pattern, with BertKM to be the slowest method. Finally, we compare all acquisition functions in the smallest (in terms of size of ) datasets and find that Entropy is the fastest method followed by Alps and Cal that require almost 3 times more computation time. The other clustering methods, BertKM and Badge, are significantly more computationally expensive, requiring respectively 13 and 100(!) times more time than Entropy.


dbpedia  imdb  sst  qnli  agnews  pubmed  qqp  
Random  
Alps  
BertKM  
Badge  
Entropy  
Cal  



dbpedia  imdb  sst  qnli  agnews  pubmed  qqp  avg.all  avg.3  avg.6  
Random  
Alps  
BertKM  
Badge  
Entropy  
Cal  

Interestingly, we observe the effect of the acquisition size ( of in our case) and the size of in the clustering methods. As these parameters increase, the computation of the corresponding acquisition function increases dramatically. For example, we observe that in the smallest datasets that Alps requires similar time to Cal. However, when we increase and (i.e. as we move from dbpedia with examples in to qnli with etc  see Table 1) we observe that the acquisition time of Alps becomes twice as much as that of Cal. For instance, in qqp with acquisition size we see that Alps requires seconds on average, while Cal . This shows that even though our approach is more computationally expensive as the size of increases, the complexity is linear, while for the other hybrid methods that use clustering, the complexity grows exponentially.
a.3 Reproducibility
All code for data preprocessing, model implementations, and active learning algorithms is made available at https://github.com/mourga/contrastiveactivelearning. For questions regarding the implementation, please contact the first author.