1. Related Work
1.1. Training neural language models
Authors in (Bengio et al., 2003a) were the first to define the training of a neural probabilistic language models using a softmax crossentropy training loss. Softmax being computationally slow, fast approximations were proposed to speed up the training. Sampled Softmax, introduced by authors in (Bengio et al., 2003b), showed both huge improvements in time complexity and stateoftheart results (Bengio and Senécal, 2008; Jean et al., 2014)
. Sampled softmax avoids computing scores for every word in the vocabulary as one chooses a proposal distribution from which it is cheap to sample, and performs a biased importance sampling approximation of softmax’s gradient. Noise Contrastive Estimation
(Gutmann and Hyvärinen, 2010)was also introduced as an unbiased estimator of the softmax and was shown to be efficient for learning word embeddings
(Mnih and Teh, 2012). On the contrary, the authors of (Mikolov et al., 2013b; Mikolov et al., 2013a) proposed a negative samplingloss whose goal is not to approximate softmax but to learn highquality vector representations for words. They consequently train a classifier that can discriminative between positive pairs coming from the true distribution and fake ones sampled from a negative sampling distribution.
The introduction of the previously mentioned negative sampling schemes, along with the Word2Vec architecture proposed by (Mikolov et al., 2013b; Mikolov et al., 2013a), has enabled stateoftheart results in terms of the quality of the learned word embeddings. Word2Vec has become a widely used architecture, having found use in applications such as document classification (Le and Mikolov, 2014), user modeling (Vasile et al., 2016), and graph embedding (Grover and Leskovec, 2016; Narayanan et al., 2016). Recent work has led to further improvements in words embedding models (Bojanowski et al., 2016; Pennington et al., 2014); a notable example is the FastText model (Joulin et al., 2016)
, which leverages ngram features to improve word embedding quality.
1.2. Negative sampling from a PU learning perspective
Positive and Unlabeled (PU) learning was originally defined in (Li and Liu, 2005). It has been shown that applying PU learning to language modeling and recommendation can lead to improved performances. Authors in (Liu et al., 2002) were the first to apply the framework of PU learning to the classification of text documents and showed significant uplifts in accuracy. For the training of word embeddings, authors of (Shazeer et al., 2016)
define an heuristic on all unobserved pairs and encourage the model to respect a pairwisedependent upperbound on its PMI estimate, making use of greater information. In the field of collaborative filtering, authors of
(Liang et al., 2016) incorporate a model of user exposure to items such that each unobserved pair is treated differently. Interestingly, negative sampling schemes introduced in the previous subsection can be seen as a way of moving the problem to the PU learning case. Contrary to conventional generic sampling losses such as Sampled Softmax or Noise Contrastive Estimation, we advocate that negative sampling schemes should not try to approximate the softmax function but rather act as a way to induce an informative bias on the model of negatives. To this end, authors in (Chen et al., 2018) provide an insightful analysis of negative sampling and show that negative samples that have high inner product scores with the context word provide more informative gradients. PU learning approaches have already shown some successes in the context of Recommender Systems as shown in (Liang et al., 2016). Also, as several previous works (Blei et al., 2003; Mikolov and Zweig, 2012; Yang et al., 2018) stressed that language modeling is highly contextdependent, we advocate in this paper for a contextbased negative sampling scheme.1.3. An active learning based negative sampling scheme
In the case of active learning, the model has accessed to a small labeled dataset and a bigger pool of unlabeled data. The goal then is to query the labels of the most valuable data points among this extensive unlabeled set. Therefore, using a sampling method, one can extend the training set with wisely chosen points that reduce the generalization error made by the model ((Roy and McCallum, 2001)
) or the variance over the dataset (
(Yu et al., 2006)). In active learning, a famous area of research is uncertainty sampling ((Huang et al., 2015; Tong and Koller, 2001)). In this setting, one queries the least certain instances to help the model refine the decision boundary. On the opposite, in maximum model change sampling, one is looking to sample points with highest norm gradients to increase the performance ((Bachman et al., 2017; Käding et al., 2016)). We refer the interested reader to (Yang and Loog, 2018) for a more exhaustive comparison of active learning approaches. In our case, we will never have access to the true label of the sampled points, but we argue that an ideal negative sampling should be a relative tradeoff between sampling data points close to the decision boundary and sampling very unlikely data.2. NextItem Prediction
We will start this section by formally defining the task. Then, we will present the Relaxed Softmax loss and will motivate the efficiency of this loss. Finally, we will discuss the properties of an ideal negative sampling distribution and show its efficiency on a toy dataset.
2.1. Framework
2.1.1. Notations
Let and denote two sets of objects. The set will be called the context set while the set will be denoted the target set. This setting can be applied to language processing, user activity modeling and graph theory (respectively next word, event and node prediction). and can also be identical.
In the oneclass setting applied to learning pairwise cooccurences, there exists an unknown generating process from which we can only observe samples gathered in a dataset . For each context item , denotes the conditional process on given the context item . Similarly, for a context , we have access to limited number of observations conditionally to and we note the conditional empirical distribution on . In the task of nextitem prediction, the goal is to learn the true conditional process and we will compare performances in Section 3 on both density estimation and ranking metrics.
denotes the family of functions, with parameters . The model takes an input in and outputs a score . In this paper, we use the same architecture as the Word2Vec model proposed by (Mikolov et al., 2013b; Mikolov et al., 2013a) but change both the training loss and the negative sampling distribution. This model is composed of an embedding matrix and an output layer . For a given pair context/target , we have , where corresponds to the th and th row of respectively and and refers to the dot product operator.
2.1.2. Maximizing Likelihood Estimation
Usually, one tries to solve the oneclass problem by performing density estimation of the empirical distribution with Maximum Likelihood Estimation (MLE). For a given context item
and any parametrized model
, one defines an estimated conditional distribution on the set of target items using the softmax function as follows :(1) 
Therefore, for a given pair , minimizing the negative loglikelihood leads to the following MLE objective:
(2) 
Doing so, one models the conditional process as a categorical distribution. From a density estimation point of view, minimizing Eq. 2
means minimizing the KullbackLeibler divergence between the empirical conditional distribution
and the modeled distribution . When deriving the gradient, one has :(3)  
(4) 
where refers to the empirical true distribution. Eq 4 assumes that all data points are i.i.d. sampled and that therefore the conditional generating process is a distribution on . As mentioned in the previous section, the Sampled Softmax loss tries approximating by performing a biased importance sampling with a cheaptosample proposal distribution.
In the next subsection, we stress the different drawbacks of optimizing MLE in the specific setting of learning from pair cooccurences in language modeling or recommendation.
2.2. Relaxing Softmax
2.2.1. Drawbacks of the MLE training scheme
First, when creating a pair dataset from a text corpora, one links a context word to several target items depending on the sliding window parameter and add the following pairs to the dataset. Conditionally to the context item , the target items
are therefore not i.i.d. sampled from a categorical distribution. The independence assumption made when performing Maximum Likelihood Estimation is therefore not valid. In language modeling and recommendation, this can be corrected with architectures taking into account the impact of long range dependencies, such as recurrent and attentionbased deep learning models
(Le Son et al., 2012; Trinh et al., 2018; Vaswani et al., 2017). In addition to this, as suggested by authors of (Blei et al., 2003; Yang et al., 2018)the assumption that target items are identically distributed conditionally to a context item might not be valid. However, correcting this drawback would require a different parametrization of the model and is not a problem tackled in this paper.Also, one of the drawbacks of the softmax crossentropy is that it does not leverage any potential negative pairs of items in . We advocate that one should move away from this method and bring the problem to PU learning. In this setting, one method is to make the assumption of possible negative data in the set of unlabeled items. One defines a negative distribution and then performs ordinary supervised classification (Li and Liu, 2005)
. Approximating softmax would mean sampling negative items with a uniform distribution and therefore making too simple assumptions on the set of negatives. Instead, we advocate that one should define a more complex sampling distribution that fits the negatives. Consequently, we propose a dynamical and contextual negative sampling scheme.
Finally, many tasks in language modeling and recommender systems do not strictly fall into density estimation but are rather ranking metrics. In language modeling, similarity and analogy metrics focus on the KNearestNeighbours in the embedding space. Besides, when tackling the task of nextitem prediction, many metrics such as Mean Percentile Rank or Precision (both metrics will be detailed later in Section 3) are ranking metrics and using a density estimator as MLE might not be an efficient tool.
2.2.2. The Relaxed Softmax loss
Modeling
as a categorical distribution, one assumes all target items to be conditionally dependent, and as discussed previously, this might lead to poor results. To correct this, we relax the constraint on the conditional probabilities by replacing the normalization constant used in the fullsoftmax formulation with one computed over a subset of chosen items. The chosen negatives are obtained with i.i.d. samples in a chosen
negative sampling distribution.Therefore, for a given context/target pair and a chosen negative sampling distribution , we define the following Relaxed Softmax (RL) loss with the following equation:
(5) 
where .
The Relaxed Softmax can be considered as a general framework as it is defined independently of both the negative sampling distribution and the number of negative samples . Using this training loss, one moves the problem to PU learning and forces the model to refine its decision boundary between the positive distribution and the chosen negative distribution. Indeed, the chosen Relaxed Softmax loss directly compares the scores of positive pairs coming from with negative pairs sampled from . At each step, this training loss will therefore penalize the target items sampled by the negative distribution. In the specific case where is the uniform distribution, the Relaxed Softmax is a MonteCarlo approximation of the softmax function. In the next subsection, we now analyze the gradient and consistency properties of this loss.
2.2.3. Gradient Analysis
The sampled negatives impact the performance of the results and play a significant role in the direction of the gradient. Deriving the loss with respect to , we have:
(6) 
where .
The gradient can be decomposed as two parts: the left maximizes likelihood of observing positive data whereas the right part penalizes pairs that were sampled under the negative distribution . We see that if a negative is too far away from the decision boundary and too easily classified by the model, the magnitude of the gradient w.r.t. this specific input will be small. In language modeling and recommendation, where there are significant differences among classes, sampling closer to the decision boundary of the model can help improving the model (Chen et al., 2018).
Interestingly, when the number of negatives sampled goes to , the gradient has the following formulation:
(7) 
where .
Comparing with MLE’s gradient from Eq. 3, the difference is that we replace with . The expectancy of the gradient is consequently not computed over the estimated density but rather a Boltzmann distribution reweighted by some negative distribution . It is therefore natural to approximate this gradient with a sampling scheme defined with a Boltzmann distribution. We present this new sampling in the next subsection.
2.3. A Boltzmann Negative Sampling distribution
Indeed, knowing the gradient when the number of samples goes to (Eq. 7), we now define a Boltzmann sampling distribution for the Relaxed Softmax loss: it is function of the energy state , a given degeneracy distribution and a temperature parameter , and has the following formulation:
(8) 
Under this formulation of the Relaxed Softmax, we notice that:

The RS loss generalizes Softmax: the softmax is a Boltzmann distribution with the specific parameters ( is the uniform distribution) and . When using and , the RS loss is the MonteCarlo approximation of the softmax crossentropy gradient. One can cover many other distributions, playing with both and .
Inspired from active learning and PU learning literature, we argue that the negative sampling scheme will hugely impact the performance of the model. Ideally, one would like to sample points that are misclassified by the model i.e. whose probability of being positive is overly estimated. Therefore, an optimal negative sampling distribution should take into account both how unlikely the data point is to be coming from the true generative process but also its estimated likelihood under our current model. This is exactly what is trying to approximate our proposed formulation: the energy state of each target item is reweighted by a distribution that should emphasize the negativity of each target. The temperature is designed to calibrate the sampling distribution.
2.3.1. Choosing the temperature parameter
In the proposed sampling scheme, the temperature parameter is capital as it controls the tradeoff between putting a stress on the energy state or the degeneracy distribution . From an optimization standpoint, with low temperatures, the Boltzmann formulation will put more mass on samples with high estimated scores and explore less. On the other hand, with high temperatures, this Boltzmann formulation might focus too much on data points far way from the decision boundary that do not convey meaningful information:

When , we have that where . In this case, the negative chosen is the data point with highest estimated probability. We expect this model to behave poorly as it will lack exploration.

When , we have that . In this case, we are guaranteed to sample very unlikely data that might not be insightful depending on the degeneracy distribution.
Unfortunately, we do not have any heuristics to find an optimal temperature parameter. It is clear that this ideal will be a function of both the degeneracy function used but also the task tackled. In the next subsection, we analyze the temperature’s impact on synthetic datasets on the density estimation task. Later on, in Section 3, we will see that for the same degeneracy distribution, optimizing for different tasks such as Mean Percentile Rank or test likelihood maximization leads to different ideal temperature parameters.
2.3.2. Example on a toy dataset
To better understand the behavior of our negative sampling distribution and the impact of the temperature parameter, we illustrate our approach on a synthetic dataset. We model a joint probability on the discrete set with a mixture of Gaussians with 50 components. In this example, we have . From this, we train our model
to learn the conditional probability distributions. To do a comparison of the influence of the degeneracy distribution on the performance of the model, we bench three different degeneracy distributions:
the uniform distribution on the set of target items, the popularity distribution on the set of target items, and finally, as we know in this synthetic dataset, we also bench . It is interesting to note that the last proposition is a contextbased distribution that concentrates relatively more on data unlikely to be positive.Figure 1 draws attention to examples of negatives sampled conditionally to a specific context item (in this example, ) for three different temperature parameters in the case where the degeneracy distribution is . Figure 0(a) stresses that with low temperatures, we are more likely to sample points with high estimated likelihoods. However, when the underlying generative process is easy to learn as it is the case in these synthetic datasets, the model ends up sampling positive data and misclassifies it as negative, deteriorating the performance. On the opposite, with high temperatures, one only samples very unlikely data points that do not bring meaningful information for the model as shown in Figure 0(c): theses points are far from the decision boundary and the model has already classified them as negatives. We see from Figure 0(b) the existence of an optimal temperature parameter, in this case , that achieves this interesting tradeoff between exploration and sampling close to the positives.
We now bench the temperature parameter with the three presented degeneracy distributions on the task of density estimation. Figure 2
reports the KullbackLeibler divergence between the true joint distribution
and an estimated one: where refers to the popularity distribution on the context item. Figure 2 highlights the existence of an optimal temperature parameter for the three different degeneracy distributions. Interestingly, we can notice that for negative distributions that concentrate more on unlikely data, the optimal temperature parameter is reached at lower values. Indeed, in these cases, one needs smaller values of to counter the effect of the degeneracy distribution. Finally, in the specific case when , Figure 1(a), the model reaches a significantly better optimum. The newly proposed Boltzmann formulation takes therefore advantage of a negative sampling distribution that concentrates more on very unlikely data.3. Experiments
We now evaluate the proposed negative sampling methods on the tasks of density estimation, word similarities and analogies, and nextitem prediction. All experiments below fall into the case of oneclass problem where data points are sampled from a true generative process only known in the synthetic case. Our proposed method moves the problem to PU learning by artificially sampling unlabeled data points. To aid the reproducibility, we will release the source code shortly. In the following experiments, we used the following baselines: Maximum Likelihood Estimation training (MLE^{1}^{1}1https://www.tensorflow.org/api_docs/python/tf/nn/softmax_cross_entropy_with_logits), Sampled Softmax with popularity sampling (SS^{2}^{2}2https://www.tensorflow.org/api_docs/python/tf/nn/sampled_softmax_loss). Binary CrossEntropy loss with popularity sampling (BCE^{3}^{3}3https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits) was not reported because, in our experiments, it lead to worse performances. We compared these baselines with the newly proposed Relaxed Softmax (RS) loss used with a Boltzmann Sampling.
3.1. The RS loss, a regularization scheme
To better understand the impact of the RS loss in the density estimation task, we first study the case of synthetic datasets. Similarly, to the toy dataset example in Section 2, we model the joint distribution on with a discretized mixture of Gaussians . Doing so, we can have access to the true distribution and also control the complexity of the underlying data. In Figure 3 and 4, we compare the performance of the standard MLE training with the newly defined RS loss when the negative distribution is and the number of negatives sampled is 5. To stress the fact that the RS loss acts as regularizer, we want to show it improves generalization properties of the model and therefore report both training and test performances. We focus on learning conditional distributions of the context items only seen during training and report and averaged over all . Figure 3 focuses on the influence of the batch size on the performances while Figure 4 highlights the impact of the training set size.
Interestingly, we can see in Figure 2(a) that MLE performs better on learning the empirical conditional distribution when dealing with large minibatches. This makes sense as fullbatch MLE is specifically minimizing the KL between the empirical distribution and the model . However, compared to the RS loss, we can see that the true conditional distribution is poorly fitted especially when the number of training samples is small, Figure 3(b). As the number of training points increases in the dataset, the performance of MLE training matches the performance of the RS loss in terms of generalization. We can therefore interpret the Relaxed Softmax loss as a regularization scheme that can be efficient with the right prior.
3.2. The nextitem prediction task
In practice, one does not have access to the true distribution. We want to see how our method performs on the task of nextitem prediction without using any prior knowledge of the true distribution. Therefore, the degeneracy distributions used are the uniform and the popularitybased distribution. For all of the following experiments, both on synthetic and realworld datasets, we compared six different training methods:

MLE (Eq. 2)

Sampled Softmax (SS) with popularity sampling with 50 negative samples to approximate Eq. 3.

RS (Eq. 5) with Uniform Sampling (US).

RS (Eq. 5) with Popularitybased Sampling (PS).

RS (Eq. 5) with Uniform Botlzmann Sampling (UBS), .

RS (Eq. 5) with Popularity Boltzmann sampling (PBS), .
When using the RS loss, across all experiments, we set the number of negative samples to 5 which was found to be optimal after grid search and cross validation. Besides, when using the Boltzmann sampling on realworld datasets, the temperature parameter was benched with the following grid: [0.5, 1, 3, 6, 12, 36].
3.2.1. Benchmarking on synthetic datasets
Similarly to the previous subsection, we model with a 2dimensional mixtures of Gaussians. Then, we generate three different datasets with respectively 10, 50 and 90 components to vary the complexity of the task. We bench the performance of the Boltzmann formulation on density estimation without any previous knowledge of the true distribution. On these synthetic datasets, the training set size was 300,000 data points and the batch size was 512 (as it was the optimal batch size for MLE for this training set size). We can see in Figure 5, that the Boltzmann formulation brings a significant uplift for both the uniform and popularity distributions on all three datasets. On the three different datasets, our RS loss with Boltzmann sampling beats the baselines.
3.2.2. Word Similarity and Analogy tasks
Method  MLE  SS  US  UBS  PS  PBS 

Similarities  
WS353  0.46  0.44  0.38  0.50  0.52  0.55 
SimLex999  0.13  0.26  0.25  0.25  0.21  0.25 
Rare Word  0.30  0.24  0.18  0.30  0.35  0.35 
Bruni men  0.42  0.38  0.33  0.49  0.52  0.52 
Mturk  0.47  0.40  0.34  0.61  0.53  0.64 
Semantic Analogies  
Precision@1  15.4  11.7  13.5  13.0  13.3  15.9 
Precision@5  29.5  24.8  21.9  26.6  25.0  24.5 
Precision@15  39.7  33.0  31.3  40.0  39.5  40.5 
Syntactic Analogies  
Precision@1  2.8  2.0  1.8  4.0  2.1  3.2 
Precision@5  13.2  12.7  11.2  16.8  14.8  16.1 
Precision@15  20.0  17.9  16.5  25.1  23.6  26.1 
Learning useful representations for words has been a largely studied field in Machine Learning and is of primal importance in both language modeling and recommendation where one is interested in computing similarities between products or movies. Authors in
(Mikolov et al., 2013b) proposed to test their word embeddings on both similarity and analogy tasks to measure the expresiveness of the learned embedding space. Both tasks have now largely been accepted by the Natural Language Processing community. To learn our word embeddings, we trained a Word2vec model on the text8 dataset, a textual data excerpt from Wikipedia^{4}^{4}4http://mattmahoney.net/dc/, and used the 15,000 mostcommon words for our experiments. On all experiments, we set the window size to 3. Then, we used different negative sampling distributions and benched them on both tasks of word similarities and analogies. All results are shown in Table 1. For the similarity task, we use five public datasets that list pairs of words and their similarity scores such as : WordSimilarity353 ^{5}^{5}5http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/, SimLex999 ^{6}^{6}6https://fh295.github.io/simlex.html, Rare Word ^{7}^{7}7https://nlp.stanford.edu/~lmthang/morphoNLM/, Men Test collection ^{8}^{8}8https://staff.fnwi.uva.nl/e.bruni/MEN and the MTurk dataset ^{9}^{9}9http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.205.8607&rep=rep1&type=pdf. We compute the Pearson correlation between these humanannotated scores and the similarity scores output by the model. Analogy tasks are also used to compare embedding structures. With the pairbased analogy task used here, given word and its analogy , one has to find, given a third word , its analogy . We consider two types of analogy tasks: semantic and syntactic analogies. We used the Google analogy test set ^{10}^{10}10http://download.tensorflow.org/data/questionswords.txt developed by (Mikolov et al., 2013b) for our experiments. Semantic analogies test the robustness of word embeddings to transformations such as "man" to "woman", while syntactic analogies focus on transformations such as "run" to "runs". We show quantitative results for the prec@1, prec@5 and prec@15 metrics in the Table 1.The Boltzmann formulation brings an uplift for both similarity and analogy task. Optimal temperatures are lower compared with those of the task of density estimation on synthetic datasets: in this case, when and when . In ranking tasks, one focuses on predicting correctly the top of the distribution, thus the need for harder negatives.
3.2.3. NextItem prediction
Method  MLE  SS  US  UBS   PS  PBS  

Likelihood  
Mvls (e4)  5.1  4.5  6.0  6.0   6.5  6.5  
Netflix (e4)  6.0  5.2  3.2  3.2   3.0  3.0  
Text8 (e2)  1.65  1.7  1.9  1.9   1.8  1.8  
MPR  
Mvls  91.7  91.5  89.0  92.5  6  92.2  92.1  6 
Netflix  87.3  86.7  87.8  88.7  12  87.6  89.0  12 
Text8  87.2  88.2  89.7  90.7  12  89.4  90.8  12 
Prec50  
Mvls  4.9  5.5  6.1  7.0  6  6.2  7.3  6 
Netflix  3.7  3.9  4.2  5.1  6  4.1  5.5  6 
Text8  31.9  31.2  31.5  34.2  6  31.5  34.5  6 
Prec15  
Mvls  1.3  1.7  2.3  2.6  3  2.2  3.0  3 
Netflix  1.2  1.4  1.2  1.8  3  1.3  1.5  3 
Text8  13.1  12.0  12.5  14.2  1  11.6  14.5  1 
Prec5  
Mvls  0.4  0.5  0.7  0.7  3  0.6  1.0  3 
Netflix  0.4  0.4  0.5  0.7  3  0.5  0.5  3 
Text8  4.5  4.2  4.4  5.1  1  4.1  4.6  1 
Likelihood, MPR (in %) and Precision (in %) on the nextitem prediction task. Confidence intervals are computed over 10 runs and results statistically better have been highlighted. On all ranking tasks (MPR and Prec@k), our Botlzmann formulation outperform the different baselines.
Finally, we want to compare the different methods on the task of the nextitem prediction. We therefore performed experiments on three different public datasets: two movie recommendation datasets listing movies seen by users (the Movielens and the Netflix datasets), and the same text dataset excerpt from Wikipedia. For the movie datasets, we kept only movies ranked over 4 stars for the Movielens dataset and over 5 stars for the Netflix dataset, resulting in 17,128 and 17,770 movies for each dataset, respectively. From the usertoitem cooccurrence matrix we create an itemtoitem cooccurrence matrix, where a pair is considered positive if both items are seen together in a user timeline. For the text dataset, we kept only the 15k most common words and considered a pair to be positive if both words cooccurred within a window of size 3. In order to learn relevant conditional probabilities, the sequential aspect of the data was kept and the cooccurence matrix was not made symmetrical. We use all of these datasets to perform an implicit matrix factorization, with access to only positive data.
We split each dataset so that 70% is used for training, 10% for validation, and 20% for testing. In Table 2, we report the following metrics on the nextitem prediction task:

Likelihood of pairs in the test set. Given the true generative process and the estimated output probability , likelihood is calculated as follows:
(9) 
Approximated Mean Percentile Rank: frequency with which positive target items are ranked over uniformly sampled negative target items .
(10) 
Precision@k : Frequency with which positive target items are ranked in the top k among all potential targets . We report Prec@5, Prec@15 and Prec@50.
The likelihood metric is a density estimation metric whereas Approximated MPR and Precision@k are ranking metrics. Interestingly, Precision@k only focuses on the top of the distribution disregarding how well can we rank the tail.
As we can see in Table 2, our proposed solution brings significant uplift from conventional methods when dealing with ranking metrics such as MPR and Prec@k. We can notice that when ranking metrics are focusing on the top of the distribution, as with Prec@5 and Prec@15 for example, the optimal temperatures get smaller. Indeed, optimality for these metrics is reached when negatives are sampled really close to the positive data. In these specific cases, it does not matter to sample data likely to be positive as the task really focuses on the top items. Contrary to previous generic sampling schemes, the advantage of the Botlzmann formulation is that we can now fit the negative sampling distribution to the given task.
Interestingly, on the task of density estimation on these realworld dataset, we can see that the Botlzmann formulation does not impact much the result. In this task, the optimal temperature parameter is for both degeneracy distributions. Table 2 stresses the importance of having a negative sampling distribution fit to the task. The Boltzmann formulation can prove itself efficient as it enables us to bench a variety of distributions.
Finally, we can notice from the results that the uplifts in performance are larger when it comes to textual data. We argue that compare to language modeling, sequences of movies convey less information. Additional works will analyze the impact of our proposed solution on the useritem prediction task.
4. Conclusions
In this paper, we proposed a Relaxed Softmax loss, a samplingbased loss that is more suited to the case of PU learning. With a newly defined Boltzmann Sampling, where one can fit the negative sampling distribution to a specific task, we have shown significant improvements over generic sampling methods on tasks such as density estimation, language modeling and nextitem prediction.
In future work, we will investigate the effectiveness of this training procedure on more complex models that can also be trained with negative sampling, such as CNN, RNN or Transformer like models (Vaswani et al., 2017). Models with more capacity than Word2Vec could benefit even more from using our dynamic contextbased sampling.
Building upon the work in (Blei et al., 2003; Yang et al., 2018) and assuming that language is multitopic, we would like to develop a new sampling scheme that also takes into account the topic of the target item.
Finally, we have shown the existence of an optimal temperature parameter for different degeneracy distributions and tasks. However, in all our experiments, this temperature was kept fixed for each run. Later research work should try and look for the advantages of an adaptive temperature during the training.
References
 (1)
 Bachman et al. (2017) Philip Bachman, Alessandro Sordoni, and Adam Trischler. 2017. Learning algorithms for active learning. In Proceedings of the 34th International Conference on Machine LearningVolume 70. JMLR. org, 301–310.
 Bengio et al. (2003a) Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003a. A neural probabilistic language model. Journal of machine learning research 3, Feb (2003), 1137–1155.
 Bengio and Senécal (2008) Yoshua Bengio and JeanSébastien Senécal. 2008. Adaptive importance sampling to accelerate training of a neural probabilistic language model. IEEE Transactions on Neural Networks 19, 4 (2008), 713–722.
 Bengio et al. (2003b) Yoshua Bengio, JeanSébastien Senécal, et al. 2003b. Quick Training of Probabilistic Neural Nets by Importance Sampling.. In AISTATS. 1–9.
 Blei et al. (2003) David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993–1022.
 Bojanowski et al. (2016) Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching Word Vectors with Subword Information. CoRR abs/1607.04606 (2016). arXiv:1607.04606 http://arxiv.org/abs/1607.04606
 Chen et al. (2018) Long Chen, Fajie Yuan, Joemon M Jose, and Weinan Zhang. 2018. Improving Negative Sampling for Word Representation using Selfembedded Features. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. ACM, 99–107.
 Elkan and Noto (2008) Charles Elkan and Keith Noto. 2008. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 213–220.
 Grover and Leskovec (2016) Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 855–864.
 Gutmann and Hyvärinen (2010) M. Gutmann and A. Hyvärinen. 2010. Noisecontrastive estimation: A new estimation principle for unnormalized statistical models. JMLR WCP, Vol. 9. Journal of Machine Learning Research  Proceedings Track, 297–304.
 Huang et al. (2015) TzuKuo Huang, Alekh Agarwal, Daniel J Hsu, John Langford, and Robert E Schapire. 2015. Efficient and parsimonious agnostic active learning. In Advances in Neural Information Processing Systems. 2755–2763.
 Jean et al. (2014) Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2014. On Using Very Large Target Vocabulary for Neural Machine Translation. CoRR abs/1412.2007 (2014). arXiv:1412.2007 http://arxiv.org/abs/1412.2007
 Joulin et al. (2016) Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of Tricks for Efficient Text Classification. CoRR abs/1607.01759 (2016). arXiv:1607.01759

Käding et al. (2016)
Christoph Käding,
Alexander Freytag, Erik Rodner,
Andrea Perino, and Joachim Denzler.
2016.
Largescale active learning with approximations of
expected model output changes. In
German Conference on Pattern Recognition
. Springer, 179–191.  Le and Mikolov (2014) Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International Conference on Machine Learning. 1188–1196.
 Le Son et al. (2012) Hai Le Son, Alexandre Allauzen, and François Yvon. 2012. Measuring the influence of long range dependencies with neural network language models. In Proceedings of the NAACLHLT 2012 Workshop: Will We Ever Really Replace the Ngram Model? On the Future of Language Modeling for HLT. Association for Computational Linguistics, 1–10.

Lee and Liu (2003)
Wee Sun Lee and Bing
Liu. 2003.
Learning with positive and unlabeled examples using weighted logistic regression. In
ICML, Vol. 3. 448–455.  Li and Liu (2003) Xiaoli Li and Bing Liu. 2003. Learning to classify texts using positive and unlabeled data. In IJCAI, Vol. 3. 587–592.
 Li and Liu (2005) XiaoLi Li and Bing Liu. 2005. Learning from positive and unlabeled examples with different data distributions. In European Conference on Machine Learning. Springer, 218–229.
 Liang et al. (2016) Dawen Liang, Laurent Charlin, James McInerney, and David M Blei. 2016. Modeling user exposure in recommendation. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 951–961.
 Liu et al. (2002) Bing Liu, Wee Sun Lee, Philip S Yu, and Xiaoli Li. 2002. Partially supervised classification of text documents. In ICML, Vol. 2. Citeseer, 387–394.
 Mikolov et al. (2013a) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).
 Mikolov et al. (2013b) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111–3119.

Mikolov and Zweig (2012)
Tomas Mikolov and
Geoffrey Zweig. 2012.
Context dependent recurrent neural network language model.
(2012).  Mnih and Teh (2012) Andriy Mnih and Yee Whye Teh. 2012. A fast and simple algorithm for training neural probabilistic language models. arXiv preprint arXiv:1206.6426 (2012).
 Narayanan et al. (2016) Annamalai Narayanan, Mahinthan Chandramohan, Lihui Chen, Yang Liu, and Santhoshkumar Saminathan. 2016. subgraph2vec: Learning distributed representations of rooted subgraphs from large graphs. arXiv preprint arXiv:1606.08928 (2016).
 Paquet and Koenigstein (2013) Ulrich Paquet and Noam Koenigstein. 2013. Oneclass collaborative filtering with random graphs. In Proceedings of the 22nd international conference on World Wide Web. ACM, 999–1008.
 Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP). 1532–1543. http://www.aclweb.org/anthology/D141162
 Ren et al. (2014) Yafeng Ren, Donghong Ji, and Hongbin Zhang. 2014. Positive Unlabeled Learning for Deceptive Reviews Detection. In EMNLP. 488–498.
 Roy and McCallum (2001) N Roy and A McCallum. 2001. Toward optimal active learning through sampling estimation of error reduction. Int. Conf. on Machine Learning.
 Shazeer et al. (2016) Noam Shazeer, Ryan Doherty, Colin Evans, and Chris Waterson. 2016. Swivel: Improving embeddings by noticing what’s missing. arXiv preprint arXiv:1602.02215 (2016).
 Tong and Koller (2001) Simon Tong and Daphne Koller. 2001. Support vector machine active learning with applications to text classification. Journal of machine learning research 2, Nov (2001), 45–66.
 Trinh et al. (2018) Trieu H Trinh, Andrew M Dai, MinhThang Luong, and Quoc V Le. 2018. Learning longerterm dependencies in rnns with auxiliary losses. arXiv preprint arXiv:1803.00144 (2018).
 Vasile et al. (2016) Flavian Vasile, Elena Smirnova, and Alexis Conneau. 2016. Metaprod2vec: Product embeddings using sideinformation for recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 225–232.
 Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998–6008.
 Yang and Loog (2018) Yazhou Yang and Marco Loog. 2018. A benchmark and comparison of active learning for logistic regression. Pattern Recognition 83 (2018), 401–415.
 Yang et al. (2018) Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W Cohen. 2018. Breaking the Softmax Bottleneck: A HighRank RNN Language Model. In International Conference on Learning Representations (ICLR).
 Yu et al. (2006) Kai Yu, Jinbo Bi, and Volker Tresp. 2006. Active learning via transductive experimental design. In Proceedings of the 23rd international conference on Machine learning. ACM, 1081–1088.
Comments
There are no comments yet.