# SQL-Rank: A Listwise Approach to Collaborative Ranking

In this paper, we propose a listwise approach for constructing user-specific rankings in recommendation systems in a collaborative fashion. We contrast the listwise approach to previous pointwise and pairwise approaches, which are based on treating either each rating or each pairwise comparison as an independent instance respectively. By extending the work of (Cao et al. 2007), we cast listwise collaborative ranking as maximum likelihood under a permutation model which applies probability mass to permutations based on a low rank latent score matrix. We present a novel algorithm called SQL-Rank, which can accommodate ties and missing data and can run in linear time. We develop a theoretical framework for analyzing listwise ranking methods based on a novel representation theory for the permutation model. Applying this framework to collaborative ranking, we derive asymptotic statistical rates as the number of users and items grow together. We conclude by demonstrating that our SQL-Rank method often outperforms current state-of-the-art algorithms for implicit feedback such as Weighted-MF and BPR and achieve favorable results when compared to explicit feedback algorithms such as matrix factorization and collaborative ranking.

## Authors

• 11 publications
• 107 publications
• 19 publications
• ### SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback

The recent development of online recommender systems has a focus on coll...
02/23/2020 ∙ by Chao Wang, et al. ∙ 0

• ### Factorization Machines for Data with Implicit Feedback

In this work, we propose FM-Pair, an adaptation of Factorization Machine...
12/19/2018 ∙ by Babak Loni, et al. ∙ 0

• ### Permutation Models for Collaborative Ranking

We study the problem of collaborative filtering where ranking informatio...
07/23/2014 ∙ by Truyen Tran, et al. ∙ 0

• ### IteRank: An iterative network-oriented approach to neighbor-based collaborative ranking

Neighbor-based collaborative ranking (NCR) techniques follow three conse...
11/04/2018 ∙ by Bita Shams, et al. ∙ 0

• ### Efficient Rectangular Maximal-Volume Algorithm for Rating Elicitation in Collaborative Filtering

Cold start problem in Collaborative Filtering can be solved by asking ne...
10/16/2016 ∙ by Alexander Fonarev, et al. ∙ 0

• ### Scalable Personalised Item Ranking through Parametric Density Estimation

Learning from implicit feedback is challenging because of the difficult ...
05/11/2021 ∙ by Riku Togashi, et al. ∙ 0

• ### Revisit Recommender System in the Permutation Prospective

Recommender systems (RS) work effective at alleviating information overl...
02/24/2021 ∙ by Yufei Feng, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

We study a novel approach to collaborative ranking—the personalized ranking of items for users based on their observed preferences—through the use of listwise losses, which are dependent only on the observed rankings of items by users. We propose the SQL-Rank algorithm, which can handle ties and missingness, incorporate both explicit ratings and more implicit feedback, provides personalized rankings, and is based on the relative rankings of items. To better understand the proposed contributions, let us begin with a brief history of the topic.

### 1.1 A brief history of collaborative ranking

Recommendation systems, found in many modern web applications, movie streaming services, and social media, rank new items for users and are judged based on user engagement (implicit feedback) and ratings (explicit feedback) of the recommended items. A high-quality recommendation system must understand the popularity of an item and infer a user’s specific preferences with limited data. Collaborative filtering, introduced in (Hill et al., 1995), refers to the use of an entire community’s preferences to better predict the preferences of an individual (see (Schafer et al., 2007) for an overview). In systems where users provide ratings of items, collaborative filtering can be approached as a point-wise prediction task, in which we attempt to predict the unobserved ratings (Pan et al., 2017)

. Low rank methods, in which the rating distribution is parametrized by a low rank matrix (meaning that there are a few latent factors) provides a powerful framework for estimating ratings

(Mnih & Salakhutdinov, 2008; Koren, 2008). There are several issues with this approach. One issue is that the feedback may not be representative of the unobserved entries due to a sampling bias, an effect that is prevalent when the items are only ‘liked’ or the feedback is implicit because it is inferred from user engagement. Augmenting techniques like weighting were introduced to the matrix factorization objective to overcome this problem (Hsieh et al., 2015; Hu et al., 2008). Many other techniques are also introduced (Kabbur et al., 2013; Wang et al., 2017; Wu et al., 2016). Another methodology worth noting is the CofiRank algorithm of (Weimer et al., 2008) which minimizes a convex surrogate of the normalized discounted cumulative gain (NDCG). The pointwise framework has other flaws, chief among them is that in recommendation systems we are not interested in predicting ratings or engagement, but rather we must rank the items.

Ranking is an inherently relative exercise. Because users have different standards for ratings, it is often desirable for ranking algorithms to rely only on relative rankings and not absolute ratings. A ranking loss is one that only considers a user’s relative preferences between items, and ignores the absolute value of the ratings entirely, thus deviating from the pointwise framework. Ranking losses can be characterized as pairwise and listwise. A pairwise method decomposes the objective into pairs of items for a user , and effectively asks ‘did we successfully predict the comparison between and for user ?’. The comparison is a binary response—user liked more than or less than

—with possible missing values in the event of ties or unobserved preferences. Because the pairwise model has cast the problem in the classification framework, then tools like support vector machines were used to learn rankings;

(Joachims, 2002) introduces rankSVM and efficient solvers can be found in (Chapelle & Keerthi, 2010). Much of the existing literature focuses on learning a single ranking for all users, which we will call simple ranking (Freund et al., 2003; Agarwal, 2006; Pahikkala et al., 2009). This work will focus on the personalized ranking setting, in which the ranking is dependent on the user.

Pairwise methods for personalized ranking have seen great advances in recent years, with the AltSVM algorithm of (Park et al., 2015), Bayesian personalized ranking (BPR) of (Rendle et al., 2009), and the near linear-time algorithm of (Wu et al., 2017). Nevertheless, pairwise algorithms implicitly assume that the item comparisons are independent, because the objective can be decomposed where each comparison has equal weight. Listwise losses instead assign a loss, via a generative model, to the entire observed ranking, which can be thought of as a permutation of the items, instead of each comparison independently. The listwise permutation model, introduced in (Cao et al., 2007), can be thought of as a weighted urn model, where items correspond to balls in an urn and they are sequentially plucked from the urn with probability proportional to where is the latent score for user and item and is some non-negative function. They proposed to learn rankings by optimizing a cross entropy between the probability of

items being at the top of the ranking and the observed ranking, which they combine with a neural network, resulting in the ListNet algorithm.

(Shi et al., 2010) applies this idea to collaborative ranking, but uses only the top-1 probability because of the computational complexity of using top-k in this setting. This was extended in (Huang et al., 2015) to incorporate neighborhood information. (Xia et al., 2008) instead proposes a maximum likelihood framework that uses the permutation probability directly, which enjoyed some empirical success.

Very little is understood about the theoretical performance of listwise methods. (Cao et al., 2007) demonstrates that the listwise loss has some basic desirable properties such as monotonicity, i.e. increasing the score of an item will tend to make it more highly ranked. (Lan et al., 2009) studies the generalizability of several listwise losses, using the local Rademacher complexity, and found that the excess risk could be bounded by a term (recall, is the number of users). Two main issues with this work are that no dependence on the number of items is given—it seems these results do not hold when

is increasing—and the scores are not personalized to specific users, meaning that they assume that each user is an independent and identically distributed observation. A simple open problem is: can we consistently learn preferences from a single user’s data if we are given item features and we assume a simple parametric model? (

.)

### 1.2 Contributions of this work

We can summarize the shortcomings of the existing work: current listwise methods for collaborative ranking rely on the top- loss, algorithms involving the full permutation probability are computationally expensive, little is known about the theoretical performance of listwise methods, and few frameworks are flexible enough to handle explicit and implicit data with ties and missingness. This paper addresses each of these in turn by proposing and analyzing the SQL-rank algorithm.

• We propose the SQL-Rank method, which is motivated by the permutation probability, and has advantages over the previous listwise method using cross entropy loss.

• We provide an

linear algorithm based on stochastic gradient descent, where

is the set of observed ratings and is the rank.

• The methodology can incorporate both implicit and explicit feedback, and can gracefully handle ties and missing data.

• We provide a theoretical framework for analyzing listwise methods, and apply this to the simple ranking and personalized ranking settings, highlighting the dependence on the number of users and items.

## 2 Methodology

### 2.1 Permutation probability

The permutation probability, (Cao et al., 2007), is a generative model for the ranking parametrized by latent scores. First assume there exists a ranking function that assigns scores to all the items. Let’s say we have items, then the scores assigned can be represented as a vector . Denote a particular permutation (or ordering) of the items as

, which is a random variable and takes values from the set of all possible permutations

(the symmetric group on elements). denotes the index of highest ranked item and is the lowest ranked. The probability of obtaining is defined to be

 Ps(π):=m∏j=1ϕ(sπj)∑ml=jϕ(sπl), (1)

where is an increasing and strictly positive function. An interpretation of this model is that each item is drawn without replacement with probability proportional to for item in each step. One can easily show that

is a valid probability distribution, i.e.

. Furthermore, this definition of permutation probability enjoys several favorable properties (see (Cao et al., 2007)). For any permutation if you swap two elements ranked at generating the permutation (, , ), if then . Also, if permutation satisfies , , then we have . Both of these properties can be summarized: larger scores will tend to be ranked more highly than lower scores. These properties are required for the negative log-likelihood to be considered sound for ranking (Xia et al., 2008).

In recommendation systems, the top ranked items can be more impactful for the performance. In order to focus on the top ranked items, we can compute the partial-ranking marginal probability,

 P(k,¯m)s(π)=min{k,¯m}∏j=1ϕ(sπj)∑¯ml=jϕ(sπl). (2)

It is a common occurrence that only a proportion of the items are ranked, and in that case we will allow to be the number of observed rankings (we assume that are the complete list of ranked items). When , the first summation vanishes and top- probability can be calculated straightforwardly, which is why is widely used in previous listwise approaches for collaborative ranking. Counter-intuitively, we demonstrate that using a larger tends to improve the ranking performance.

We see that computing the likelihood loss is linear in the number of ranked items, which is in contrast to the cross-entropy loss used in (Cao et al., 2007), which takes exponential time in . The cross-entropy loss is also not sound, i.e. it can rank worse scoring permutations more highly, but the negative log-likelihood is sound. We will discuss how we can deal with ties in the following subsection, namely, when the ranking is derived from ratings and multiple items receive the same rating, then there is ambiguity as to the order of the tied items. This is a common occurrence when the data is implicit, namely the output is whether the user engaged with the item or not, yet did not provide explicit feedback. Because the output is binary, the cross-entropy loss (which is based on top- probability with very small) will perform very poorly because there will be many ties for the top ranked items. To this end, we propose a collaborative ranking algorithm using the listwise likelihood that can accommodate ties and missingness, which we call Stochastic Queuing Listwise Ranking, or SQL-Rank.

### 2.2 Deriving objective function for SQL-Rank

The goal of collaborative ranking is to predict a personalized score that reflects the preference level of user towards item , where and . It is reasonable to assume the matrix to be low rank because there are only a small number of latent factors contributing to users’ preferences. The input data is given in the form of “user gives item a relevance score ”. Note that for simplicity we assume all the users have the same number of ratings, but this can be easily generalized to the non-uniform case by replacing with (number of ratings for user ).

With our scores and our ratings , we can specify our collaborative ranking model using the permutation probability (2). Let be a ranking permutation of items for user (extracted from ), we can stack , row by row, to get the permutation matrix . Assuming users are independent with each other, the probability of observing a particular given the scoring matrix can be written as

 P(k,¯m)X(Π)=n∏i=1P(k,¯m)Xi(Πi). (3)

We will assume that

is the sigmoid function. This has the advantage of bounding the resulting weights,

Typical rating data will contain many ties within each row. In such cases, the permutation is no longer unique and there is a set of permutations that coincides with rating because with any candidate we can arbitrarily shuffle the ordering of items with the same relevance scores to generate a new candidate matrix which is still valid (see Figure 1). We denote the set of valid permutations as , where is the set of all pairs such that is observed. We call this shuffling process the Stochastic Queuing Process, since one can imagine that by permuting ties we are stochastically queuing new ’s for future use in the algorithm.

The probability of observing therefore should be defined as . To learn the scoring matrix , we can naturally solve the following maximum likelihood estimator with low-rank constraint:

 minX∈X−log∑Π∈S(R,Ω)P(k,¯m)X(Π), (4)

where is the structural constraint of the scoring matrix. To enforce low-rankness, we use the nuclear norm regularization .

Eq (4) is hard to optimize since there is a summation inside the . But by Jensen’s inequality and convexity of function, we can move the summation outside and obtain an upper bound of the original negative log-likelihood, leading to the following optimization problem:

 minX∈X−∑Π∈S(R,Ω)logP(k,¯m)X(Π) (5)

This upper bound is much easier to optimize and can be solved using Stochastic Gradient Descent (SGD).

Next we discuss how to apply our model for explicit and implicit feedback settings. In the explicit feedback setting, it is assumed that the matrix is partially observed and the observed entries are explicit ratings in a range (e.g., to ). We will show in the experiments that (using the full list) leads to the best results. (Huang et al., 2015) also observed that increasing is useful for their cross-entropy loss, but they were not able to increase since their model has time complexity exponential to .

In the implicit feedback setting each element of is either or , where means positive actions (e.g., click or like) and means no action is observed. Directly solving (5) will be expensive since and the computation will involve all the elements at each iteration. Moreover, the ’s in the matrix could mean either a lower relevance score or missing, thus should contribute less to the objective function. Therefore, we adopt the idea of negative sampling (Mikolov et al., 2013) in our list-wise formulation. For each user (row of ), assume there are ’s, we then sample unobserved entries uniformly from the same row and append to the back of the list. This then becomes the problem with and then we use the same algorithm in explicit feedback setting to conduct updates. We then repeat the sampling process at the end of each iteration, so the update will be based on different set of ’s at each time.

### 2.3 Non-convex implementation

Despite the advantage of the objective function in equation (5) being convex, it is still not feasible for large-scale problems since the scoring matrix leads to high computational and memory cost. We follow a common trick to transform (5) to the non-convex form by replacing : with so that the objective is,

 ∑Π∈S(R,Ω)−n∑i=1¯m∑j=1logϕ(uTivΠij)∑¯ml=jϕ(uTivΠil)f(U,V)+λ2(∥U∥2F+∥V∥2F),

where are columns of respectively. We apply stochastic gradient descent to solve this problem. At each step, we choose a permutation matrix using the stochastic queuing process (Algorithm 2) and then update by . For example, the gradient with respect to is ( is the sigmoid function),

 ∂f∂vj =∑i∈Ωj¯m∑t=1{−g′(uTivt)ui +1(ranki(j)≥t)ϕ(uTivj)¯m∑l=tϕ(uTivΠil)g′(uTivj)ui}

where denotes the set of users that have rated the item and is a function gives the rank of the item for that user . Because is the sigmoid function, . The gradient with respect to can be derived similarly.

As one can see, a naive way to compute the gradient of requires time, which is very slow even for one iteration. However, we show in Algorithm 3 (in the appendix) that there is a smart way to re-arranging the computation so that can be computed in time, which makes our SQL-Rank a linear-time algorithm (with the same per-iteration complexity as classical matrix factorization).

## 3 Theory

Throughout this section, we will establish a theoretical framework for understanding listwise ranking algorithms. We do not consider ties and missing data and reserve this extension of the theory developed here for future work. These tools can be employed to analyze any problem of the constrained form

 ^X:=argmin−logPX(Π) such that X∈X. (6)

We will consider two main settings of listwise ranking, the simple ranking setting where for each ,

 Xij=β⊤zij,β∈Rs,∥β∥≤cb, (7)

where the feature vectors are known, and the personalized setting,

 Xij=u⊤ivj,ui,vj∈Rr,∥U∥F≤cu,∥V∥F≤cv. (8)

The simple ranking setting, among other listwise programs was considered in (Lan et al., 2009), and it was determined that the excess risk is bounded by a term. Critically, these results assumed that the number of items is bounded, severely limiting their relevance to realistic recommendation systems. It seems that we should be able to learn something about a user’s preferences by having them rank more items, yet the existing theory does not reflect this.

The main engine of our theoretical analysis is a generative mechanism for listwise ranking, which demonstrates that the permutation probability model, (1), is also the probability of a row-wise ordering of an exponential ensemble matrix. We demonstrate that the excess risk in the parametric setting scales like , achieving parametric rates in and sub-linear excess risk in when the feature dimension is fixed. In the personalized setting, (8), we bound the excess risk by when the rank is fixed, which matches comparable results for matrix factorization up to log factors.

### 3.1 Generative mechanism for listwise ranking

We give an alternative generative mechanism which will prove useful for understanding the listwise ranking objective.

###### Theorem 1.

Consider a matrix, , with independent entries,

that are drawn from an exponential distribution with rate

. Let be the ordering of the entries of from smallest to largest, then the probability of is exactly .

The proof is in the appendix. A key aspect of this generative mechanism is that the listwise likelihood can be written as a function of the exponential ensemble. This allows us to establish concentration of measure results for the listwise loss via bounded differences.

### 3.2 Statistical guarantees

As a first step to controlling the excess risk, we establish a basic inequality. This bounds the excess risk by an empirical process term, which is a random function of and for a fixed it has mean zero. The excess risk (the difference in expected loss between the estimate and the truth) can also be written as the KL divergence between the estimated model and the true model.

###### Lemma 1.

Consider the minimizer, , to the constrained optimization, (6). Suppose that there exists a such that independently for all . The KL-divergence between the estimate and the truth is bounded

 D(X⋆,^X):=1nn∑i=1ElogPX⋆i(Πi)P^Xi(Πi) (basic) ≤−1nn∑i=1(logPX⋆i(Πi)P^Xi(Πi)−ElogPX⋆i(Πi)P^Xi(Πi)).

Because the RHS of (basic), the empirical process term, has mean zero and is a function of the random permutation, we can use Theorem 1 to bound it with high probability for a fixed . Because is random, we need to control the empirical process term uniformly over the selection of . To this end, we employ Dudley’s chaining, which gives us the following theorem (see the Supplement for the complete proof).

###### Theorem 2.

Assume the conditions of Lemma 1. Define the matrix norm, for the matrix ,

 ∥Z∥∞,2:= ⎷n∑i=1∥Zi∥2∞

and define where is applied elementwise. Also, let be the -covering number of in the norm (the fewest number of radius balls in the norm required to cover ). Then, if (where is the elementwise absolute maximum), then

 D(X⋆,^X)=OP(√mln(m)n⋅g(Z)),

where

 g(Z):=∫∞0√lnN(u,Z,∥.∥∞,2)du,

and is bounded by a constant in .

Theorem 2 bounds the KL-divergence by the geometric quantity . For the derived corollaries, we will assume that is 1-Lipschitz, which is true when is the sigmoid function. The results do not significantly change by increasing the Lipschitz constant, so for simplicity of presentation we set it to be 1.

###### Corollary 1.

Assume the conditions to Lemma 1, the simple ranking setting (7), that is 1-Lipschitz, and is bounded uniformly, then

 D(X⋆,^X)=OP(√smnlnm).

Notably when this bound is on the order of . In the event that is concentrated primarily on a single permutation for this user, and we resort to random guessing (i.e. ) then the KL divergence will be close to . So, a reduction of the KL-divergence from order to is a large improvement, and the above result should be understood to mean that we can achieve consistency even when (where consistency is measured relative to random guessing).

###### Corollary 2.

Assume the conditions to Lemma 1, the personalized ranking setting, (8), and that is 1-Lipschitz,

 D(X⋆,^X)=OP(√rmnlnm).

Notably, even in the personalized setting, where each user has their own preferences, we can achieve rates for fixed . Throughout these results the notation only hides the constants , and any dependence on is explicitly given. While Theorem 2 gives us a generic result that can be applied to a large range of constraint sets, we believe that the parametric simple ranking and the low-rank personalized setting are the two most important listwise ranking problems.

## 4 Experiments

In this section, we compare our proposed algorithm (SQL-Rank) with other state-of-the-art algorithms on real world datasets. Note that our algorithm works for both implicit feedback and explicit feedback settings. In the implicit feedback setting, all the ratings are or ; in the explicit feedback setting, explicit ratings (e.g., to ) are given but only to a subset of user-item pairs. Since many real world recommendation systems follow the implicit feedback setting (e.g., purchases, clicks, or checkins), we will first compare SQL-Rank on implicit feedback datasets and show it outperforms state-of-the-art algorithms. Then we will verify that our algorithm also performs well on explicit feedback problems. All experiments are conducted on a server with an Intel Xeon E5-2640 2.40GHz CPU and 64G RAM.

### 4.1 Implicit Feedback

In the implicit feedback setting we compare the following methods:

• SQL-Rank: our proposed algorithm implemented in Julia .

• Weighted-MF: the weighted matrix factorization algorithm by putting different weights on and ’s (Hu et al., 2008; Hsieh et al., 2015).

• BPR: the Bayesian personalized ranking method motivated by MLE (Rendle et al., 2009). For both Weighted-MF and BPR, we use the C++ code by Quora .

Note that other collaborative ranking methods such as Pirmal-CR++ (Wu et al., 2017) and List-MF (Shi et al., 2010) do not work for implicit feedback data, and we will compare with them later in the explicit feedback experiments. For the performance metric, we use precision@ for defined by

 precision@k=∑ni=1|{1≤l≤k:RiΠil=1}|n⋅k, (9)

where is the rating matrix and gives the index of the -th ranked item for user among all the items not rated by user in the training set.

We use rank and tune regularization parameters for all three algorithms using a random sampled validation set. For Weighted-MF, we also tune the confidence weights on unobserved data. For BPR and SQL-Rank, we fix the ratio of subsampled unobserved ’s versus observed ’s to be , which gives the best performance for both BPR and SQL-rank in practice.

We experiment on the following four datasets. Note that the original data of Movielens1m, Amazon and Yahoo-music are ratings from to , so we follow the procedure in (Rendle et al., 2009; Yu et al., 2017) to preprocess the data. We transform ratings of into ’s and the rest entries (with rating and unknown) as ’s. Also, we remove users with very few ’s in the corresponding row to make sure there are enough ’s for both training and testing. For Amazon, Yahoo-music and Foursquare, we discard users with less than ratings and randomly select ’s as training and use the rest as testing. Movielens1m has more ratings than others, so we keep users with more than ratings, and randomly sample 50 of them as training.

• Movielens1m: a popular movie recommendation data with users and items.

• Amazon: the Amazon purchase rating data for musical instruments with users and items.

• Yahoo-music: the Yahoo music rating data set which contains users and items.

• Foursquare: a location check-in data. The data set contains users and venues with check-ins. The data set is already in the form of “0/1” so we do not need to do any transformation.

The experimental results are shown in Table 1. We find that SQL-Rank outperforms both Weighted-MF and BPR in most cases.

### 4.2 Explicit Feedback

Next we compare the following methods in the explicit feedback setting:

• SQL-Rank: our proposed algorithm implemented in Julia. Note that in the explicit feedback setting our algorithm only considers pairs with explicit ratings.

• List-MF: the listwise algorithm using the cross entropy loss between observed rating and top probability (Shi et al., 2010). We use the C++ implementation on github.

• MF: the classical matrix factorization algorithm in (Koren, 2008) utilizing a pointwise loss solved by SGD. We implemented SGD in Julia.

• Primal-CR++: the recently proposed pairwise algorithm in (Wu et al., 2017). We use the Julia implementation released by the authors.

Experiments are conducted on Movielens1m and Yahoo-music datasets. We perform the same procedure as in implicit feedback setting except that we do not need to mask the ratings into “0/1”.

We measure the performance in the following two ways:

• NDCG: defined as:

 NDCG@k=1nn∑i=1DCG@k(i,Πi)% DCG@k(i,Π∗i),

where represents -th user and

 DCG@k(i,Πi)=k∑l=12RiΠil−1log2(l+1).

In the DCG definition, represents the index of the -th ranked item for user in test data based on the learned score matrix . is the rating matrix and is the rating given to item by user . is the ordering provided by the ground truth rating.

• Precision@: defined as a fraction of relevant items among the top recommended items:

 precision@k=∑ni=1|{1≤l≤k:4≤RiΠil≤5}|n⋅k,

here we consider items with ratings assigned as or as relevant. follows the same definitions above but unlike before gives the index of the -th ranked item for user among all the items that are not rated by user in the training set (including both rated test items and unobserved items).

As shown in Table 2, our proposed listwise algorithm SQL-Rank outperforms previous listwise method List-MF in both NDCG@ and precision@. It verifies the claim that log-likelihood loss outperforms the cross entropy loss if we use it correctly. When listwise algorithm SQL-Rank is compared with pairwise algorithm Primal-CR++, the performances between SQL-Rank and Primal-CR++ are quite similar, slightly lower for NDCG@ but higher for precision@. Pointwise method MF is doing okay in NDCG but really bad in terms of precision. Despite having comparable NDCG, the predicted top items given by MF are quite different from those given by other algorithms utilizing a ranking loss. The ordered lists based on SQL-Rank, Primal-CR++ and List-MF, on the other hand, share a lot of similarity and only have minor difference in ranking of some items. It is an interesting phenomenon that we think is worth exploring further in the future.

### 4.3 Training speed

To illustrate the training speed of our algorithm, we plot precision@ versus training time for the Movielen1m dataset and the Foursquare dataset. Figure 2 and Figure 3 (in the appendix) show that our algorithm SQL-Rank is faster than BPR and Weighted-MF. Note that our algorithm is implemented in Julia while BPR and Weighted-MF are highly-optimized C++ codes (usually at least 2 times faster than Julia) released by Quora. This speed difference makes sense as our algorithm takes time, which is linearly to the observed ratings. In comparison, pair-wise model such as BPR has pairs, so will take

time for each epoch.

### 4.4 Effectiveness of Stochastic Queuing (SQ)

One important innovation in our SQL-Rank algorithm is the Stochastic Queuing (SQ) Process for handling ties. To illustrate the effectiveness of the SQ process, we compare our algorithm with and without SQ. Recall that without SQ means we fix a certain permutation matrix and optimize with respect to it throughout all iterations without generating new , while SQ allows us to update using a new permutation at each time. As shown Table 3 and Figure 4 (in the appendix), the performance gain from SQ in terms of precision is substantial (more than ) on Movielen1m dataset. It verifies the claim that our way of handling ties and missing data is very effective and improves the ranking results by a lot.

### 4.5 Effectiveness of using the Full List

Another benefit of our algorithm is that we are able to minimize top probability with much larger and without much overhead. Previous approaches (Huang et al., 2015) already pointed out increasing leads to better ranking results, but their complexity is exponential to so they were not able to have . To show the effectiveness of using permutation probability for full lists rather than using the top probability for top partial lists in the likelihood loss, we fix everything else to be the same and only vary in Equation (5). We obtain the results in Table 4 and Figure 5 (in the appendix). It shows that the larger we use, the better the results we can get. Therefore, in the final model, we set to be the maximum number (length of the observed list.)

## 5 Conclusions

In this paper, we propose a listwise approach for collaborative ranking and provide an efficient algorithm to solve it. Our methodology can incorporate both implicit and explicit feedback, and can gracefully handle ties and missing data. In experiments, we demonstrate our algorithm outperforms existing state-of-the art methods in terms of top recommendation precision. We also provide a theoretical framework for analyzing listwise methods highlighting the dependence on the number of users and items.

## Acknowledgements

JS is partially supported by NSF DMS-1712996. CJH acknowledge the support by NSF IIS-1719097, Google Cloud and Nvidia.

Side Note by Liwei Wu: SQL in SQL-Rank is not only the abbreviation for Stochastically Queuing Listwise, but also name initials of Liwei’s girlfriend ShuQing Li. Special thanks for her support.

## References

• Agarwal (2006) Agarwal, S. Ranking on graph data. In

Proceedings of the 23rd international conference on Machine learning

, pp. 25–32. ACM, 2006.
• Cao et al. (2007) Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., and Li, H. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th international conference on Machine learning, pp. 129–136. ACM, 2007.
• Chapelle & Keerthi (2010) Chapelle, O. and Keerthi, S. S. Efficient algorithms for ranking with svms. Information Retrieval, 13(3):201–215, 2010.
• Freund et al. (2003) Freund, Y., Iyer, R., Schapire, R. E., and Singer, Y. An efficient boosting algorithm for combining preferences. Journal of machine learning research, 4(Nov):933–969, 2003.
• Hill et al. (1995) Hill, W., Stead, L., Rosenstein, M., and Furnas, G. Recommending and evaluating choices in a virtual community of use. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 194–201. ACM Press/Addison-Wesley Publishing Co., 1995.
• Hsieh et al. (2015) Hsieh, C.-J., Natarajan, N., and Dhillon, I. Pu learning for matrix completion. In International Conference on Machine Learning, pp. 2445–2453, 2015.
• Hu et al. (2008) Hu, Y., Koren, Y., and Volinsky, C. Collaborative filtering for implicit feedback datasets. In Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on, pp. 263–272. Ieee, 2008.
• Huang et al. (2015) Huang, S., Wang, S., Liu, T.-Y., Ma, J., Chen, Z., and Veijalainen, J. Listwise collaborative filtering. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 343–352. ACM, 2015.
• Joachims (2002) Joachims, T. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 133–142. ACM, 2002.
• Kabbur et al. (2013) Kabbur, S., Ning, X., and Karypis, G. Fism: factored item similarity models for top-n recommender systems. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 659–667. ACM, 2013.
• Koren (2008) Koren, Y. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 426–434. ACM, 2008.
• Lan et al. (2009) Lan, Y., Liu, T.-Y., Ma, Z., and Li, H. Generalization analysis of listwise learning-to-rank algorithms. In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 577–584. ACM, 2009.
• Mikolov et al. (2013) Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pp. 3111–3119, 2013.
• Mnih & Salakhutdinov (2008) Mnih, A. and Salakhutdinov, R. R. Probabilistic matrix factorization. In Advances in neural information processing systems, pp. 1257–1264, 2008.
• Pahikkala et al. (2009) Pahikkala, T., Tsivtsivadze, E., Airola, A., Järvinen, J., and Boberg, J. An efficient algorithm for learning to rank from preference graphs. Machine Learning, 75(1):129–165, 2009.
• Pan et al. (2017) Pan, W., Yang, Q., Duan, Y., Tan, B., and Ming, Z. Transfer learning for behavior ranking. ACM Transactions on Intelligent Systems and Technology (TIST), 8(5):65, 2017.
• Park et al. (2015) Park, D., Neeman, J., Zhang, J., Sanghavi, S., and Dhillon, I. Preference completion: Large-scale collaborative ranking from pairwise comparisons. In International Conference on Machine Learning, pp. 1907–1916, 2015.
• Rendle et al. (2009) Rendle, S., Freudenthaler, C., Gantner, Z., and Schmidt-Thieme, L. Bpr: Bayesian personalized ranking from implicit feedback. In

Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence

, pp. 452–461. AUAI Press, 2009.
• Schafer et al. (2007) Schafer, J. B., Frankowski, D., Herlocker, J., and Sen, S. Collaborative filtering recommender systems. In The adaptive web, pp. 291–324. Springer, 2007.
• Shi et al. (2010) Shi, Y., Larson, M., and Hanjalic, A. List-wise learning to rank with matrix factorization for collaborative filtering. In Proceedings of the fourth ACM conference on Recommender systems, pp. 269–272. ACM, 2010.
• Talagrand (2006) Talagrand, M. The generic chaining: upper and lower bounds of stochastic processes. Springer Science & Business Media, 2006.
• Wang et al. (2017) Wang, J., Yu, L., Zhang, W., Gong, Y., Xu, Y., Wang, B., Zhang, P., and Zhang, D. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 515–524. ACM, 2017.
• Weimer et al. (2008) Weimer, M., Karatzoglou, A., Le, Q. V., and Smola, A. J. Cofi rank-maximum margin matrix factorization for collaborative ranking. In Advances in neural information processing systems, pp. 1593–1600, 2008.
• Wu et al. (2017) Wu, L., Hsieh, C.-J., and Sharpnack, J. Large-scale collaborative ranking in near-linear time. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 515–524. ACM, 2017.
• Wu et al. (2016) Wu, Y., DuBois, C., Zheng, A. X., and Ester, M. Collaborative denoising auto-encoders for top-n recommender systems. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pp. 153–162. ACM, 2016.
• Xia et al. (2008) Xia, F., Liu, T.-Y., Wang, J., Zhang, W., and Li, H. Listwise approach to learning to rank: theory and algorithm. In Proceedings of the 25th international conference on Machine learning, pp. 1192–1199. ACM, 2008.
• Yu et al. (2017) Yu, H.-F., Bilenko, M., and Lin, C.-J. Selection of negative samples for one-class matrix factorization. In Proceedings of the 2017 SIAM International Conference on Data Mining, pp. 363–371. SIAM, 2017.

## Appendix A Supplement to “A Listwise Approach to Collaborative Ranking”

### a.1 Proofs in Theory section

###### Proof of Theorem 1..

Notice that is the argument, , that minimizes , and . Furthermore, is exponential with rate parameter and is independent of . Hence,

 P{Yik≤min{Yij}j≠k} =∫∞0ϕ(Xik)e−uϕ(Xik)e−∑j≠kuϕ(Xij)du =ϕ(Xik)∑jϕ(Xij).

Furthermore,

 P{Πi|Πi1}=P{YΠi2≤…≤YΠim|YΠij≥YΠi1,∀j} =P{YΠi2−YΠi1≤…≤YΠim−YΠi1|YΠij≥YΠi1,∀j}.

By the memorylessness property, we have that the joint distribution of

, is equivalent to the distribution of . Hence, we can apply induction with the previous argument, and the tower property of conditional probability. ∎

###### Proof of Lemma 1..

By optimality,

 1nn∑i=1−logP^Xi(Πi)≤1nn∑i=1−logPX⋆i(Πi).

Which is equivalent to

 1nn∑i=1−logPX⋆i(Πi)P^Xi(Πi)≥0.

Thus, we can subtract the expectation,

 1nn∑i=1ElogPX⋆i(Πi)P^Xi(Πi) ≤−1nn∑i=1(logPX⋆i(Πi)P^Xi(Πi)−ElogPX⋆i(Πi)P^Xi(Πi))

where the expectation is with respect to the draw of conditional on . ∎

###### Lemma 2.

Let be a permutation vector and be a score vector each of length . Suppose that for all

. Define the relative loss function,

 Lx,x′(π):=logPx(π)Px′(π).

Consider translating an item in position to position in the permutation , thus forming where . Specifically, if or ; if then for ; if then for . The relative loss function has bounded differences in the sense that

 |Lx,x′(π)−Lx,x′(π′)|≤C0∥logϕ(x)−logϕ(x′)∥∞,

where is applied elementwise and .

###### Proof.

Suppose that , and define the following shorthand,

 λj=ϕ(xπj),Λj=m∑k=jλk,

and let be similarly defined with . Then by replacing the permutation with causes the to be replaced with for . Hence,

 logQx(π)−logQx(π′) =ℓ′∑j=ℓ+1log(Λj−λj+λℓ)−log(Λj) =ℓ′∑j=ℓ+1log(Λj−1+λℓ)−log(Λj) =ℓ′−1∑j=ℓlog(1+λℓΛj)+logΛℓ−logΛℓ′.

So we can bound the difference,

 |Lx,x′(π)−Lx,x′(π′)|≤∣∣∣logΛℓΛ′ℓ−logΛℓ′Λ′ℓ′∣∣∣ +ℓ′−1∑j=ℓ∣∣ ∣∣log(1+λℓΛj)−log(1+λ′ℓΛ′j)∣∣ ∣∣.

Suppose that for each , and that . Then we have that

 ∣∣∣logΛℓΛ′ℓ∣∣∣≤maxj≥ℓ∣∣ ∣∣logλjλ′j∣∣ ∣∣≤δ.

The same equation can be made for this term with . Let

 αj=max{λℓΛj,λ′ℓΛ′j}.

Then we have

 ∣∣ ∣∣log(1+λℓΛj)−log(1+λ′ℓΛ′j)∣∣ ∣∣ ≤∣∣log(1+αj)−log(1+e−δαj)∣∣≤∣∣1−e−δ∣∣|αj|.

Furthermore, because then and

 |Lx,x′(π)−Lx,x′(π′)|≤2δ +ℓ′−1∑j=ℓ|1−e−δ|e2Cm−j+1 ≤2δ+|1−e−δ|e2CHm ≤δ(2+e2Cln(m+1)).

In the above equation, is the th harmonic number. A similar calculation can be made when . Setting concludes the proof. ∎

###### Proof of Theorem 2..

Define the empirical process function to be

 ρn(x):=1nn∑i=1(logPX⋆i(Πi)PXi(Πi)−ElogPX⋆i(Πi)PXi(Πi)).

By the listwise representation theorem, is a function of independent exponential random variables. Moreover, if we were to change the value of a single element then this would result in a change of permutation of the type described in Lemma 2. Notice that the bound on the change in the relative loss is , where , and notice that the sum of squares of these bounds are,

 ∑i,kC20∥logϕ(Xi)−logϕ(X′i)∥2∞ =mC20n∑i=1∥logϕ(Xi)−logϕ(X′i)∥2∞ =mC20∥Z−Z′∥2∞,2,

where are applied elementwise to respectively. By Lemma 2 and McDiarmid’s inequality,

 P{n(ρn(x)−ρn(x′))>ϵ}≤exp(−2ϵ2mC20∥Z−Z′∥2∞,2).

Hence, the stochastic process is a subGaussian field with canonical distance,

 d(X,X′):=√mC0∥Z−Z′∥∞,2.

The result follows by Dudley’s chaining (Talagrand, 2006). ∎

###### Lemma 3.

If is 1-Lipschitz then we have that .

###### Proof.

Let , and . Then

 |zij−z′ij|≤|xij−x′ij|,

by the Lipschitz property. Hence, , and so

 N(u,Z,∥.∥∞,2)≤N(u,X,∥.∥∞,2).

###### Proof of Corollary 1..

Consider two matrices in the model (7) then

 ∥xi−x′i∥∞=maxj|β⊤zij−β′⊤zij|≤∥β−β′∥2∥Zi∥2,∞.

Let then

 ∥X−X′∥∞,2≤ζ∥β−β′∥2.

The covering number of is therefore bounded by

 N(X,u,∥.∥∞,2)≤N(Bcb,u/ζ,∥.∥2)≤(ζC0cbu)s,

for an absolute constant , where is the ball of radius . The result follows by Lemma 3 and Theorem 2. ∎

###### Proof of Corollary 2..

Let and such that bounded by 1 in Frobenius norm (modifying the Frobenius norm bound does not change the substance of the proof). Consider

 |u⊤ivj−u′⊤iv′j|≤|u⊤ivj−u′⊤ivj|+|u′⊤ivj−u′⊤iv′j| ≤∥ui−u′i∥∥vj∥+∥u′i∥∥vj−v′j∥.

Maximizing this over the selection of ,

 maxj|u⊤ivj−u′⊤iv′j| ≤∥ui−u′i∥2∥V∥2,∞+∥ui∥2∥V−V′∥2,∞.

Hence,

 ∥X−X′∥∞,2 ≤∥U−U′∥F∥V∥2,∞+∥U′∥F∥V−V′∥2,∞ ≤∥U−U′∥F+∥V−V′∥2,∞.

Consider the vectorization mapping from the matrix to the dimensional vectors. The Frobenius norm is mapped to the norm, and we can consider the norm to be, the norm . The -norm unit ball () is just the Cartesian product of the norm ball in dimensions. The volume of a -dimensional ball, , is bounded by

 Cl≤Vd(eπ)d/2dd2≤Cu,

where are universal constants. So the volume ratio between the norm ball and the norm ball is bounded by

 V(B2)V(Bρ)≤C(rr/2