Community Question Answering services (CQAs) such as Stack Exchange and Yahoo! Answers are examples of social media sites, with their usage being examples of an important type of computer supported cooperative work in practice. In recent years, the usage of CQAs has seen a dramatic increase in both the frequency of questions posted and general user activity. This, in turn, has given rise to several interesting problems ranging from expertise estimation to question difficulty estimation, and from automated question routing to incentive mechanism design on such CQAs[30, 2, 23].
In this work, we focus on the problem of routing newly posted questions to suitable experts before answers are written (item cold-start). Usually, there are two types of questions in CQAs – resolved (questions with answers) and newly posted questions (questions that have not received any answers). The newly posted questions may themselves be posted by new askers (such as newly registered users who have not asked a question earlier) or existing askers (such as users who have asked several questions previously). We refer to these newly posted questions as cold questions. The majority of approaches have focused on evaluating content quality after the fact (after questions have been resolved) . Yet, as the CQAs continue to grow, routing the cold questions to matching experts before answers have been provided has become a critical problem. For example, in Stack Overflow, about million questions have not been answered 111https://stackexchange.com/sites#.
Recently, the usage of graphs has seen an increase in solving the question routing problem. For example, Sun et al.  proposed QDEE to leverage Expertise Gain Assumption (EGA) to build the competition graph and then applied social agony [26, 27] to infer graph hierarchy and assign each node a scalar value to represent where it stands in the competition graph. The assigned scalar value of question nodes are question difficulty scores, and the assigned scalar value of user nodes are user expertise scores. QDEE then routed cold questions to users with matching expertise based on the corresponding questions’ difficulty level. It’s worth mentioning that both question difficulty and user expertise follow the characteristic of asymmetric transitivity. For example, given a question is easier than and is easier than , we can infer that is easier than conveniently. Sun et al.  proposed ATP to tackle the challenge of directed graph embedding with asymmetric transitivity preservation and then they leveraged the proposed embedding results to appropriately route and assign newly posted questions to users with the suitable expertise and interest in CQAs. However, a multiple pipeline including directed graph embedding generation and experts finding is required where each step has to be optimized separately. Another drawback is that ATP can only leverage graph structure information, while other information (such as textual information) that does not follow the asymmetric transitivity can not be incorporated. It limits ATP in inferring the embedding of cold questions asked by new askers who have no activities in CQAs and hence ATP cannot work well for routing such questions to suitable answerers.
In Stack Exchange sites, voting is central for providing quality questions and answers 222https://stackoverflow.com/help/why-vote. The more that people vote on a post, the more certain future visitors can be confident of the quality of information contained within the post. Voting indicates a CQA community’s long-term review for a given user’s expertise level under a specific topic. ColdRoute  views the problem of identification of best answerers as a regression problem (finding the answerers who have the highest predicted voting scores). The input features of ColdRoute are users’ past asking and answering activities in CQAs, and ColdRoute can leverage Factorization Machines (FMs) [19, 20] to model pairwise interactions among different objects (questions, askers, answerers, and question tags). While in theory FMs can model high-order interactions, in practice it only models order-2 feature interactions due to the high complexity .
Therefore, we propose to solve the cold question routing problem by marrying the merits of the above two types of methods. On one hand, we use graph embedding techniques to encode useful high-order graph structure information, and on the other hand, we enable the incorporation of heterogeneous information (e.g. textual) in the node embedding as well. Specifically, users (including both askers and answerers), questions, and question tags are represented as three different types of nodes, which are connected by their historical interactions into an undirected CQA graph. We then employ a Graph Convolutional Network (GCN)  to generate deeper and neighbor-aware node representations in the undirected graph. Our proposed method allows more flexible node interactions and can enrich the question/user node embeddings with the textual information from question tags. And question tags can bridge the gap between cold questions and exiting nodes in the CQA graph. This overall end-to-end framework can be optimized directly with the cold question routing objective, which enables the learning of task-specific embeddings for heterogeneous nodes in the CQA graph, and a better cold question routing performance.
In summary, we made the following contributions:
Encode users’ past asking and answering activities and textual information by using an undirected heterogeneous graph, of which nodes are questions, tags, and users (askers and answerers) and edges are interactions among them.
Build an end-to-end framework which can route cold questions in CQAs effectively.
Demonstrate the effectiveness of our proposed framework on various Stack Exchange sites.
Recently, Sun et al.  observed that users typically gain expertise across multiple interactions with the CQA and tend to ask more difficult questions within the same domain over time, which is referred to as Expertise Gain Assumption (EGA). They leveraged EGA and added additional edges between questions asked by the same user to the previous competition graph [13, 14, 28] to combat the sparseness problem. They then proposed QDEE to lever social agony [26, 27] to infer graph hierarchy and assign each node a scalar value to represent where it stands in the competition graph. The assigned scalar value of question nodes are question difficulty scores, and the assigned scalar value of user nodes are user expertise scores. There is some work which can model the users’ past asking and answering activities by using a graph and then some graph embedding methods [2, 33, 32] are proposed to address the above limitation. The problem of graph embedding seeks to represent vertices of a graph in a low-dimensional vector space in which meaningful semantic, relational and structural information conveyed by the graph can be accurately captured . Recently, one has seen a surge of interest in developing such methods including ones for learning such representations for directed graphs while preserving important properties such as asymmetric transitivity [17, 22]. Question difficulty and user expertise follow the characteristic of asymmetric transitivity. For example, given a question is easier than and is easier than , we can infer that is easier than easily. It happens to estimate user expertise too. We can infer that has more expertise than based on the fact that has more expertise than and has more expertise than in a specific domain. Sun et al.  proposed ATP to tackle the challenge of directed graph embedding with asymmetric transitivity preservation and then they leveraged the proposed embedding results to appropriately route and assign newly posted questions to users with the suitable expertise and interest in CQAs. However, a multiple pipeline including directed graph embedding generation and experts finding is required where each step has to be optimized separately. Another drawback is that ATP can only leverage graph structure information while cannot leverage textual information (such as question tags) since these textual information cannot be naturally leveraged to construct the input directed graph. It indicates that ATP has some limitation on inferring the embedding of cold questions asked by new askers who have no activities in CQAs and hence cannot work well for routing cold question asked by new askers to suitable answerers.
Question difficulty and user expertise can vary in different topics. In Stack Exchange sites, users are required to use tags (a tag is a word or phrase) to describe the topic(s) of the question 333https://stackoverflow.com/help/tagging. Each question can be assigned multi-tags to represent its most relevant topics. Hence tags are important user-generated category information that achieves fine-grained and dynamic topic representation. Users who use a particular tag when posting questions or answers might prefer topic summaries most relevant to that tag . In our experiments, the average number of tags per question is and in Stack Exchange site Apple and Physics respectively. Hence a solely scalar value to represent question difficulty level or user expertise is not thorough. By iteratively introducing questions tags and textual descriptions (question title and body), ColdRoute  demonstrated that question tags play a more important role than question title and body in terms of experts finding. ColdRoute views the problem of identification of best answerers as finding the answerers who have the highest predicted voting scores. Given a cold question and a set of potential answerers , ColdRoute predicts each candidate ’s voting score for , where , and then selects the user who achieves the highest predicted voting score as the best answerer for . In Stack Exchange sites, voting is central for providing quality questions and answers 444https://stackoverflow.com/help/why-vote. Voting up a post signals to the rest of the community that the post is interesting, well-researched, and useful. A highly voted post reflects the quality of the post – which may be viewed by the future visitors. The more that people vote on a post, the more certain future visitors can be confident of the quality of information contained within the post. Voting indicates a CQA community’s long-term review for a given user’s expertise level under a specific topic. Each voting score is an integer, which is calculated based on the difference between the corresponding answer’s up-votes and down-votes which are assigned to it by users who viewed the question or provided answers in the CQAs. The input features of ColdRoute are users’ past asking and answering activities in CQAs, and ColdRoute can leverage Factorization Machines (FMs) [19, 20] to model the interactions among different objects (questions, askers, answerers, and question tags).
Assume we are given four relational sets of data in terms of Questions , Askers , Answerers , and Question Tags . For each question , we have a tuple of the form , , , and , where , , , . Each voting is an integer, which is calculated based on the difference between ’s up-votes and down-votes which are assigned to it by users who viewed the question or provided answers for that in the CQA environment. Note that the for a question may not be specified by .
Given the preliminaries (above), in this work, we focus on the problem of routing newly posted questions to suitable experts before answers are written (item cold-start). Each quadruple case , where , , , has a voting score , which is equal to the difference between times of up-voting and down-voting. Given the testing question set , the predicted ranking list of all potential answerers for a test question is for all . The answerer who’s ranked top among all candidates in for the corresponding target question will be selected as the best answerer for .
In this section, we talk about the design of our proposed framework (both sequential and end-2-end). The key steps of the sequential and end-2-end model for routing newly posted questions are shown in Figure 1. We refer the sequential model as Seq and the end-2-end model as EndCold. The key steps of Seq are:
Build the undirected heterogeneous CQA graph which incorporates users past asking and answering activities and their corresponding textual information in CQAs
Apply suitable graph embedding methods to learn representations for each node (including question, user, and tag) in the corresponding input CQA graph
Model the question routing problem as a regression problem, and concatenate the embeddings of question, asker, answerer, and tag as feature vectors and use corresponding voting scores as targets to train a suitable regression model
Apply to predict the voting score a candidate user can achieve on a given cold question
Routing to its corresponding predicted best answerer, identified by selecting the user who’s predicted to achieve the highest voting score among all candidate answerers
The biggest difference between our sequential model Seq and end-2-end framework EndCold is that EndCold optimizes the graph embedding module (step in Seq) and regression module (step in Seq) simultaneously as shown in Figure 1.
Build the undirected CQA graph
Figure 2 illustrates how we build the corresponding heterogeneous CQA graph of which nodes are questions, tags, and users (including askers and answerers) and edges represent the interactions among questions, askers, answerers, and question tags. User asked a question which has tags and . User , , and answered question . User asked a question with tag and . Question was answered by user , and . Edges are interactions among all questions, tags, askers and answerers.
Both users’ past asking and answering activities and their corresponding textual information (question tags) are incorporated in the above CQA graph, which can help us bridge the gap between cold question nodes and existing nodes in the CQA graph. Cold questions can be connected with the existing CQA graph via the common question tag nodes they have and existing asker node (not applicable for cold questions asked by new askers).
The Sequential Module for Cold Question Routing
In this section, we talk about the sequential model named as Seq-TA (By default, Seq is referred to Seq-TA in our paper) which takes the undirected CQA Graph with question tags as the input and leverages any suitable graph embedding algorithms to generate embeddings for the question, user, and tag nodes. Take our experiments for example, we use node2vec  to generate corresponding embeddings555Other state-of-the-art undirected graph embedding methods can be applied to this module too. Each answering thread between a question and an answerer can be represented as a quadruple of the target question, its asker, the corresponding answerer, and question tags. The corresponding quadruple case can be represented as a concatenated feature vector , where , , and are the embedding vectors of question , answerer , and asker respectively. is the average embedding of tags in set . The voting score of achieved on is an integer. A regression model is trained based on the above feature vectors as inputs and corresponding voting scores as the targets.
This design gives us the flexibility to explore the different features’ relative importance in cold question routing. For example, information of asker and question tags can be iteratively introduced to our model to explore their relative importance as follows:
Seq-A: explore the importance of question asker by using triples of on routing cold questions asked by existing askers. Other settings are as same as Seq-TA.
Seq-T: explore the importance of question tags by using triples of on routing cold questions either from existing askers or new askers. Other settings are as same as Seq-TA.
Seq-Un: explore the importance of question asker and tags by using tuples of on routing cold questions. Other settings are as same as Seq-TA.
Regarding the regression module, we examined two types of SVM based regressors implemented by scikit-learn. One is a Epsilon-Support Vector Regression (SVM)666http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html with the polynomial kernel (degree is set as ). Another is the LinearSVR777http://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVR.html888https://keras.io has also been taken into examination.
The End-2-End Module for Cold Question Routing
The end-2-end module consists of two layers: embedding layer and regression layer. The embedding layer is built based on the Graph Convolutional Networks (GCNs) . The core idea behind GCNs is to learn how to iteratively aggregate feature information from local graph neighborhoods using neural networks . A single “convolution” operation transforms and aggregates feature information from a node’s one-hop graph neighborhood, and by stacking multiple such convolutions information can be propagated across far reaches of a graph 
. The embedding layer is followed by a multilayer perceptron (MLP) to do the regression. In our experiments, we uselayers in GCNs and hidden layers in MLP. We evaluate the mean squared error for all the training examples.
Experts Finding for Cold Questions
The final step of both Seq and EndCold is to identify the best answerers for cold questions, which works as follows:
Given a cold question and a set of potential answerers , predict each candidate ’s voting score for , where
Select the user who achieves the highest voting score as the best answerer for
Given a newly posted question asked by an asker using tags and a potential answerer , the prediction function in Seq can treat the new question as a missing value by . The user who achieves the highest value of will be selected as the best answerer for question . It’s possible that the newly posted question can be asked by a new asker who has no information to learn the corresponding embedding vector. Then the prediction function can treat the new asker as a missing value by .
Instead of using missing value to represent the feature vector of in Seq, EndCold can leverage GCNs to propagate information from existing tag nodes and asker nodes (if applicable) to . can accumulate these information and get its corresponding embedding vector as . The regression function in EndCold can predict the voting score can achieve on as .
It is possible that the potential answerer is a newly registered user who has not provided any answer before in CQAs (user cold-start). In this scenario, the prediction function can be simplified as . All new registered users will receive the same predicted voting score for the same target question. More efforts will be spent to make accurate predictions for the user cold-start problem in our future work.
Experiments and Analyses
We conducted our cold question routing experiments following the same settings as ColdRoute on large Stack Exchange sites . We compare our proposed models with state-of-the-art methods based on two popular evaluation criteria (Precision [35, 33, 2, 32, 24, 22] and Accuracy [33, 2, 32, 24, 22])999Other evaluation criteria such as Precision and MRR  used in ColdRoute  follow the same performance pattern as Precision and Accuracy. To save space, we don’t report their results here..
Performance of Different Sequential Models
In this section, we compare the routing performance of different sequential models with different embedding methods (such as node2vec and ATP) on various input graphs (such as directed competition graph used by QDEE and undirected CQA graph with question tags proposed in this work) with our proposed models. The key steps of each method in this section follow the same sequential framework as we discussed in Methodology section. The differences fall in how to build the input graph and get the embedding vector for each node in the corresponding input graph. The details of all comparison partners are as follows:
Seq-Dir-ATP: The input graph of Seq-Dir-ATP is the same directed competition graph used in QDEE. Embeddings of nodes (questions and users) are generated by ATP. Given a newly posted question , Seq-Dir-ATP applies Expertise Gain Assumption (EGA)  to use the source and target embedding of the question which has the highest difficulty level among all questions asked by the same asker to approximate ’s source and target embedding respectively. Given a candidate user , the number of votes can achieve on can be predicted by feeding the concatenated source and target feature vectors of and to a regression model , which can be represented as , where and represent corresponding node’s source and target feature vector respectively. The dimension of both source and target feature vector is tuned to set as . Hence the dimension of concatenated feature for is .
Seq-Dir: It uses the same directed competition graph as Seq-Dir-ATP. However, Seq-Dir leverages node2vec  to generate embeddings for question and user nodes. Like Seq-Dir-ATP, EGA is applied to generate embeddings for newly posted questions. Feature vector representations of and are concatenated as inputs of the regression model. Parameters are tuned to set as 101010https://github.com/snap-stanford/snap/tree/master/examples/node2vec. Hence the dimension of concatenated feature for is .
It’s worth mentioning Seq-Dir-ATP and Seq-Dir cannot handle the scenario of cold questions asked by new askers since EGA can only be applied to questions asked by existing askers to infer the embedding vector of the newly posted question. Hence, we only show their corresponding performance on routing cold questions asked by existing askers. The experiment results are summarized in Figure 3, where we can make the following observations:
The best overall performance is achieved by Seq-AT, which concatenates the embedding of all four nodes in the quadruple as input features and feeds them to the regression model. The fact that Seq-AT consistently outperforms its feature-ablated variants Seq-A, Seq-T and Seq-Un gives us some guideline that it is necessary to incorporate both user past activities (asking and answering) and textual information when designing question routing algorithms.
For all the CQA sites and under all metrics, the four methods using the undirected CQA graph with question tags as input perform significantly better than Seq-Dir and Seq-Dir-ATP) which use the direct competition graph. It demonstrates the advantage of using our proposed undirected CQA Graph with question tags in comparison with using the directed competition graph leveraged by QDEE  and ATP  on the task of cold question routing.
Note that despite Seq-Un does not explicitly use the embedding of question tag nodes, it still significantly outperforms Seq-Dir. This is because the graph embedding algorithm (node2vec ) in our experiments) allows neighboring nodes to share information, so the question node embeddings are regulated by their assigned tags as well.
Pointwise v.s. Pairwise
In this section, We would like to compare the performance difference between pointwise and pairwise based sequential models on cold question routing in CQAs, with hoping that the corresponding results can guide our future design of the question routing system in CQAs. The difference between pointwise and pairwise based sequential models are listed as follows:
Pointwise: its goal is to learn a regression function . The user who achieves the highest value of will be selected as the best answerer for question .
Pairwise: We introduce the relative quality rank to model users’ past answering activities in CQAs, which is in the form of , meaning that the answer provided by answerer , obtains more voting scores than the answer provided by the answerer for question asked by asker with tags set . Let denote the set of ranking pair constraints derived from the community votes. More formally, we aim to learn a ranking function that for any , the inequality holds: . The user who satisfies the most number of above constraints will be selected as the best answerer for question
. Given the representations of question and answerers, and the relative quality rank in CQA sites, the loss functionin Pairwise is designed as follows:
where denotes the voting score achieved by high-quality answerers and represents the voting score of low-quality answerers for question routing. The hyper-parameter () controls the margin in the loss function and is the set of pairwise relative ranking constraints.
In our experiments, the pairwise based model Seq (pairwise) leveraged  to learn the partial order of two potential answerers given a newly posted question. As shown in Table 1 (cold questions asked by existing askers) and 2 (cold questions asked by new askers), Seq (pairwise) does not show significant improvement over the pointwise model Seq (pointwise) consistently. The reason is that Seq (pairwise) suffers from the data sparseness problem. Table 3 shows the statistical information of Stack Exchange sites used in our experiments. The average number of answers per question among the Stack Exchange sites is , and of questions have only answer, which indicate that Seq (pairwise) cannot extract enough valid question-answerer pairs for training. With CQAs growing and information of answers becoming dense, Seq (pairwise) can have more valid relative ranking constraints in to train and then be more robust and effective.
|Avg # answers per question||1.8122||1.6391||1.7796||1.7823||2.1405||1.8521||1.5371||1.7917|
|# questions with only one answer (percentage)||0.5803||0.6339||0.5723||0.5516||0.4876||0.5463||0.6507||0.5524|
Sequential Model v.s. End-2-End Model
In this section, we aim to compare the difference between our sequential model and end-2-end model. As shown in Table 1 and 2, the overall performance of end-2-end model EndCold is better than the performance of sequential models (both pointwise and pairwise). For example, EndCold
improves upon the evaluation metrics (Accuracy and Precision) over Seq (pointwise) by and respectively on routing cold questions asked by new askers. Above results demonstrate the effectiveness of our end-2-end framework in comparing with the sequential models.
Comparison with the State-of-the-art on Cold Question Routing
In this section, we compared our proposed model with ColdRoute , a state-of-the-art model on routing cold questions posted by both new and existing askers to suitable experts. ColdRoute
encodes users’ past asking and answering activities by using one-hot encoding, which can cause sparsity problem. Instead of using one-hot encoding and then suffering from the sparsity problem, our proposed model leverages the dense embedding vectors of question, asker, answerer and question tags (average) to build the input feature vector for training. It’s worth mentioning thatColdRoute fails to take the higher-order interactions among questions, askers, answerers, and question tags into consideration due to the high complexity. ColdRoute-A, ColdRoute-T, and ColdRoute-TA are three variants of ColdRoute and they were designed to explore the importance of asker information, question tags, and both respectively.
We also compare our model with some semantic matching based approaches [21, 12, 11, 1, 29, 25, 30, 24].These matching models have demonstrated their power on finding suitable experts recently . Due to space limitation111111Other semantic matching models such as Bag of Words [3, 34] and Latent Dirichlet Allocation [6, 7], and Word2Vec  follow the same pattern as Doc2Vec, we only show the performance of Doc2Vec [10, 1] The Doc2Vec used in our paper is implemented by gensim 121212https://radimrehurek.com/gensim/models/doc2vec.html. The dimension of the feature vector is tuned to set as .
Conclusion and Future Work
In this paper, we proposed an end-to-end framework for cold question routing in CQAs. The input of the framework is an undirected heterogeneous CQA graph with encodes users’ past asking and answering activities and textual information of questions. Our proposed model can leverage the higher-order graph structure and content information to embed nodes in the input graph, and do the question routing simultaneously. Extensive experiments show that our model performs better than the state-of-the-art models on routing cold questions (asked by both existing askers and new askers).
As a future work, we plan to test our models on other CQAs with different settings (such as having more dense askers and answerers). In order to increase the expertise of the entire community, we plan to address the problem of routing newly posted questions (item cold-start) to newly registered users (user cold-start) in CQAs.
Acknowledgments This work is supported by NSF grants EAR-1520870, CCF-1645599, CCF-1629548, IIS-1550302, and CNS-1513120, and a grant from the Ohio Supercomputer Center (PAS0166). All content represents the opinion of the authors, which is not necessarily shared or endorsed by their sponsors.
Predicting best answerers for new questions: an approach leveraging distributed representations of words in community question answering. In FCST, Cited by: Comparison with the State-of-the-art on Cold Question Routing.
-  (2016) Community-based question answering via heterogeneous social network learning. In AAAI, Cited by: Introduction, Related Work, Experiments and Analyses.
-  (2013) Learning to rank effective paraphrases from query logs for community question answering. In AAAI, Cited by: footnote 11.
-  (2016) Node2Vec: scalable feature learning for networks. In KDD, Cited by: The Sequential Module for Cold Question Routing, 2nd item, 3rd item.
-  (2017) DeepFM: a factorization-machine based neural network for ctr prediction. In IJCAI, Cited by: Introduction.
-  (2008) Tapping on the potential of qa community by recommending answer providers. In CIKM, Cited by: footnote 11.
-  (2012) Question-answer topic model for question retrieval in community question answering. In CIKM, Cited by: footnote 11.
-  (2006) Training linear svms in linear time. In KDD, Cited by: Pointwise v.s. Pairwise.
-  (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. Cited by: Introduction, The End-2-End Module for Cold Question Routing.
-  (2014) Distributed representations of sentences and documents. In ICML, Cited by: Comparison with the State-of-the-art on Cold Question Routing.
-  (2011) Question routing in community question answering: putting category in its place. In CIKM, Cited by: Comparison with the State-of-the-art on Cold Question Routing.
-  (2010) Routing questions to appropriate answerers in community question answering services. In CIKM, Cited by: Comparison with the State-of-the-art on Cold Question Routing.
-  (2011) Competition-based user expertise score estimation. In SIGIR, Cited by: Related Work.
-  (2013) Question difficulty estimation in community question answering services. In EMNLP, Cited by: Related Work.
-  (2018) Multi-dimensional network embedding with hierarchical structure. In WSDM, Cited by: Related Work.
-  (2013) Distributed representations of words and phrases and their compositionality. In NIPS, Cited by: footnote 11.
-  (2016) Asymmetric transitivity preserving graph embedding. In KDD, Cited by: Related Work.
-  (2009) Labeled lda: a supervised topic model for credit attribution in multi-labeled corpora. In EMNLP, Cited by: Related Work.
-  (2010) Factorization machines. In ICDM, Cited by: Introduction, Related Work.
-  (2012) Factorization machines with libfm. In ACM Trans. Intell. Syst. Technol., Cited by: Introduction, Related Work.
-  (2016) A comprehensive survey and classification of approaches for community question answering. ACM Trans. Web. Cited by: Comparison with the State-of-the-art on Cold Question Routing.
-  (2019) ATP: directed graph embedding with asymmetric transitivity preservation. In AAAI, Cited by: Introduction, Related Work, 2nd item, Experiments and Analyses.
-  (2018) QDEE: Question Difficulty and Expertise Estimation in Community Question Answering Sites. In ICWSM, Cited by: Introduction, Introduction, Related Work, 1st item, 2nd item.
-  (2018) ColdRoute: effective routing of cold questions in stack exchange sites. In ECML PKDD, Cited by: Introduction, Related Work, Comparison with the State-of-the-art on Cold Question Routing, Comparison with the State-of-the-art on Cold Question Routing, Experiments and Analyses, footnote 9.
-  (2013) When relevance is not enough: promoting diversity and freshness in personalized question recommendation. In WWW, Cited by: Comparison with the State-of-the-art on Cold Question Routing.
-  (2014) Faster way to agony discovering hierarchies in directed graphs. In ECML PKDD, Cited by: Introduction, Related Work.
-  (2015) Hierarchies in directed networks. In ICDM, Cited by: Introduction, Related Work.
-  (2014) A regularized competition model for question difficulty estimation in community question answering services. In EMNLP, Cited by: Related Work.
-  (2014) Exploring user expertise and descriptive ability in community question answering. In ASONAM, Cited by: Comparison with the State-of-the-art on Cold Question Routing.
-  (2013) CQArank: jointly model topics and expertise in community question answering. In CIKM, Cited by: Introduction, Introduction, Comparison with the State-of-the-art on Cold Question Routing.
Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, pp. 974–983. External Links: Cited by: The End-2-End Module for Cold Question Routing.
-  (2017) Community-based question answering via asymmetric multi-faceted ranking network learning. In AAAI, Cited by: Related Work, Experiments and Analyses.
-  (2016) Expert finding for community-based question answering via ranking metric network learning. In IJCAI, Cited by: Related Work, Experiments and Analyses.
-  (2013) Improving question retrieval in community question answering using world knowledge. In IJCAI, Cited by: footnote 11.
-  (2014) Ranking user authority with relevant knowledge categories for expert finding. In WWW, Cited by: Experiments and Analyses, footnote 9.