Learning to Represent Human Motives for Goal-directed Web Browsing

08/07/2021 ∙ by Jyun-Yu Jiang, et al. ∙ Amazon Microsoft 0

Motives or goals are recognized in psychology literature as the most fundamental drive that explains and predicts why people do what they do, including when they browse the web. Although providing enormous value, these higher-ordered goals are often unobserved, and little is known about how to leverage such goals to assist people's browsing activities. This paper proposes to take a new approach to address this problem, which is fulfilled through a novel neural framework, Goal-directed Web Browsing (GoWeB). We adopt a psychologically-sound taxonomy of higher-ordered goals and learn to build their representations in a structure-preserving manner. Then we incorporate the resulting representations for enhancing the experiences of common activities people perform on the web. Experiments on large-scale data from Microsoft Edge web browser show that GoWeB significantly outperforms competitive baselines for in-session web page recommendation, re-visitation classification, and goal-based web page grouping. A follow-up analysis further characterizes how the variety of human motives can affect the difference observed in human behavioral patterns.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

page 9

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Constructs such as motives and goals are recognized in the psychology literature as the fundamental forces that guide human behavior. While a variety of interpretations of goals exist, there is consensus that these constructs regulate controlled cognitive processes such as planning and resource allocation, and direct behavioral sequences intended to enact specific performances (Chulef, 1993). When people browse the web, which is one important behavior common in modern lives, goals also underlie why they do what they do. For example, they may buy products to work towards their fitness goals, plan trips to stay connected with friends and family, conduct business to pursue career success, or do research or seek health advice in support of their well-being. Understanding peoples’ goals not only helps answer the “why” questions about their activities (Faaborg and Lieberman, 2006; Smith and Lieberman, 2010), but also signifies where the opportunities sit for potential improvements.

Although goals and motives are fundamental to browsing behaviors, they stay unobserved and cannot be easily inferred due to the variety and complexity of activities prevalent on modern web browsers. Moreover, the question of how goals and motives can be utilized to assist browsing remains unanswered. To bridge the gap, we propose to adopt the well-established bodies of psychological theories (Moskowitz and Grant, 2009; Barua et al., 2014; Chulef et al., 2001) regarding human motives to the need for browser-centric activities. For instance, the motive of staying physically healthy may energize a person to consider purchasing a rowing machine or to learn nutritious ingredients via a number of web interactions. As a result, a better understanding of human motives can guide how we might enhance browsing experiences pertaining to people’s needs.

Previous research on supporting user goals on the web has primarily sought to understand and categorize a person’s intent manifested through search queries. In their seminal studies, Broder (2002) and Rose and Levinson (2004)classified the goal of a user query into one of three categories: navigational, informational, and transactional. Successive work refined intent categorization by introducing purchase, sell or job search intents (Dai et al., 2006; Li et al., 2008), as well as examined the variability of intents across different users (Teevan et al., 2008). The main objective of research in this field is often to inform and improve the search process (Broder, 2002; Caruccio et al., 2015; Rose and Levinson, 2004; Lee et al., 2005), and the characteristics of user goals tend to be task-oriented or situation-specific. In contrast to that, we emphasize incorporating the fundamental human goals into the context of web browsing, since they are recognized to be key to the core values in people’s lives (Barua et al., 2014) (e.g. health, well-being, sustainability, and learning).

In this paper, we focus on tackling two research agendas toward our goal-directed vision, namely, how to represent influential human goals111Unless otherwise mentioned, we use human motives and goals interchangeably in this paper. and how to leverage the learned representations to provide assistance in browsing sessions. We propose a unified neural framework, Goal-directed Web Browsing (GoWeB), for both objectives laid out in two phases. The first phase concerns human motives. Psychology research has developed decades of knowledge informing which human goals are considered influential through theoretical and empirical evidence. Inspired by the findings, GoWeB uses these goals as the backbone and builds on the top to learn intrinsic goal representations following a distance-based reconstruction objective to retain inherent hierarchical structures222For example, in Chulef et al. (2001)’s human goal taxonomy, the goals be affectionate and share feelings are both decedents of a parent goal friendship.

. GoWeB further devises a neural goal estimator, which learns to transfer the knowledge entailed in these goal representations to estimating people’s motives for visiting web pages. The second phase focuses on enhancing web experiences with what we have learned. Specifically, GoWeB integrates the goal estimator with modern language models 

(Devlin et al., 2019) to assist three common interactions or needs people have on the web. We consider an in-session web page recommender that recommends web pages to help people advance their goals, a web page re-visitation classifier that predicts if a person will revisit a web page in the future due to recurring or unfinished goals, and a web page grouping method that clusters in-session web pages according to underlying goals. The overall framework is illustrated in Figure 4.

We evaluate GoWeB using anonymized browsing logs obtained from the Microsoft Edge browser. The experimental results show that GoWeB consistently outperforms competitive baselines in all three browser-centric applications, suggesting that capturing fundamental human motives can empirically improve the intelligence of web browsers. To understand the effectiveness, we find that the learned intrinsic goal representations largely preserve the human-curated hierarchical relations. Meanwhile, the goal estimator can effectively predict the goals behind web page visits evaluated based on quantitative and qualitative exercises. Our follow-up analysis further characterizes the browsing patterns when people engage in pursuing different goals. The results suggest that people may explore multiple directions simultaneously when the categories of session’s focal goals are broad (e.g., Ethics & Idealism), while they may revisit web resources in shorter intervals for goals concerning social interactions (e.g., Friendship).

2. Related Work

Studying goals is an important, long-lasting topic across multiple research disciplines. We examine the different notions of goals that have been investigated in the literature.

Task-oriented Goals in Search and Web Browsing. Prior work has a strong focus on studying search goals as they contribute to a critical segment of web browser usage. The mainstream work in this space follows Broder (2002), and Rose and Levinson (2004) who classified the goal of user queries into navigational, informational, and/or transactional. Lee et al. (2005) identified that 60% of search queries can be associated with informational or navigational goals, and proposed using clicks and anchors to predict those. Caruccio et al. (Caruccio et al., 2015) introduced a lightweight taxonomy that added a handful of sub-classes to informational and transactional queries. Casting as a classification problem, they proposed content and behavioral features to predict which class in the taxonomy a query should belong to using data collected from a custom browser extension. For more complex search needs, Jones and Klinkner (Jones and Klinkner, 2008) leveraged human annotations and built predictive models that were trained to segment sequences of user queries into same or different search goals. Among a set of hand-crafted features, they showed that lexical features such as words and characters were identified as the most useful. Similarly, Law and Zhang (Law and Zhang, 2011) showed that the ability to decompose a complex search goal into sub-problems (i.e., a set of queries and corresponding search results) can better support a user’s information need.

General web browsing behavioral patterns tend to involve higher flexibility and complexity than search. Kumar and Tomkins (2010) analyzed large-scale commercial logs and showed that page visits related to search comprise only one-sixth of the log sample. In addition to search, they identified that content consumption (e.g., news, portals, games, verticals, multimedia) comprise about half of online page views, while communication (e.g., email, social networking, forums, blogs, chat) represent about one-third of those. As a result, general web browsing patterns require different types of support. For example, past studies have shown that people are often unable to focus on or return to their goals when using a web browser due to the prevalence of distractions on the web  (Aagaard, 2015; Tseng et al., 2019). Research on Cyberloafing (Glassman et al., 2015; Lim, 2002) and task resumption on the web (Gillie and Broadbent, 1989; McFarlane and Latorella, 2002; O’Conaill and Frohlich, 1995) have shown that individuals experience difficulty when resuming their main task after external- or self-interruptions (Czerwinski et al., 2004; Adler and Benbunan-Fich, 2013).

Faaborg and Lieberman (2006) introduced a program-by-example goal-oriented web browser that allows people to customize how they organize and interact with the web. A person who visits a recipe website, for example, may want to know the nutritional information. By explicitly demonstrating how to complete nutrition extraction once, the system could potentially evoke the same human-specified macros on similar websites. Dix et al. (2010) attempted to understand the goals behind user behavior by connecting actions and user-created personal ontology structures.

Higher-ordered Human Goals. Understanding human goals and motives has long been a central area of research in the psychology literature, as the goals of individuals largely direct the behavior in which they engage (Klinger, 1987). Earlier attempts in creating goal taxonomies relied on theoretical viewpoints. McDougall (McDougall, 1933) presented a list of 13 instincts while Murray (Murray, 1938) articulated 44 variables of personality as forces determining behavior. Chulef et al. (Chulef et al., 2001) took an empirical approach and recruited participants of diverse backgrounds to delve into developing a hierarchical human goal taxonomy based on similarity between goals, providing a concrete and comprehensive structure. More recently, Talevich et al. (Talevich et al., 2017) iterated on the taxonomy derived in (Chulef et al., 2001) with several added classes of human motives.

As our work incorporates long-term goals in browsing sessions to provide goal-directed assistance, we review the classic and state-of-the-art technical methods applicable in the three browser-centric tasks, and conclude with a brief summary comparing previous approaches and ours.

Recommendation. The task of web page recommendation (Göksedef and Ögüdücü, 2010) aims to predict the information needs of users and provide them with recommendations to facilitate their navigation. Classic approaches to recommendation relies on mining user-item relationships through collaborative filtering (Su and Khoshgoftaar, 2009) and matrix factorization (He et al., 2017; Rendle et al., 2009). Session-based approaches (Tang and Wang, 2018)

take into account session-level user actions as a sequential stream of events, based on which recommendations are made to users. In recent years, large-scale deep neural networks

(Zhang et al., 2019) have shown top performing effectiveness in recommender systems, including leveraging GRU (Hidasi et al., 2016), self-attention blocks (Kang and McAuley, 2018) and BERT-based architectures (Sun et al., 2019) for session-based recommendation.

Revisitation. Web page revisitation behavior is prevalent on the web. Obendorf et al. (2007) suggested that people may revisit web pages to access the same resource again (e.g., for unfinished goals) or may re-access a resource as they expect changed content (e.g., for recurring goals, such as new headlines on a news site). Adar et al. (2008) combined quantitative and qualitative approaches and showed that people revisit web pages with varying speed. To help resume previously encountered information, Dontcheva et al. (2006) presented a web extension that allows people to extract entities of interest and creates an interactive summary of extracted entities (e.g., hotels that a person considers for a trip).

Grouping. Automatic managing and grouping the vast amount of unorganized web pages can help reduce the cognitive load required when navigating the web. A common strategy is to classify web pages (Baykan et al., 2011; Qi and Davison, 2009) by the encapsulated topics; popular class choices include the Open Directory Project, the Yahoo! Directory, categorized URLs from the Wikipedia, etc. To relax the requirement of pre-defined classes, clustering techniques have been studied to mine related web pages based on semantics or graph partitioning (Patel and Zaveri, 2011). Different from prior studies, we adopt the notion of goals as the basis to associate web pages for web page grouping.

In this work, we incorporate long-term goals to facilitate effective web browsing experiences, including recommendation, revisitation and goal-based grouping, which differ significantly from previous perspectives and approaches. We leverage Poincaré embeddings (Nickel and Kiela, 2017) to derive the goal representations, which are combined with state-of-the-art transformer-based architectures (Devlin et al., 2019) to model users’ web session behavior.

3. Representation Learning for Human Motives

Figure 1. Framework for learning intrinsic goal representations (iGoalRep) and training the goal estimator to predict visit goal representations (vGoalRep).

This section presents our approach to learning distributed intrinsic goal representations (iGoalRep ) for human motives, which are grounded by an expert-curated goal taxonomy in the psychology literature. Moreover, we propose a goal estimator to embed any web page visit in browsing sessions into the same goal embedding space as , and we denote them as visit goal representation (vGoalRep) . To learn the goal estimator, we collect and rely on a weak supervision dataset through the associations between related queries and web pages determined by the Bing search engine. Figure 1 summarizes the flow of the steps, and we describe the details of each step next.

3.1. Learning Intrinsic Goal Representations

Figure 2. Illustration of the constructed 3-layer goal taxonomy with 166 nodes.

Taxonomy of Human Goals. We leverage the hierarchical human goal taxonomy curated by Chulef et al. (2001) to form the basis of the goal space. Specifically, the taxonomy leaf nodes were curated based on an aggregation of psychology literature (Maslow, 1970; Murray, 1938; Rokeach, 1973; Wicker et al., 1984) and the responses from participants of diverse background, followed by a similarity-based sorting process to create the hierarchy relationship. The resulting set consists of 135 leaf goal nodes descended from 30 parent categorical nodes. We further introduce a “non-goal” node as the root that covers all the categorical nodes. Conceptually, the root represents prosaic behavior that might not be tied with specific goals; examples could include random distractions on the web or one-time matters. Figure 2 illustrates the final adopted human goal taxonomy consisting of 166 nodes in three layers. Formally, denotes the set of goals in the goal taxonomy and denotes the hierarchical structure.

Representation Learning in Hyperbolic Space. To induce the structural bias , we propose to learn intrinsic goal representations (iGoalRep) in a hyperbolic space. Hyperbolic geometry brings the advantage of learning compact representations that capture both hierarchy and similarity, which, for our case, is desirable as we aim to preserve the hierarchical properties of the goal taxonomy.

Concretely, we adopt Poincaré embeddings (Nickel and Kiela, 2017) to derive -dimensional iGoalRep for each . Let be an open Poincaré ball, where indicates the Euclidean norm. In contrast to Euclidean distance, the hyperbolic distance between any two points is given as:

Subsequent representation learning is then guided by this distance measure where a similar pair of goals should be closer to each other than a remote pair.

Reconstruction Optimization. To start learning, we treat the edges in the goal taxonomy as transitive closure and reconstruct the relationship by minimizing the Poincaré distances between the embeddings of their endpoints. Following prior work (Nickel and Kiela, 2017)

, we define the loss function

as:

where denotes a set of negative examples333In practice, we randomly sample 50 negative examples for each positive example. for and contains symmetric relationship. We adopt RSGD (Bonnabel, 2013), a stochastic Riemannian optimization method, to learn goal embeddings within the Poincaré ball as in (Nickel and Kiela, 2017). To do so, we need to rely on the Riemannian manifold structure of the Poincaré Ball when minimizing

. Since Euclidean gradient is not directly applicable in hyperbolic space, rescaling gradients depends on the Riemannian metric tensor

equipped in the Poincaré ball: where ; denotes the Euclidean metric tensor.

Figure 3. Illustration of learned 2-dimensional Poincaré goal representations with the 3-layer goal taxonomy from (Chulef et al., 2001). The center red node indicates the concept of “non-goal,” while the blue nodes represent goal categories that cover more specific goals presented as orange nodes.

Figure 3 visualizes the resulting iGoalRep with two dimension444We select dim=2 for visualization convenience here; in the actual experiments, dim=64 is used as described in Section 5. learned from the goal taxonomy. The results demonstrate that the learned embeddings preserve the desired hierarchical properties and align with the original goal taxonomy curated by human.

3.2. Estimating Visit Goal Representations (vGoalRep)

When a person visits a web page, we aim to predict a visit goal representation (vGoalRep) such that

can best reflect which goals may have driven the visit. To do so, we first build a weak supervision data collection that provide the association information of what the most probable goals are when web pages are being visited. Then, a parameterized goal estimator is introduced to predict

based on the weak labels according to a multi-class classification objective.

Weak Supervision Data Collection. Inspired by past work (Strohmaier and Kröll, 2012) that demonstrated search engine queries can contain explicit user objectives (e.g., get rid of belly fat), we first reify every goal in the goal taxonomy with a set of seed queries. For example, we manually generate queries such as “how to be charismatic” and “how to meet new friends” to elicit the high-ordered goal of being likeable, making friends, drawing others near. Then we expand the seed query set with related queries by requesting publicly available Microsoft Bing search API. In total, 300 seed queries and 1,932 related queries represent 165 defined human goals. The last step involves querying the same API using the enlarged query set together with the original goals; we then keep the top 5 returned web pages as the weakly-positive instances for the corresponding goal. For the concept of “non-goal,” i.e., the root of the goal taxonomy, we randomly select 1,000 web page visits from general web browsing logs as “negative examples”. Here we denote this data collection as .

Goal Estimator Construction. Next, we aim to devise a goal estimator that can predict vGoalRep by projecting each web page to the same goal embedding space in which exists. A web page visit is commonly characterized by the semantics conveyed via its textual content , as well as the website  where it is hosted (e.g., pages one the same site often demonstrate higher associations). To build an effective goal estimator, we take as input the two sources of information,  (), and estimate vGoalRep by:

where projects the host to a -dimensional embedding space; is a contextualized content encoder, such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019); as a fully-connected hidden layer derives ultimate -dimensional estimated goal embeddings. The output of the goal estimator, , can be regarded as a continuous representation of certain underlying motives for visiting a particular web page .

Multi-class Goal Classification. While the architecture of goal estimator is the same as web page encoder, we learn the parameters using a multi-class goal classification objective guided by . The multi-class classification loss function is defined by categorical cross-entropy (Goodfellow et al., 2016) as  : where is an indicator function; is the labeled goal of the web page ; indicates the softmax function calculating the probabilistic distribution over goals in the goal taxonomy as: Here denotes the similarity score between the web page and the goal

in the Poincaré embedding space. In other words, by treating the Poincaré similarity scores as classification logits,

is trained to reflect its ground-truth goal embedding . Formally: where and indicate the angles of embeddings in the hyperbolic coordinate system.

A final remark to note is that we could derive the most probable goal associated with a page based on the learned , by simply calculating the similarity between and : When designing GoWeB in Section 4, we choose to incorporate rather than because a continuous representation can encode more information and reduce sparsity compared to a discrete one.

4. Goal-directed Web Browsing (GoWeB)

Figure 4. Overview of GoWeB. The goal estimator produces goal representations , which are then used to derive goal-aware visit and session representations, as well as personal goal representation . These representations facilitate goal-directed experiences in browser applications. The details of the goal estimator are described in Section 3.

Based on the goal estimator and visit goal representations introduced in Section 3, we present a unified framework, Goal-directed Web Browsing (GoWeB), to assist common activities that people perform when browsing the web. We first architect a generic neural framework that can encapsulate information incurred in browsing using goal-aware representations. Second, we employ this framework to assist people to advance their goals through in-session recommendation, to pick up their goals through re-visitation prediction, and to focus on certain goals through goal-based clustering. Figure 4 further illustrates an overview of GoWeB.

4.1. Goal-aware Browsing Session Modeling

A web browsing session involves three key actors: individual web pages that a person visits, the sequence of visits in the same session, and the person who performs these visits. To support goal-directed browsing, GoWeB needs to derive representations that effectively incorporate goal awareness for each of these. More concretely, suppose a web browsing session consists of web page visits performed by a person . In the below, we describe the steps to form individual goal-aware visit representation (gVisitRep) as , goal-aware session representation (gSessionRep) as , and personal goal representation (pGoalRep) as .

Goal-aware Visit Representation (gVisitRep). To represent a web page visit, we consider two types of essential signals, namely the lexical content expressed inside the page as well as the probable goals accounting for the visit. Specifically, we derive content-based embeddings by initializing a separate set of weights using the same encoder architecture described in Section 3.2. For each visit , the content embedding is determined by its underlying content and the host of the page . Formally, we compute content embedding by where projects the host to a -dimensional embedding space; is a contextualized language model; as a fully-connected hidden layer derives the final -dimensional content embedding. While the content-based embeddings are important, we take a step further to incorporate probable goals for the act of visiting, by concatenating the vGoalRep  and the content embedding as Thus, visits in the same session can be denoted as , where is the number of pages and .

To make page visits sensitive to the session context in which they appear, we propose to apply the multi-head attention mechanism (Vaswani et al., 2017) to derive contextualized visit representations. Formally, for each web page visit, the gVisitRep can be computed as:

where ; is the number of attention heads; , ; and are the dimension numbers of attention keys and values; .

Goal-aware Session Representation (gSessionRep).

To represent the overall browsing session, we learn a context vector

for each head to dynamically estimate the importance of each visit for a specific application. The gSessionRep of the browsing session can then be derived as where ; , ,

Personal Goal Representation (pGoalRep). As an individual’s past goals could be predictive of their future goals or behavior, conventionally it is a common practice to rely on an individual’s past browsing activities to derive their personal representation (Jiang et al., 2020). We argue that it is equally important to account for the activities in a currently active session, according to Shah et al. (2002) that the activation of a given focal goal tends to result in an inhibition of alternative goals. To implement this idea, GoWeB constructs pGoalRep for an individual by aggregating the current session activities as a query to attend to past activities of the same individual. Suppose denotes the vGoalReps for all page visits in a current session; is the set of web pages visited in the past by a user . We adopt the Luong’s attention mechanism (Luong et al., 2015) to aggregate current session activities as:

where is the weight for ; is the context vector to estimates the importance of each visit; is a fully-connected hidden layer. The pGoalRep is computed by using as a query to discover and aggregate past goals related to current session:

where is the weight for each historical visit.

To summarize, we have described how to create goal-aware representations for a page visit (), a browsing session (), and an individual acting in the session (). These representations can, when designed properly, be used for directing people towards their goals while they browse the web. In particular, we consider three opportunities for goal-aware assistance described as follows.

4.2. Task 1: In-session Web Page Recommender

Making progress towards goals when navigating the web can be challenging, which could be due to unfamiliarity with subject matter or other external factors. To address this, GoWeB aims to recommend most goal-related, unseen resources that can help people advance their goals. Concretely, GoWeB formulates an in-session web page recommender. Given preceding web pages visited in a session, GoWeB ranks web pages from a candidate set according to their likelihood of being visited later in the same session. We follow prior work (Covington et al., 2016) and cast recommendation as a classification task. Based on the information available in , we combine the gSessionRep and pGoalRep as a feature vector. The ranking scores of each web page in can then be calculated as classification logits: where is a fully-connected hidden layer; projects the hidden state to the ranking scores of candidate web pages. Finally, the ranked list of recommendations can be generated by retrieving candidates with top ranking scores.

4.3. Task 2: Re-visitation Prediction

Web page re-visitation is prevalent as people may visit the same resources for unfinished or recurring goals. The ability to forecast potential future re-visitations can be useful for supporting people resuming their goals when predicted correctly. For instance, the predictions can be stored in the browser backend and resurface to users when they start a new session as a reminder. To support this, GoWeB aims to predict whether any of web pages in a current session will be revisited in a future session. For each visit in the session , we first construct its feature vector by concatenating the gVisitRep and the pGoalRep . The probability of the web page being re-visited can then be estimated as:

where is a fully-connected hidden layer;

projects the hidden state to a logit so that the sigmoid function

can derive a probabilistic score. We note that for the first and second tasks, GoWeB is trained end-to-end to directly optimize the target objective.

4.4. Task 3: Goal-based Web Page Grouping

Another common challenge with web browsing is being able to rationalize with goals and focus on subsets of web pages by goals. Considering that people often visit multiple, potentially diverse web pages in a session, this may create high cognitive load when people switch in-between goals. For this, we employ GoWeB to group in-session web pages by goals, such that the resulting groups of pages can be used to categorize ongoing flows and help people concentrate on certain goals of choice. Specifically, we cast this as a clustering task where the vGoalRep of each visit is used as features, based on which subsequent feature-based clustering algorithms can be applied.

5. Experiments

This section examines the effect of incorporating high-ordered goals into the context of web browsing. We compare each of the browser-centric tasks with competitive baselines and demonstrate how GoWeB can enhance multiple browsing experiences.

5.1. Experimental Setup

Experimental Datasets. Our main experimental dataset, denoted as , consists of web browsing sessions that are used for evaluating the three goal-based applications supported by GoWeB. was constructed by randomly sampling the anonymized logs of Microsoft Edge web browser555To the best of our knowledge, unfortunately, there is no publicly available user web browsing dataset that is suitable for evaluating our hypothesis and framework, but we will release our implementation to facilitate the community development.. A session is composed of a sequence of web page visits of a user, where a boundary is found between two consecutive visits that are at least 30 minutes apart. The logs contain records of web pages visits assembled by sessions, where each page comes with a host, a title and a timestamp when the page was visited; the titles are used as the web page content. The data was sampled from June 2020, where the training and test sets were respectively gathered from the periods of June 1st to 23rd and June 24th to 30th. To avoid tail behavior, a web page is discarded if it appears less than 10 times in the sampling period, and short sessions with less than 10 page visits are disregarded as the associated goals are likely to be simpler. In total, our dataset contains web pages originating from 79,695 unique hosts (websites).

# of Users # Sessions Avg. Session Length
Training 57,715 836,073 21.689 11.922
Test (Warm) 6,701 38,737 22.257 12.124
Test (Cold) 6,062 75,339 22.076 12.127
Table 1. Statistics of web session experimental datasets.

We prepare two disjoint test sets to study the effect of incorporating the notion of personal goals. The warm-start sample includes users whose behavior can be found in the training period, while the cold-start sample draws from users new to the system. The statistics of used for the experiments are shown in Table 1. For cold-start users, since we do not have access to their past behavior, we exclude the personal goal representations from the framework and denote this variant as GoWeB (NP).

Pop BPRMF NCF Caser GRU4Rec SASRec BERT4Rec SemRec GoWeB (NP) GoWeB
MRR@10 0.0386 0.0484 0.0537 0.0506 0.0533 0.0693 0.0727 0.0817 0.0968 (+18.52%) 0.1080 (+32.22%)
HR@1 0.0169 0.0243 0.0259 0.0216 0.0269 0.0365 0.0382 0.0472 0.0581 (+23.24%) 0.0674 (+42.96%)
Test HR@5 0.0618 0.0751 0.0864 0.0870 0.0837 0.1093 0.1150 0.1252 0.1445 (+15.38%) 0.1585 (+26.53%)
(Warm) HR@10 0.1123 0.1243 0.1365 0.1329 0.1445 0.1681 0.1748 0.1801 0.2090 (+16.05%) 0.2276 (+26.38%)
NDCG@5 0.0202 0.0276 0.0295 0.0278 0.0303 0.0398 0.0418 0.0476 0.0563 (+18.13%) 0.0626 (+31.34%)
NDCG@10 0.0292 0.0354 0.0373 0.0363 0.0406 0.0503 0.0521 0.0571 0.0676 (+18.33%) 0.0751 (+31.43%)
MRR@10 0.0454 0.0513 0.0528 0.0508 0.0559 0.0675 0.0683 0.0856 0.0993 (+16.09%) N/A
HR@1 0.0215 0.0234 0.0268 0.0251 0.0267 0.0353 0.0337 0.0499 0.0597 (+19.66%) N/A
Test HR@5 0.0752 0.0882 0.0884 0.0829 0.0937 0.1034 0.1094 0.1350 0.1496 (+10.83%) N/A
(Cold) HR@10 0.1206 0.1355 0.1269 0.1340 0.1493 0.1696 0.1761 0.1874 0.2150 (+14.68%) N/A
NDCG@5 0.0252 0.0318 0.0316 0.0280 0.0335 0.0392 0.0405 0.0518 0.0598 (+15.30%) N/A
NDCG@10 0.0341 0.0398 0.0383 0.0375 0.0435 0.0505 0.0520 0.0611 0.0712 (+16.50%) N/A
Table 2. Performance of methods in session-based web page recommendation. Note that cold-start users do not have personalized historical goal embeddings from training data, so GoWeB with personalization is not available for the cold-start testing dataset. GoWeB (NP) denotes the non-personalized version of GoWeB.

Implementation Details.

We implement GoWeB and baseline methods with the PyTorch framework 

(Paszke et al., 2019). The dimensions of goals and hosts (i.e., and ) are set to 64, while the number of hidden units in fully-connected layers and the dimension of the remaining embeddings are set to 128. The number of attention heads is set to 8. We use a learning rate of 0.3 for RSGD (Bonnabel, 2013) in the Riemannian optimizer, while a learning rate and are used in the Adam optimizer (Kingma and Ba, 2014) for optimizing the supervised goal-based applications and the goal estimator. For contextualized language modeling (i.e., the BERT functions in Section 4 and 3), we use the pre-trained RoBERTa (Liu et al., 2019) models provided by HuggingFace (Wolf et al., 2019)

. We will open-source our implementation.

5.2. Web Page Recommendation Performance

Recall that the in-session recommender predicts goal-related web pages that are likely to be visited later in the same session. Specifically, for a session, we take the first half of page visits as observed input information and use the second half as ground-truth to evaluate our model. The ranking candidates of web pages are generated by removing the top 10 popular web pages (e.g., msn.com and google.com) and selecting the top 50,000 frequent web pages666The number was selected in accordance with the prior study  (Adar et al., 2008) that analyzed large-scale web data. from the remaining set. For both test sets, we assert that the ground-truth (i.e., the second half) of every session contains at least one web page from the candidate set. We treat the task as a retrieval problem and evaluate it by conventional metrics, including MRR@10, HR@{1, 5, 10}, and NDCG@{5, 10}. We compare our method with state-of-the-art top- session-based recommenders, including popularity (Pop), BPRMF (Rendle et al., 2009), NCF (He et al., 2017), Caser (Tang and Wang, 2018), GRU4Rec (Hidasi et al., 2016), SASRec (Kang and McAuley, 2018), and BERT4Rec (Sun et al., 2019). We further consider a modern content-based modeling method that incorporates web page semantics using BERT and contextualization. We implement this by reusing the GoWeB neural architecture but with all the goal-awareness removed (i.e. excluding and any representations derived based on ), which is denoted as SemRec.

Table 2 demonstrates the recommendation performance in two test sets. Among the baseline methods, SemRec performs better than conventional session-based recommenders that disregard web page content. It advocates for the importance of modeling the semantics of web pages, as the open-domain nature of the web can make recommendation more challenging than domain-specific applications (e.g. movies). Without personalization, GoWeB (NP) performs the best and outperforms SemRec by 18.52% and 16.09% in MRR@10 for two test sets. This empirical finding sheds light on the opportunity of modeling human motives and higher-level goals carried by the goal estimator without accessing historical data, since only the in-session page visits are considered. For the warm-start test set where access to past page visits is available, GoWeB can further provide more satisfactory recommendations. Overall, the results suggest that incorporating people’s motives can lead to better recommendation in web browsing sessions.

5.3. Re-visitation Prediction Performance

For re-visitation prediction, given a browsing session, the task is formulated as a binary classification problem to predict if each web page visit in the session will be re-visited by the same user in any future session. Similarly, we consider two families of baseline methods. The conventional sequence modeling methods learn an embedding vector for each unique web page without using web page content, including CNN, RNN, SAS (Kang and McAuley, 2018), BERT (Devlin et al., 2019; Liu et al., 2019). To account for page content, we likewise build a strong semantic classifier, SemCLS, based on the sequence modeling component embedded in GoWeB without using any goal-related representations. We utilize common binary classification metrics, including F1-score, precision, recall, and accuracy, for evaluation.

F1 Precision Recall Accuracy
CNN 0.5249 0.4818 0.5765 0.6066
RNN 0.5387 0.6269 0.4722 0.6951
Test SAS 0.5302 0.6376 0.4537 0.6969
(Warm) BERT 0.5948 0.5479 0.6505 0.6659
SemCLS 0.6718 0.6155 0.7394 0.7277
GoWeB (NP) 0.6874 0.6247 0.7642 0.7380
GoWeB 0.7010 0.6234 0.8007 0.7425
CNN 0.5454 0.5271 0.5650 0.6075
RNN 0.5369 0.6826 0.4425 0.6819
Test SAS 0.5290 0.6956 0.4268 0.6833
(Cold) BERT 0.6127 0.6093 0.6160 0.6754
SemCLS 0.6824 0.6829 0.6820 0.7355
GoWeB (NP) 0.7050 0.6984 0.7117 0.7518
Table 3. Performance of methods in re-visitation classification. Similar to Table 2, GoWeB with personalization is N/A for cold-start users. GoWeB (NP) denotes the non-personalized version of GoWeB.

Table 3 shows the classification performance of methods in two test sets. Consistent with the results in Table 2, SemCLS that incorporates web page content is the best-performing baseline method. Compared to SemCLS, GoWeB (NP) respectively provides 2.32% and 3.30% gains in F1 scores for the two test sets. Similar to in-session recommender, GoWeB introduces further gains when using the focal goals to attend to past goals. These results indicate that the GoWeB framework can provide goal-aware assistance in supervised ranking and classification tasks.

5.4. Goal-based Grouping Performance

The objective of the web page grouping task is to create coherent clusters of web pages inside a session in an unsupervised fashion. We treat the goal representations derived in GoWeB as features and apply -means++ algorithm (Arthur and Vassilvitskii, 2006) to cluster page visits into different groups. We consider two popular text modeling methods, doc2vec (Le and Mikolov, 2014) and BERT (Devlin et al., 2019), as the baselines where the corresponding embedding vectors are used for clustering.

To create ground-truth for evaluation, we leverage the outcome of a proprietary hierarchical topical classifier as references that different methods can compare to. The proprietary classifier predicts the topics of web pages according to a 3-layer taxonomy consisting of a root node, 22 category nodes, and 288 leaf nodes. This setup resembles the traditional web page classification task (Baykan et al., 2011; Qi and Davison, 2009) where a topical organization of the web was found to be useful. Once we obtain the outcome topics, the number of ground-truth clusters in each session is also determined accordingly. Note that, following previous studies (Yang et al., 2019; Law et al., 2017), our focus is to evaluate the clustering quality using different embedding features. We acknowledge past work on determining the optimal number of clusters (Kodinariya and Makwana, 2013) as a different research problem.

doc2vec BERT GoWeB (Goal Estimator)
NMI 0.7290 0.7612 0.7683 (+0.93%)
AMI 0.4320 0.4929 0.5155 (+4.59%)
Table 4. Performance of methods in goal-based clustering.

Table 4 shows the resulting grouping performance on NMI and AMI metrics (Manning et al., 2008); here we combine the two test sets together since this is an unsupervised task. BERT, as is powered by a decent pretrained contextualized language model, performs better than doc2vec. Meanwhile, the goal representations generated by goal estimatoroutperforms both baselines, suggesting its efficacy for tackling unsupervised tasks. Figure 5 further shows the percentage of clusters over the number of unique ground-truth topics included in a cluster. An ideal cluster should contain exactly one topic.

Figure 5. The percentage of clusters over different numbers of incorporated unique topics.

The results show that GoWeB tends to derive purer clusters with fewer unique classes compared to baselines. It may suggest that the goal-aware representations are also more topically coherent.

6. Analysis and Discussions

In section 5, we demonstrate empirically the effectiveness of GoWeB when applying to browser applications. Now we turn our attention to deepen our understanding for the effectiveness of the goal estimator, and how the variety of goals may affect the differences observed in people’s behavioral patterns.

Figure 6.

The probabilistic confusion matrix for our goal estimator in goal classification. The grid partitions goals into categories. The “non-goal” class has goal ID 1.

Effectiveness of Goal Estimator. To uncover the details, we scrutinize the goal estimator as it powers the prediction for goal representations when people make visits to web pages. Recall that the goal estimator is learned with a multi-class objective, where a 136-way classification is carried out to predict among the 135 leaf goals plus a non-goal class. We use 90% of weakly-labeled instances from for training, and evaluate on the remaining 10%. As a result, we achieve 55.27% and 83.01% F1 scores for predicting individual goals and goal categories. Figure 6 depicts the confusion matrix for our goal estimator, where the grid partitions leaf goals into goal categories. We can see that most of the predictions are aligned with the diagonal as correct predictions; among other cases, most of the misclassifications are in the same grid block, indicating that errors are bounded within the same goal category.

(a)
(b)
Figure 7. (a) The percentage of being in a web browsing session with a single goal for each goal category. (b) Percentage of re-visits within different duration interval buckets (Adar et al., 2008) and three duration scales for goal categories. Note that each line represents the statistics of goals in a certain category.

We further conduct a lightweight qualitative exercise to analyze the relations between hosts and predicted goals. We first identify top hosts that cover a higher proportion of web pages being predicted closest to each goal category. Two annotators were then asked to judge whether these hosts are topically relevant to the corresponding categories. Table 5 shows the assessment results with 100% agreement from the two annotators. The assessments suggest that the goal estimator can make reasonable predictions in the majority of cases.

Goal Category Top 1 Top 2 Top 3
Appearance alphacute.com (✓) interesticle.com (✓) trend-chaser.com (✓)
Career indeed.com (✓) careerbuilder.com (✓) linkedin.com (✓)
Entertainment en.wikipedia.org gocomics.com (✓) nytimes.com
Finances cnbc.com (✓) zillow.com (✓) realtor.com (✓)
Order microsoft.com (✓) indeed.com lowes.com (✓)
Physical Health healthygeorge.com (✓) myfitnesspal.com (✓) financialadvisorheroes.com
Religion biblegateway.com (✓) kingjamesbibleonline.org (✓) bibletrivia.com (✓)
Table 5. Top 3 hosts whose web pages are more likely to belong to some goal categories. Hosts with (✓) are labeled as relevant websites to the corresponding categories. The goal “Order” means “(Keep Things in) Order.”

Characterization of In-session Goals. To start, we note that people tend to involve in multiple goals in a single session; for example, the average number of goals can be more than 5 for sessions containing 10 page visits. We also find that the increase in the number of goals pursued is sub-linear to the session length; for example, sessions of 60 page visits are associated with fewer than 14 goals on average. Given that people may seek multiple goals in a session, to which degree each goal category could be considered as the session’s focal goal? Figure 6(a) illustrates the percentage of being the only goal in a single session. On one hand, for sessions that contain a Finances goal, about 16% of the times people concentrate solely on this goal category. We conjecture that it might be related to the time-dependent sensitivity required in the process of financial decision making; further studies need to be conducted in the future. On the other hand, we find that more nebulous goals such as Ethics & Idealism are seldom being pursued singularly (i.e., only 3% of the times). The results suggest people behave differently according to the types of goals, and fathoming the underlying objectives is key to providing delightful web browsing experiences. For example, enabling a focus mode on detecting goals that require high concentration, or proactively recommending new resource for more exploratory goals could be new designs to consider.

Characterization of Revisitation Patterns by Goals. People’s goals that motivate visits to certain web pages may also have an impact on how these pages are revisited. We analyze temporal cross-session revisitation patterns. A revisit is identified by two consecutive visits of the same web page, where the elapsed time between the former visit and the latter is considered as the duration. Following Adar et al. (2008), we group all collected revisitation patterns into 14 unequal-size buckets according to the length of duration. Figure 6(b) illustrates the distributions of revisits in different buckets for different goal categories. Interestingly, at a high level, all the 30 goal categories exhibit similar revisiting patterns. Most of the revisits (37%) occur roughly after a day or within few days, while between 23% to 36% of revisits occur within 7 hours. We observe far less revisiting after a week has passed. This could be due to the limitation of our data being sampled from a month period.

Rank Duration Scale
Hours Days Weeks
1 Friendship Romance (Keep Things in) Order
2 Appearance Social Qualities Receiving from Others
3 Entertainment Career Career
4 Social Recognition Self-sufficiency Stability & Safety
5 Marriage Religion Social Qualities
Table 6. Top 5 goal categories with different duration scales in re-visitations.

Beyond the distributional similarity at a macro level, we observe distinct patterns in the top 5 goal categories for three revistation duration scales in Table 6. The results suggest that goal categories concerning social interactions (Friendship, Social Recognition, and Marriage), addictability (Entertainment), and timeliness (Appearance) can lead to relatively quicker revisits within hours. It may imply that inter-personal relationships tend to urge people to voluntarily stay up-to-date, potentially due to the desire for intimacy or attractive looks, or the fear of missing out. On the contrary, goal categories (Career development, Self-sufficiency, and Social Qualities) that demand continuous investment associate more frequently with slower revisits. More abstract, higher-ordered goals, such as Order, Receiving from Others and Stability & Safety may result in the slowest revisitation compared to more concrete goals such as Entertainment. This highlights the opportunities for supporting people browsing the web, where an ambient reminder for a potential revisit can reduce the overhead of manual retrieval.

7. Conclusion

In this paper, we highlight the importance of modeling human motives grounded by the long history of psychology literature. A unified neural framework, GoWeB, is presented to fulfill this vision. We build on the top of existing taxonomy and concertize these goals with structure-preserving representation learning in hyperbolic space, based on which a goal estimator is introduced to tighten the loop of how goals could be employed for enhancing browsing experiences. We showcase the generality of GoWeB and adopt it in three browser-centric applications. On real-world data, GoWeB consistently outperforms competitive baselines for both warm-start and cold-start users, and demonstrates additional gains when using the focal goals to attend to past goals. Our follow-up analysis reveals the effectiveness of the goal estimator via quantitative and qualitative exercises, and characterizes the similarities and differences found in behavioral patterns when people pursue different goals.

Our work brings new perspectives in multiple ways. We present a promising paradigm where we capture the fundamental motives that drive people in their actions and reflect those in digital applications. Broadly speaking, we introduce and transfer the knowledge from psychology findings to modeling browsing sessions on the web, while keeping the framework flexible such that future research could incorporate other types of taxonomies. It is our hope that these findings can lift the burden of understanding and characterizing complex human goals for ubiquitous web browsing applications.

References

  • J. Aagaard (2015) Drawn to distraction: a qualitative study of off-task use of educational technology. Computers & Education 87, pp. 90–97. Cited by: §2.
  • E. Adar, J. Teevan, and S. T. Dumais (2008) Large scale analysis of web revisitation patterns. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’08, pp. 1197–1206. Cited by: §2, Figure 7, §6, footnote 6.
  • R. F. Adler and R. Benbunan-Fich (2013) Self-interruptions in discretionary multitasking. Computers in Human Behavior 29 (4), pp. 1441–1449. Cited by: §2.
  • D. Arthur and S. Vassilvitskii (2006) K-means++: the advantages of careful seeding. Technical report Stanford. Cited by: §5.4.
  • D. Barua, J. Kay, B. Kummerfeld, and C. Paris (2014) Modelling long term goals. In User Modeling, Adaptation, and Personalization, pp. 1–12. Cited by: §1, §1.
  • E. Baykan, M. Henzinger, L. Marian, and I. Weber (2011) A comprehensive study of features and algorithms for url-based topic classification. ACM Trans. Web 5, pp. 15:1–15:29. Cited by: §2, §5.4.
  • S. Bonnabel (2013) Stochastic gradient descent on riemannian manifolds. IEEE Transactions on Automatic Control 58 (9), pp. 2217–2229. Cited by: §3.1, §5.1.
  • A. Broder (2002) A taxonomy of web search. SIGIR Forum 36 (2), pp. 3–10. Cited by: §1, §2.
  • L. Caruccio, V. Deufemia, and G. Polese (2015) Understanding user intent on the web through interaction mining. Journal of Visual Languages & Computing 31, pp. 230 – 236. Note: Special Issue on DMS2015 Cited by: §1, §2.
  • A. S. Chulef, S. Read, and D. Walsh (2001) A hierarchical taxonomy of human goals. Motivation and Emotion 25, pp. 191–232. Cited by: §1, §2, Figure 3, §3.1, footnote 2.
  • A. S. Chulef (1993) A hierarchical taxonomy of human goals. Dissertation. Cited by: §1.
  • P. Covington, J. Adams, and E. Sargin (2016) Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, pp. 191–198. Cited by: §4.2.
  • M. Czerwinski, E. Horvitz, and S. Wilhite (2004) A diary study of task switching and interruptions. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 175–182. Cited by: §2.
  • H. Dai, L. Zhao, Z. Nie, J. Wen, L. Wang, and Y. Li (2006) Detecting online commercial intention (oci). In Proceedings of the 15th international conference on World Wide Web, pp. 829–837. Cited by: §1.
  • J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Cited by: §1, §2, §3.2, §5.3, §5.4.
  • A. Dix, G. Lepouras, A. Katifori, C. Vassilakis, T. Catarci, A. Poggi, Y. Ioannidis, M. Mora, I. Daradimos, N. M. Akim, et al. (2010) From the web of data to a world of action. Journal of Web Semantics 8 (4), pp. 394–408. Cited by: §2.
  • M. Dontcheva, S. M. Drucker, G. Wade, D. Salesin, and M. F. Cohen (2006) Summarizing personal web browsing sessions. UIST ’06, pp. 115–124. Cited by: §2.
  • A. Faaborg and H. Lieberman (2006) A goal-oriented web browser. In CHI, Cited by: §1, §2.
  • T. Gillie and D. Broadbent (1989) What makes interruptions disruptive? a study of length, similarity, and complexity. Psychological research 50 (4), pp. 243–250. Cited by: §2.
  • J. Glassman, M. Prosch, and B. B. Shao (2015) To monitor or not to monitor: effectiveness of a cyberloafing countermeasure. Information & Management 52 (2), pp. 170–182. Cited by: §2.
  • M. Göksedef and S. G. Ögüdücü (2010) Combination of web page recommender systems. Expert Syst. Appl. 37, pp. 2911–2922. Cited by: §2.
  • I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio (2016) Deep learning. Vol. 1, MIT press Cambridge. Cited by: §3.2.
  • X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua (2017) Neural collaborative filtering. Proceedings of the 26th International Conference on World Wide Web. Cited by: §2, §5.2.
  • B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk (2016)

    Session-based recommendations with recurrent neural networks

    .
    CoRR abs/1511.06939. Cited by: §2, §5.2.
  • J. Jiang, T. Wu, G. Roumpos, H. Cheng, X. Yi, E. Chi, H. Ganapathy, N. Jindal, P. Cao, and W. Wang (2020) End-to-end deep attentive personalized item retrieval for online content-sharing platforms. In Proceedings of The Web Conference 2020, pp. 2870–2877. Cited by: §4.1.
  • R. Jones and K. L. Klinkner (2008) Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM ’08, pp. 699–708. Cited by: §2.
  • W. Kang and J. McAuley (2018) Self-attentive sequential recommendation. 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206. Cited by: §2, §5.2, §5.3.
  • D. P. Kingma and J. Ba (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. Cited by: §5.1.
  • E. Klinger (1987) The interview questionnaire technique: reliability and validity of a mixed idiographic-nomothetic measure of motivation. J. N. Butcher & C. D. Spielberger (Eds.), Advances in personality assessment 6, pp. 31–48. Cited by: §2.
  • T. M. Kodinariya and P. R. Makwana (2013) Review on determining number of cluster in k-means clustering. International Journal 1 (6), pp. 90–95. Cited by: §5.4.
  • R. Kumar and A. Tomkins (2010) A characterization of online browsing behavior. In Proceedings of the 19th International Conference on World Wide Web, WWW ’10, pp. 561–570. Cited by: §2.
  • E. Law and H. Zhang (2011) Towards large-scale collaborative planning: answering high-level search queries using human computation. In

    Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence

    ,
    AAAI’11, pp. 1210–1215. Cited by: §2.
  • M. T. Law, R. Urtasun, and R. S. Zemel (2017)

    Deep spectral clustering learning

    .
    In

    International Conference on Machine Learning

    ,
    pp. 1985–1994. Cited by: §5.4.
  • Q. Le and T. Mikolov (2014) Distributed representations of sentences and documents. In International conference on machine learning, pp. 1188–1196. Cited by: §5.4.
  • U. Lee, Z. Liu, and J. Cho (2005) Automatic identification of user goals in web search. In Proceedings of the 14th International Conference on World Wide Web, WWW ’05, pp. 391–400. Cited by: §1, §2.
  • X. Li, Y. Wang, and A. Acero (2008) Learning query intent from regularized click graphs. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 339–346. Cited by: §1.
  • V. K. Lim (2002) The it way of loafing on the job: cyberloafing, neutralizing and organizational justice. Journal of organizational behavior: the international journal of industrial, occupational and Organizational Psychology and Behavior 23 (5), pp. 675–694. Cited by: §2.
  • Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. Cited by: §3.2, §5.1, §5.3.
  • M. Luong, H. Pham, and C. D. Manning (2015)

    Effective approaches to attention-based neural machine translation

    .
    In

    Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

    ,
    pp. 1412–1421. Cited by: §4.1.
  • C. D. Manning, H. Schütze, and P. Raghavan (2008) Introduction to information retrieval. Cambridge university press. Cited by: §5.4.
  • A. H. Maslow (1970) Motivation and personality (2nd ed.). New York: Harper & Row. Cited by: §3.1.
  • W. McDougall (1933) The energies of men. New York: Scribner’s. Cited by: §2.
  • D. C. McFarlane and K. A. Latorella (2002) The scope and importance of human interruption in human-computer interaction design. Human-Computer Interaction 17 (1), pp. 1–61. Cited by: §2.
  • G. B. Moskowitz and H. Grant (2009) The psychology of goals. Cited by: §1.
  • H. A. Murray (1938) Explorations in personality : a clinical and experimental study of fifty men of college age. Cited by: §2, §3.1.
  • M. Nickel and D. Kiela (2017) Poincaré embeddings for learning hierarchical representations. NIPS’17, pp. 6341–6350. Cited by: §2, §3.1, §3.1.
  • B. O’Conaill and D. Frohlich (1995) Timespace in the workplace: dealing with interruptions. In Conference companion on Human factors in computing systems, pp. 262–263. Cited by: §2.
  • H. Obendorf, H. Weinreich, E. Herder, and M. Mayer (2007) Web page revisitation revisited: implications of a long-term click-stream study of browser usage. In CHI, Cited by: §2.
  • A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. (2019) Pytorch: an imperative style, high-performance deep learning library. In Advances in neural information processing systems, pp. 8026–8037. Cited by: §5.1.
  • D. Patel and M. Zaveri (2011) A review on web pages clustering techniques. Cited by: §2.
  • X. Qi and B. D. Davison (2009) Web page classification: features and algorithms. ACM Comput. Surv. 41, pp. 12:1–12:31. Cited by: §2, §5.4.
  • S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme (2009) BPR: bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI ’09, pp. 452–461. Cited by: §2, §5.2.
  • M. Rokeach (1973) The nature of human values. New York: Free Press. Cited by: §3.1.
  • D. E. Rose and D. Levinson (2004) Understanding user goals in web search. In Proceedings of the 13th International Conference on World Wide Web, WWW ’04, pp. 13–19. Cited by: §1, §2.
  • J. Y. Shah, R. Friedman, and A. W. Kruglanski (2002) Forgetting all else: on the antecedents and consequences of goal shielding.. Journal of personality and social psychology 83 (6), pp. 1261. Cited by: §4.1.
  • D. A. Smith and H. Lieberman (2010) The why ui: using goal networks to improve user interfaces. In Proceedings of the 15th international conference on Intelligent user interfaces, pp. 377–380. Cited by: §1.
  • M. Strohmaier and M. Kröll (2012) Acquiring knowledge about human goals from search query logs. Information Processing & Management 48 (1), pp. 63 – 82. Cited by: §3.2.
  • X. Su and T. M. Khoshgoftaar (2009) A survey of collaborative filtering techniques. Cited by: §2.
  • F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang (2019) BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer. CIKM ’19, pp. 1441–1450. Cited by: §2, §5.2.
  • J. Talevich, S. J. Read, D. A. Walsh, R. Iyer, and G. Chopra (2017) Toward a comprehensive taxonomy of human motives. PLoS ONE 12. Cited by: §2.
  • J. Tang and K. Wang (2018) Personalized top-n sequential recommendation via convolutional sequence embedding. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. Cited by: §2, §5.2.
  • J. Teevan, S. T. Dumais, and D. J. Liebling (2008) To personalize or not to personalize: modeling queries with variation in user intent. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 163–170. Cited by: §1.
  • V. W. Tseng, M. L. Lee, L. Denoue, and D. Avrahami (2019) Overcoming distractions during transitions from break to work using a conversational website-blocking system. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13. Cited by: §2.
  • A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. In NIPS, pp. 5998–6008. Cited by: §4.1.
  • F. Wicker, F. Lambert, F. Richardson, and J. Kahler (1984) Categorical goal hierarchies and classification of human motives. Journal of Personality 52, pp. 285 – 305. Cited by: §3.1.
  • T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. (2019) HuggingFace’s transformers: state-of-the-art natural language processing. ArXiv, pp. arXiv–1910. Cited by: §5.1.
  • X. Yang, C. Deng, F. Zheng, J. Yan, and W. Liu (2019)

    Deep spectral clustering using dual autoencoder network

    .
    In

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    ,
    pp. 4066–4075. Cited by: §5.4.
  • S. Zhang, L. Yao, A. Sun, and Y. Tay (2019) Deep learning based recommender system: a survey and new perspectives. 52 (1). Cited by: §2.