Click-through rate (CTR) prediction is to predict the probability that a user will click on an item. It plays an important role in online advertising systems. For example, the ad ranking strategy generally depends on CTR
bid, where bid is the benefit the system receives if an ad is clicked. Moreover, according to the common cost-per-click charging model, advertisers are only charged once their ads are clicked by users. Therefore, in order to maximize the revenue and to maintain a desirable user experience, it is crucial to estimate the CTR of ads accurately.
. One line of research is to take advantage of machine learning approaches to predict the CTR for each ad independently. For example, Factorization Machines (FMs)(Rendle, 2010)
are proposed to model pairwise feature interactions in terms of the latent vectors corresponding to the involved features. In recent years, Deep Neural Networks (DNNs) are exploited for CTR prediction and item recommendation in order to automatically learn feature representations and high-order feature interactions(Van den Oord et al., 2013; Zhang et al., 2016; Covington et al., 2016). To take advantage of both shallow and deep models, hybrid models are also proposed. For example, Wide&Deep (Cheng et al., 2016)
combines Logistic Regression (LR) with DNN, in order to improve both the memorization and generalization abilities of the model. DeepFM(Guo et al., 2017) combines FM with DNN, which further improves the model ability of learning feature interactions.
This line of research considers each ad independently, but ignores other ads that may impact the CTR of the target ad. In this paper, we explore auxiliary ads beyond the target ad for improving the CTR prediction (cf. Figure 1). In particular, we explore auxiliary ads from two viewpoints. One is from the spatial domain: we consider the contextual ads shown above the target ad on the same page111Contextual ads are available to advertising systems when ads are ranked sequentially. Please refer to §4.1 for more detail.. The intuition is that ads shown together may compete for a user’s attention. The second viewpoint is from the temporal domain: we consider the historically clicked and unclicked ads222Unclicked ads are ads that are shown to a user but not clicked by the user. They are not created by sampling from an ad pool excluding clicked ads. of the user. The intuition is that clicked ads reflect a user’s preferences and unclicked ads may indicate what a user dislikes to certain extent.
In order to effectively utilize these auxiliary data, we must address the following issues: 1) As the numbers of each type of auxiliary ads may vary, the model must be able to accommodate all possible cases. For example, there are 1 contextual ad, 2 clicked ads and 4 unclicked ads with target ad and there are 0 contextual ad, 3 clicked ads and 2 unclicked ads with target ad
. 2) As auxiliary ads may not be necessarily related to the target ad, the model should be able to distill useful information and suppress noise in auxiliary data. For example, the clicked ads are about coffee, clothing and car. Then which of these ads are more useful in predicting the target CTR if the target ad is about coffee? 3) The degree of influence of each type of auxiliary ad could be different and the model should be able to differentiate their contributions. For example, the importance of contextual ads and that of clicked ads should be treated differently. 4) The model should be able to fuse all the available information. To address these issues, we propose three variants of Deep Spatio-Temporal neural Networks (DSTNs) for CTR prediction. These variants include a pooling model, a self-attention model and an interactive attention model, with enhanced ability. The interactive attention model fully addresses the aforementioned issues. Both offline and online experimental results demonstrate the effectiveness of DSTNs for CTR prediction.
|Label||User ID||User Age||Ad Title|
|1||2135147||24||Beijing flower delivery|
|0||3467291||31||Nike shoes, sporting shoes|
|0||1739086||45||Female clothing and jeans|
The main contributions of this work are summarized as follows:
We explore three types of auxiliary data for improving the CTR prediction of the target ad. These auxiliary data include contextual ads shown above the target ad on the same page and historically clicked and unclicked ads of the user who views the target ad.
We propose DSTNs that effectively fuse these auxiliary data to predict the target CTR. The model is able to learn the interactions between auxiliary data and the target ad, and to emphasize more important hidden information. We make the implementation code publicly available333https://github.com/oywtece/dstn.
We conduct extensive offline experiments on three large-scale datasets from real advertisement systems to test the performance of DSTNs and several state-of-the-art methods. We also conduct case studies to provide further insights behind the model.
We have deployed the best-performing DSTN in Shenma Search, which is the second largest search engine in China. We have also conducted online A/B test to evaluate its performance in real-world CTR prediction tasks.
2. DSTN Models
In this section, we first introduce the CTR prediction problem and then present three variants of DSTN models.
The task of CTR prediction in online advertising is to build a prediction model to estimate the probability of a user clicking on a specific ad. Each ad instance can be described by multiple fields such as user information (“User ID”, “City”, “Age”, etc.) and ad information (“Creative ID”, “Campaign ID”, “Title”, etc.). The instantiation of a field is a feature. For example, the “User ID” field may contain features such as “2135147” and “3467291”. Table 1 shows some examples.
Classical CTR prediction models such as FM (Rendle, 2010), DNN (Zhang et al., 2016) and Wide&Deep (Cheng et al., 2016) mainly consider the target ad (illustrated in Figure 3(a)). The focus is on how to represent the ad instance in terms of informative features and on how to learn feature interactions. Differently, in this paper, we explore auxiliary data for improving the CTR prediction. We must address the following issues: 1) how to accommodate all different cases with varying numbers of auxiliary ads; 2) how to distill useful information and suppress noise in auxiliary ads; 3) how to differentiate the contributions of each type of auxiliary ads; 4) how to fuse all the available information.
Before we introduce any model structure, we first present the embedding process (layer) that is common in all the models below. The embedding process is to first map each feature into an embedding vector and then represent each ad instance as the concatenation of corresponding feature embedding vectors.
Denote the number of unique features as . We create an embedding matrix where each row represents a -dim embedding vector for a feature. Given a feature index such as , its embedding is then the -th row of the matrix . The embedding matrix is a variable to be learned during model training.
We segregate features into the following three groups, which are processed differently according to Figure 2.
1) Univalent categorical features. This type of feature contributes only a single value and “User ID” is an example (cf. Table 1). If we use the one-hot feature representation, the resulting feature vector is very sparse because the number of unique user IDs can be on the order of . We thus map sparse, high-dimensional categorical features to dense, low-dimensional embedding vectors suitable for neural networks. These embeddings also carry richer information than one-hot representations (Mikolov et al., 2013).
2) Multivalent categorical features. This type of feature contributes a set of values and the bag of bi-grams in the “Ad Title” is an example (cf. Table 1). To illustrate, the bi-grams of title “ABCD” are “AB”, “BC” and “CD”. As the set cardinality may vary, we first map each value in the set to an embedding vector and then perform sum pooling to generate an aggregated vector of fixed length.
3) Numerical features. “User Age” is an example of numerical features (cf. Table 1). Each numerical feature is first quantized into discrete buckets, and is then represented by the bucket ID. Each bucket ID is mapped to an embedding vector.
The representation of an ad instance after the embedding process is the concatenation of all the embedding vectors, each for a field (cf. Figure 2). Note that the fields used for each type of ads may be different. For example, the fields for the target ad contain user information such as “User ID” and “Age”, while the fields for the clicked ads will not contain such information because these ads are for the same user and duplicated information in unnecessary.
After embedding, we obtain one embedding vector for the target ad, embedding vectors for the corresponding contextual ads, embedding vectors for clicked ads, and embedding vectors for unclicked ads. () is the vector dimension.
2.3. DSTN - Pooling Model
Since the numbers , and of auxiliary ads may vary for different target ads, it creates a problem for the deep neural network. Therefore, the first issue we need to solve is to process each type of variable-length auxiliary instances into a fixed-length vector.
In the DSTN - Pooling model, we use sum pooling to achieve this goal. The model architecture is shown in Figure 3(b). The aggregated representation vectors of contextual ads, of clicked ads and of unclicked ads are given by
If one type of auxiliary ad is completely missing (e.g., no contextual ads at all), we use an all 0 vector as its aggregated representation.
Now we have the target representation and the aggregated representations , and of auxiliary ads, the next issue is to fuse the information contained in these representations. In particular, we generate the fused representation as
where , , and are weight matrices that transform different representations into the same semantic space; is a bias parameter.
As can be seen, we actually use different weights to fuse the input from different types of data. This property is desired. It is because the degree of influence by different types of auxiliary data to the target ad may be different and we do distinguish such differences. Moreover, the fused representation has a property that it is not impacted if one or more auxiliary ads are completely missing. For example, if there is no contextual ads at all, we then set . As a result, we have and thus is not impacted.
If we concatenate the representations as , we can rewrite Eq. (1) as , where is the concatenation of all the weight matrices. This much simplifies the model. Therefore, the final design is to first concatenate respective representations to obtain an intermediate representation and then let), in order to exploit high-order feature interaction as well as nonlinear transformation. Nair and Hinton (Nair and Hinton, 2010) show that ReLU has significant benefits over sigmoid and tanh activation functions in terms of the convergence rate and the quality of obtained results.
Formally, the fully connected layers are defined as follows:
where denotes the number of hidden layers; and
denote the weight matrix and bias vector (to be learned) for theth layer.
Finally, the output vector
goes through a sigmoid function to generate the predicted CTR of the target ad as
where and are the weight and bias parameters to be learned. To avoid model overfitting, we apply dropout (Srivastava et al., 2014) after each fully connected layer. Dropout prevents feature co-adaptation by setting to zero a portion of hidden units during parameter learning (Goodfellow et al., 2013).
All the model parameters are learned by minimizing the average logistic loss on a training set as
where is the true label of the target ad corresponding to the estimated CTR and is the collection of labels.
It is observed that when different target ads are shown at a given position for a given user, only varies while all the auxiliary representations , and keep unchanged. It means that the auxiliary representations only serve as static base information. Moreover, as , and are generated by sum pooling, useful information could be easily buried in noise. For example, if the target ad is about coffee but most of the clicked ads are about clothing, then these clicked ads contribute little information to the target ad but the result of sum pooling is clearly dominated by these ads.
2.4. DSTN - Self-Attention Model
Given the above limitations, we consider the attention mechanism (Bahdanau et al., 2015) that is firstly introduced in the encoder-decoder framework for the machine translation task. It allows a model to automatically search for parts of a source sentence that are relevant to predicting a target word. In our DSTN - Self-attention model, the attention is applied over the instances of a particular type of auxiliary data to emphasize more important information. Take contextual ads as an example. The aggregated representation is modeled as
is a function that transforms the vector representation to a scalar weight . A possible instantiation of the
function could be a Multilayer Perceptron (MLP)(Bahdanau et al., 2015).
This self-attention mechanism has the advantage that useful information can be emphasized and noise can be suppressed because it weights different auxiliary ads differently according to (3).
Nevertheless, it still has the following limitations: 1) The weight is calculated solely based on the auxiliary ad . It does not capture the relationship between this auxiliary ad and the target ad . For example, no matter the target ad is about coffee or clothing, the importance of auxiliary ads keeps the same. 2) The normalized weight is calculated based on the relative importance with respect to all the and . As a result, even when all the are irrelevant to the target ad , the final importance is still large due to normalization. 3) The absolute number of each type of auxiliary ads matters, but normalization does not capture such an effect either.
|Dataset||# Target ads||# Fields||# Features||Avg # ctxt ads pta||Avg # clicked ads pta||Avg # unclicked ads pta|
2.5. DSTN - Interactive Attention Model
We thus propose the DSTN - Interactive attention model in this section that avoids the above limitations. It introduces explicit interaction between each type of auxiliary ads and the target ad. The model architecture is shown in Figure 3(c).
Take contextual ads as an example. We now model the aggregated representation vector as
Comparing (4) with (3), it is observed that is now a function of both the target ad and the auxiliary ad . In this way, dynamically adjusts the importance of with respect to . Moreover, does not depend on other . If none of auxiliary ads is informative, then all the should be small. We obtain and for clicked ads and unclicked ads similarly.
We learn the weight through an MLP with one hidden layer and the ReLU activation function as
where , , and are model parameters.
In this model, the fused representation is
Comparing (6) with (1), it is observed that the auxiliary representations do not serve as static base information now, but dynamically change when the target ad changes. It is because the weights in (4) depends on the target ad as well. It means that the model adaptively distills more useful information in auxiliary data with respect to the target ad. For example, the clicked ads of a user are about coffee, clothing and car. When the target ad is about coffee, the clicked ad of coffee should contribute more to ; but when the target ad is about car, the clicked ad of car should contribute more to . The model also preserves the property that it uses different weights to fuse the input from different types of auxiliary data. Furthermore, the weight in this model is not compared with other auxiliary ads, avoiding the problems that normalization would cause.
3. Offline Experiments
In this section, we conduct experiments on three large-scale datasets to evaluate the performance of the proposed DSTNs as well as several state-of-the-art methods for CTR prediction.
The statistics of the datasets are listed in Table 2. It is observed that the number of distinct features can be up to 46 million.
1) Avito advertising dataset. This is the dataset used for the Avito context ad clicks competition444https://www.kaggle.com/c/avito-context-ad-clicks/data. Note that the “context ads” here are different from the contextual ads that we bring forward in this paper. In Avito, “context ads” refer to ads tailored to the user and the context (such as device and time), in contrast to “regular ads” that are ordered by the time when they are added and “highlighted ads” that are stuck to the top of a page for some period of time.
This dataset contains a random sample of ad logs from avito.ru, the largest general classified website in Russia. We use the ad logs from 2015-04-28 to 2015-05-18 for training, those on 2015-05-19 for validation, and those on 2015-05-20 for testing. The features used include 1) ad features such as ad ID, ad title, ad category and ad parent category, 2) user features such as user ID, IP ID, user agent, user agent OS and user device, and 3) query features such as search query, search location, search category and search parameters.
2) Search advertising dataset. This dataset contains a random sample of ad impression and click logs from a commercial search advertising system in Alibaba. We use ad logs of 7 consecutive days in June 2018 for training, logs of the next day for validation, and logs of the day after the next day for testing. The features used include 1) ad features such as ad title, ad ID and industry, 2) user features such as user ID, IP ID and user agent, and 3) query features such as query and search location.
3) News feed advertising dataset. This dataset contains a random sample of ad impression and click logs from a commercial news feed advertising system in Alibaba. We use ad logs of 7 consecutive days in July 2018 for training, logs of the next day for validation, and logs of the day after the next day for testing. The features used include 1) ad features such as ad title, ad ID and industry, 2) user features such as user ID and the number of matched ad topics, and 3) cross-features such as AdType-AdResource. The auxiliary data in this dataset do not contain contextual ads. This is because only one ad is shown on a page in our news feed advertising system.
3.2. Methods Compared
We compare the following methods for CTR prediction.
LR. Logistic Regression (Bishop, 2006). It is a generalized linear model.
FM. Factorization Machine (Rendle, 2010). It models both first-order feature importance and second-order feature interactions.
DNN. Deep Neural Network. Each target ad first goes through an embedding layer and then goes through several fully connected layers. Finally, an output layer predicts the CTR through a sigmoid function.
Wide&Deep. The Wide&Deep model in (Cheng et al., 2016). It combines LR (wide part) with DNN (deep part).
DeepFM. The DeepFM model in (Guo et al., 2017). It combines FM (wide part) with DNN (deep part), and shares the same input and the embedding vector in its wide part and deep part. DeepFM has been shown to outperform Wide&Deep, Factorization-machine supported Neural Network (FNN) (Zhang et al., 2016) and Product-based Neural Network (PNN) (Qu et al., 2016).
CRF. The Conditional Random Field-based method in (Xiong et al., 2012)
. It considers both the features of an ad and its similarity to the surrounding ads. The predicted log odds of CTR is given by, where and are model parameters, is the feature vector of the target ad, and is the sum of similarity to the surrounding ads. The similarity is manually defined on strings in ad titles and ad descriptions. CRF is somewhat impractical because in commercial advertising systems, one cannot know ads below a target ad in advance. Therefore, we only use contextual ads (i.e., ads above the target ad) as the surrounding ads.
. The Gated Recurrent Unit(Chung et al., 2014)
, one of the most advanced Recurrent Neural Networks (RNNs). It has been shown to be able to avoid gradient vanishing and explosion problems and to result in better performance than the vanilla RNN. GRU is based on the RNN model proposed in(Zhang et al., 2014). It utilizes the sequence of clicked ads of a user.
DSTN-P. The DSTN - Pooling model proposed in Section 2.3. It uses sum pooling to aggregate auxiliary data.
DSTN-S. The DSTN - Self-attention model proposed in Section 2.4. It uses self-attention to aggregate auxiliary data.
DSTN-I. The DSTN - Interaction attention model proposed in Section 2.5. It introduces explicit interaction between auxiliary data and the target ad.
Among these methods, CRF, GRU and DSTNs consider auxiliary ads, while all other methods focus on the target ad. In particular, CRF considers surrounding ads, GRU considers clicked ads, and DSTNs consider contextual ads, clicked ads and unclicked ads.
3.3. Parameter Settings
We set the dimension of the embedding vectors for each feature as 10, because the number of distinct features is huge. We set the number of fully connected layers in DNN, Wide&Deep, DeepFM, GRU and DSTNs as 2, each with dimensions 512 and 256. The dropout ratio is set to 0.5. The hidden dimension of GRU is set to 128. The function in DSTN-S is an MLP with one hidden layer, with dimension 128. The dimension of
in DSTN-I is also set to 128. All the methods are implemented in Tensorflow and optimized by the Adagrad algorithm(Duchi et al., 2011). We set the batch size as 128.
We use a user’s historical behavior in recent 3 days. To reduce the memory requirement, we further restrict that , and . That is, if there are no more than 5 clicked ads, we use them all; otherwise, we use the most recent 5.
3.4. Evaluation Metrics
We use the following metrics to evaluate the compared methods.
AUC: the Area Under the ROC Curve over the test set. It is a widely used metric for CTR prediction tasks. It reflects the probability that a model ranks a randomly chosen positive instance higher than a randomly chosen negative instance. The larger the better. A small improvement in the AUC is likely to lead to a significant increase in the online CTR (Cheng et al., 2016).
Logloss: the value of Eq. (2) over the test set. The smaller the better.
Table 3 lists the AUC and Logloss values of different methods. It is observed that Wide&Deep achieves higher AUC than LR and DNN. Similarly, DeepFM achieves higher AUC than FM and DNN. These results show that combining a wide component and a deep component can improve the prediction power of individual models. CRF performs much better than LR, because it can be considered as rectifying the LR prediction by a term which summarizes the similarity to surrounding ads. However, the similarity is manually defined based on raw strings, thus suffering from the semantic gap problem. GRU performs better than LR, FM, DNN, Wide&Deep and DeepFM on two datasets, because GRU additionally utilizes clicked ads. The improvement is the most obvious on the News Feed ad dataset. This is because users do not submit a query in news feed ads and historical behaviors are quite informative.
It is also observed that DSTN-P outperforms GRU. The reasons are two-fold. First, consecutive actions in a user’s behavior sequence may not be well correlated. For example, a user clicked an ad of toothpaste one month ago and clicked ads of snacks and coffee recently. The next clicked ad may be about toothpaste again rather than food, simply because of the need of the user, rather than the correlation with the preceding clicked ads. Therefore, considering the order of a user’s behavior may not necessarily help improve the prediction performance. Second, DSTN-P can additionally utilize the information in contextual ads and unclicked ads.
When we compare different variants of DSTNs, it is observed that DSTN-S performs better than DSTN-P, and DSTN-I further outperforms DSTN-S. These results show that self-attention can better emphasize useful information than simple sum pooling. The interactive attention mechanism explicitly introduces the interaction between the target ad and auxiliary ads, and can thus adaptively distill more relevant information than self-attention.
It is also observed that Logloss is not necessarily correlated with AUC. Nevertheless, DSTN-I also results in the smallest Logloss on different datasets, showing its effectiveness.
3.6. Effect of the Type of Auxiliary Data
In order to examine the effect of different types of auxiliary data, we feed only contextual ads, clicked ads or unclicked ads to DSTN-I. To quantify the effect, we define and compute the following two metrics: absolute AUC improvement (AbsImp) and normalized AUC improvement (NlzImp):
where ctxt is short for contextual. AbsImp considers the overall AUC improvement and NlzImp normalizes the effect to each auxiliary ad. We care about the absolute rather than the relative AUC improvement because in industrial practice the former is more meaningful and indicative.
The results are plotted in Figure 4. It is observed in Figure 4(a) that the overall effect of different types of auxiliary data varies on different datasets. Contextual ads achieve the highest AbsImp on the Avito dataset, while unclicked ads achieve the highest AbsImp on the Search ad dataset. From Figure 4(b), it is interesting to observe that once normalized, the power of a contextual or clicked ad is much higher than that of an unclicked ad. This complies with intuitions because a contextual ad may distract a user’s attention and a clicked ad usually reflects a user’s interest. In contrast, an unclicked ad is much noisy. It may indicate that the user is not interested in the ad or the user does not view the ad.
3.7. Effect of the Network Depth
Figure 5 plots the test AUC of DSTN-I vs. the number of fully connected layers. The settings are: 1 layer - 256 dimensions; 2 layers - 512 and 256 dimensions; 3 layers - 1024, 512 and 256 dimensions. It is observed that increasing the number of fully connected layers can improve the AUC in the beginning, but the benefit diminishes when more layers are added. Adding more layers may even result in slight performance degradation, possibly due to more model parameters and increased difficulty of training deeper neural networks.
3.8. Visualization of Ad Embeddings
Figure 6 illustrates the visualization of ad embeddings with t-SNE (Maaten and Hinton, 2008) learned by DNN and by DSTN-I. It is based on 20 sub-categories from 5 major categories (electronics, clothing, furniture, computers and personal care). We randomly pick 100 ads in each sub-category. Different colors represent different sub-categories.
It is observed that the embeddings learned by both methods show clear clusters, each representing a group of similar ads. Nevertheless, DNN mixes up a portion of “iPhone” vs. “Samsung phone”, “beds” vs. “cabinets”, and “dresses” vs. “footwear”. In contrast, DSTN-I learns sharper clusters and clearly distinguishes different sub-categories. These results demonstrate that DSTN-I can learn more representative embeddings by the aid of auxiliary ads.
3.9. Visualization of Attention Weights
In this section, we examine the attention weights of auxiliary ads in DSTN-I through several case studies on the Avito dataset. We examine each type of auxiliary ads separately because it is hard to find a case containing sufficient ads of all types. For ease of illustration, we sort the auxiliary ads by their semantic similarity to the target ad. The leftmost is the most dissimilar and the rightmost is the most similar.
Contextual ads. In Figure 7, the target ad is about a YotaPhone. Three contextual ads are shown, which are about a phone lens, a Samsung phone and a HTC phone. It is observed that the weights of the two phone ads do not differ much (around 0.6), but the weight of the lens ad (most dissimilar) is much higher (over 0.8). Such an observation complies with the analysis in (Xiong et al., 2012), where the authors find that the more similar the surrounding ads are to an ad, the lower the CTR of the ad is. This is because similar ads can distract a user’s attention since all these ads offer similar products or services. In contrast, a dissimilar ad can help make the target ad more notable and is thus assigned a larger weight by DSTN-I.
Clicked ads. In Figure 8, the target ad is about monopod and remote for self-portrait photograph. The first clicked ad is about a baby car seat, which is clearly not relevant to the target ad. Its weight is 0.5223. The second clicked ad is about a flash light, which is an accessory of digital cameras for photography; its weight is much higher (0.7057). The third click ad is about a tripod which is more similar to a monopod, and thus the weight is even higher (0.8449). Finally, the fourth clicked ad is also about a monopod and its weight is the highest (0.9776). These observations show that the more similar a clicked ad is to the target ad, the higher its weight during aggregation. This is because if a user has clicked a similar ad, it is likely that the user will click the target ad as well.
Unclicked ads. In Figure 9, the target ad is about a Sony camera kit. The four unclicked ads are about a bike, a monopod, a camera lens and a camera kit respectively. These ads have increased similarity to the target ad and the corresponding weights also increase. These observations show that the more similar an unclicked ad is to the target ad, the higher its weight during aggregation. It is because if a user does not click a similar ad in the past, it is likely that the user will also not click the target ad as well.
Comparing Figure 9 with Figure 8, it is interesting to observe that the average weight of unclicked ads are much smaller than that of clicked ads, even when an unclicked ad is quite similar to the target ad. This is because clicked ads reflect possible user preferences while unclicked ads are much more ambiguous.
4. System Deployment and Evaluation
In this section, we introduce the deployment of our advertising system and present the online evaluation results.
We deployed DSTN-I in Shenma Search, the second largest search engine in China. We call the model DSTN hereafter for simplicity. Figure 10 depicts the architecture of the system, which contains an offline phase, a streaming phase and an online phase.
Offline phase: Online user behaviors (ad impression / click) are continuously logged into the user log database. The system exacts training data from the log database and trains the DSTN model. The offline training is first performed in a batch manner, and the resulting model is then updated incrementally and periodically using recent log data.
Streaming phase: Online user behaviors are also sent to the User Session Server (with a delay of no more than 10 seconds), where a hashmap for each user is maintained and updated. To reduce the memory requirement and the online computation load, the hashmap records at most 5 clicked ads and 5 unclicked ads in the recent 3 days for each user.
Online phase: Once a user request is sent, the Ad Server first retrieves the user history data from the User Session Server. The Ad Server then requests the Model Server for pCTR (predicted CTR) of a set of candidate ads. This is done in several steps as shown in Figure 11.
The steps are: 1⃝ The Ad Server sends the set of candidate target ads, along with the clicked and unclicked ads of the given user to the Model Server. There is no contextual ad now. 2⃝ The Model Server returns the pCTRs. 3⃝ The Ad Server picks out the target ad with the highest score based on certain strategy (which depends on the pCTR). Assume this ad is the Target ad 2. Then it becomes the contextual ad for other target ads. The Ad Server then sends the remaining target ads, along with the contextual ad, the clicked and unclicked ads of the user to the Model Server. 4⃝ The Model Server returns the pCTRs for the remaining candidate ads.
Theoretically, Steps 3⃝ and 4⃝ could be performed several times to pick out ads one by one and to update contextual ads sequentially. However, there exists a tradeoff between prediction accuracy and service delay. Therefore, in our current implementation, we only perform Steps 3⃝ and 4⃝ once. After Step 4⃝, the Ad Server picks out 2-3 remaining candidate ads with highest scores and sends the final ad list to the user. Shenma Search now holds 3-4 ad slots per mobile page.
4.2. Online A/B Test
We conducted online experiments in an A/B test framework over two weeks in Jan. 2019. The benchmark model is Wide&Deep, which is our last online serving model. Our online evaluation metric is the real CTR, which is defined as the number of clicks over the number of ad impressions. A larger online CTR indicates the enhanced effectiveness of a CTR prediction model. We observe that DSTN outperforms Wide&Deep consistently, resulting in an increase of daily online CTR from 5.13% to 9.72%. The average CTR increase is 6.92%. These results demonstrate the effectiveness of DSTN in practical CTR prediction tasks.
5. Related Work
CTR prediction. Learning the effect of feature interactions seems to be crucial for accurate CTR prediction (Rendle, 2010). Generalized linear models, such as Logistic Regression (LR) (Richardson et al., 2007) and Follow-The-Regularized-Leader (FTRL) (McMahan et al., 2013), have shown decent performance in practice. However, a linear model lacks the ability to learn sophisticated feature interactions (Chapelle et al., 2015). Factorization Machines (FMs) (Rendle, 2010; Blondel et al., 2016) are proposed to model pairwise feature interactions in terms of the latent vectors corresponding to the involved features. Field-aware FM (Juan et al., 2016) and Field-weighted FM (Pan et al., 2018) further improve FM by considering the impact of the field that a feature belongs to.
In recent years, Deep Neural Networks (DNNs) have shown powerful ability of automatically learning informative feature representations (LeCun et al., 2015). DNNs are thus also exploited for CTR prediction and item recommendation in order to automatically learn feature representations and high-order feature interactions (Van den Oord et al., 2013; Covington et al., 2016; Wang et al., 2017). Factorization-machine supported Neural Network (FNN) (Zhang et al., 2016) pre-trains an FM before applying a DNN. Product-based Neural Network (PNN) (Qu et al., 2016) introduces a product layer between the embedding layer and the fully connected layer. The Wide&Deep model (Cheng et al., 2016) combines LR and DNN to capture both low- and high-order feature interactions. Such a structure also improves both the memorization and generalization abilities of the model. DeepFM (Guo et al., 2017) models low-order feature interactions like FM and models high-order feature interactions like DNN. Neural Factorization Machine (He and Chua, 2017) combines the linearity of FM and the non-linearity of neural networks.
Exploiting auxiliary data for CTR prediction. Another line of research exploits auxiliary data for improving the CTR prediction performance. Zhang et al. (Zhang et al., 2014) consider users’ historical behaviors (e.g., what ads she clicked). They use Recurrent Neural Networks (RNNs) to model the dependency on users’ sequential behaviors. Tan et al. (Tan et al., 2016)
propose improved RNNs for session-based recommendations. One major problem with RNN-based models is that it generates an overall embedding vector of a behavior sequence, which can only preserve very limited information of a user. Long-term dependencies are still hard to be preserved even using the advanced memory cell structures like Long Short-Term Memory (LSTM)(Hochreiter and Schmidhuber, 1997) and Gated Recurrent Unit (GRU) (Chung et al., 2014). Moreover, both the offline training and the online prediction process of RNNs are time-consuming, due to its recursive structure.
Xiong et al. (Xiong et al., 2012) consider the pairwise relationship between ads shown on the same page and propose a Conditional Random Field (CRF)-based model for CTR prediction. Yin et al. (Yin et al., 2014) consider various contextual factors such as ad depth, query diversity and ad interaction for click modeling. One major problem of these models is that the vertex and edge feature functions need to be manually defined based on data analysis and it is difficult to generalize the model to other types of data.
Differences. DSTNs proposed in this paper differ from prior work in that: 1) DSTNs integrate heterogeneous auxiliary data (i.e., contextual, clicked and unclicked ads) in a unified framework, while the RNN-based model (Zhang et al., 2014) cannot utilize contextual and unclicked ads, and the CRF-based model (Xiong et al., 2012) cannot incorporate clicked and unclicked ads; 2) DSTNs are not based on RNNs and they are much easier to implement and are much faster to train and evaluate online.
In this paper, we address the problem of CTR prediction in online advertising systems. In contrast to classical CTR prediction models that focus on the target ad, we explore three types of auxiliary data (i.e., contextual, clicked and unclicked ads) and propose DSTNs for improving the CTR prediction. DSTNs are able to distill useful information in auxiliary ads and to fuse heterogeneous data in a unified framework. Offline experimental results on three large-scale datasets demonstrate the effectiveness of DSTNs over several state-of-the-art methods. Case studies show that DSTN-I is able to learn representative ad embeddings and meaningful attention weights. We have deployed DSTN-I in Shenma Search. Online A/B test results show that the online CTR is also improved compared to our last serving model, demonstrating the effectiveness of DSTN-I in real-world CTR prediction tasks.
- Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.
- Bishop (2006) Christopher M Bishop. 2006. Pattern Recognition and Machine Learning. (2006).
- Blondel et al. (2016) Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. 2016. Higher-order factorization machines. In NIPS. 3351–3359.
- Chapelle et al. (2015) Olivier Chapelle, Eren Manavoglu, and Romer Rosales. 2015. Simple and scalable response prediction for display advertising. ACM Transactions on Intelligent Systems and Technology 5, 4 (2015), 61.
- Cheng et al. (2016) Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In DLRS. ACM, 7–10.
- Chung et al. (2014) Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).
- Covington et al. (2016) Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In RecSys. ACM, 191–198.
- Duchi et al. (2011) John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, Jul (2011), 2121–2159.
- Goodfellow et al. (2013) Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio. 2013. Maxout networks. In ICML. 1319–1327.
- Guo et al. (2017) Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. Deepfm: a factorization-machine based neural network for ctr prediction. In IJCAI. 1725–1731.
- He and Chua (2017) Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In SIGIR. ACM, 355–364.
- He et al. (2014) Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. 2014. Practical lessons from predicting clicks on ads at facebook. In ADKDD. ACM, 1–9.
- Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
- Juan et al. (2016) Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware factorization machines for CTR prediction. In RecSys. ACM, 43–50.
- LeCun et al. (2015) Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436.
- Maaten and Hinton (2008) Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579–2605.
- McMahan et al. (2013) H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. 2013. Ad click prediction: a view from the trenches. In KDD. ACM, 1222–1230.
- Mikolov et al. (2013) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS. 3111–3119.
- Nair and Hinton (2010) Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML. 807–814.
- Pan et al. (2018) Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018. Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising. In WWW. International World Wide Web Conferences Steering Committee, 1349–1357.
- Qu et al. (2016) Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-based neural networks for user response prediction. In ICDM. IEEE, 1149–1154.
- Rendle (2010) Steffen Rendle. 2010. Factorization machines. In ICDM. IEEE, 995–1000.
- Richardson et al. (2007) Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: estimating the click-through rate for new ads. In WWW. ACM, 521–530.
- Shan et al. (2016) Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In KDD. ACM, 255–262.
- Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (2014), 1929–1958.
- Tan et al. (2016) Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Improved recurrent neural networks for session-based recommendations. In DLRS. ACM, 17–22.
- Van den Oord et al. (2013) Aaron Van den Oord, Sander Dieleman, and Benjamin Schrauwen. 2013. Deep content-based music recommendation. In NIPS. 2643–2651.
- Wang et al. (2017) Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In ADKDD. ACM, 12.
- Xiong et al. (2012) Chenyan Xiong, Taifeng Wang, Wenkui Ding, Yidong Shen, and Tie-Yan Liu. 2012. Relational click prediction for sponsored search. In WSDM. ACM, 493–502.
- Yin et al. (2014) Dawei Yin, Shike Mei, Bin Cao, Jian-Tao Sun, and Brian D Davison. 2014. Exploiting contextual factors for click modeling in sponsored search. In WSDM. ACM, 113–122.
- Zhang et al. (2016) Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data. In ECIR. Springer, 45–57.
- Zhang et al. (2014) Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, and Tie-Yan Liu. 2014. Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks.. In AAAI, Vol. 14. 1369–1375.
- Zhou et al. (2018) Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD. ACM, 1059–1068.