Web publishers use different channels to sell their ads. We consider in this paper two channels: RTB (real-time bidding) and Direct. In RTB, the publisher sells ads through an ad exchange where buyers compete in auctions happening in real time. In Direct, the ads are sold through direct campaigns.
A direct campaign is a contract in which a publisher commits to deliver a specified number of ads to an advertiser while ensuring that some KPIs are met (for example the publisher may guarantee a minimum number of clicks to the advertiser). Should the publisher not deliver the impressions as defined in the contract, it pays a penalty to the advertiser.
A natural way for a publisher to allocate impressions between the two channels is to bid in the RTB auction for every impression: if it wins the auction, the publisher chooses a direct campaign to sell the impression, otherwise the impression is sold in the RTB market.
The first section gives a review of past work related to this topic. The second section states the optimization problem to solve. The two following sections derive optimality conditions and present an algorithm to build an optimal delivering strategy. The last section presents the results of experiments realized on a real RTB auctions dataset.
2. Related work
The problem of maximizing a web publisher advertising revenue is much studied in the literature. Because of the complexity of the online advertising ecosystem, and because the publishers sell their ads through different channels, this problem leads to diverse approaches.
(Ostrovsky and Schwarz, 2011), (Yuan et al., 2014) and (Chahuara et al., 2017) consider a publisher selling its ads through RTB only, and define strategies for the publisher to set appropriate reserve prices in each auction to optimize its RTB revenue. In (Ostrovsky and Schwarz, 2011), the reserve prices are set in a static way (i.e. an optimal reserve price is estimated on a large set of auctions), whereas (Yuan et al., 2014) and (Chahuara et al., 2017) define methodologies to set dynamic reserve prices which are predicted in real time before an auction happens.
(Murali et al., 2015) and (Lee et al., 2013) study the optimal delivery of a budget across time in the RTB market. This problem known as budget pacing has been much studied and presents strong similarities with the problem of delivering a direct campaign. In both articles, the bids are updated online to optimize the budget delivery.
(Roels and Fridgeirsdottir, 2009) and (Bharadwaj et al., 2012) focus on Direct revenue maximization. They consider that the impressions can be sold in direct campaigns only, and define strategies to allocate optimally a set of impressions between all campaigns in order to maximize the publisher revenue (i.e. minimize the penalty the publisher pays if some campaigns are not delivered as agreed in the contract). (Roels and Fridgeirsdottir, 2009) formulates this problem as a dynamic programming problem, and (Bharadwaj et al., 2012) proposes an efficient algorithm to allocate near-optimally the impresions between the campaigns.
(Ghosh et al., 2009), (Chen et al., 2014), (Chen, 2016b) and (Balseiro et al., 2011) study the joint optimization of the publisher revenue coming from RTB and Direct. They consider a setting similar to our setting, in which the publisher bids in auctions to compete with RTB bidders.
In (Ghosh et al., 2009), the quality of a direct campaign delivery is represented by a utility function, and the quality of each impression is linked directly to its price in the RTB market, whereas we characterize a campaign delivery by some KPIs.
The model used in (Chen et al., 2014) and (Chen, 2016b) is based on the modeling of the supply and the demand on an aggregated level, while we analyze data auction per auction without needing to model the supply and demand.
In (Balseiro et al., 2011), a campaign is defined by a fixed number of impressions which is supposed to be delivered exactly, and a per-impression quality. The algorithm aims to maximize a weighted sum of the RTB revenue and of the direct campaigns qualities. The main difference with our approach is that the campaigns are supposed to be completely delivered. In our approach, we enable under-delivery of a campaign if it brings a higher revenue to the publisher, which makes the problem more complex. Also, the approach described in (Balseiro et al., 2011) may be hard to scale due to the allocation plan used to allocate similar campaigns. We solve this issue by regularizing the optimization problem.
3. Problem statement
In this section, we introduce the two channels used by the publisher to sell its ads: RTB and Direct. Then, we derive the global revenue the publisher gets from both channels, and formulate the revenue optimization problem.
All random variables are defined with respect to the filtration, which is the filtration representing the information available at time , i.e. before the first auction happens.
3.1. Rtb market
We represent the RTB market as a set of time-ordered auctions happening at times . This approach is deterministic in that we do not model the uncertainty in the auctions to come.
In auction , a set of RTB buyers compete to buy the ad slot and display their ad. We note respectively and the random variables corresponding to the highest bid and the second highest bid in the auction (possibly if there are not enough buyers bidding in the auction).
3.2. Direct campaigns
The Direct channel is represented by direct campaigns booked by the publisher at time . We assume that the set of campaigns is static, i.e. existing campaigns can not be cancelled, and no new campaign is booked.
Each campaign is defined by a contractual revenue, which is the revenue the publisher is paid to deliver the campaign, a set of goals and a penalty function giving the penalty the publisher has to pay when these goals are not met.
3.2.1. Contractual revenue of the campaign
We note the revenue the publisher gets to deliver the campaign. It is worth noting that the publisher strategy to deliver the campaign does not depend on this amount, which is contractual and fixed before the campaign is delivered.
3.2.2. Goals of the campaign
Let be the number of goals for campaign . Each goal is defined by a targeting and a volume to be delivered in the targeting .
The targeting is defined in each auction by the Bernoulli variable equal to if impression is eligible for the targeting, and otherwise. Let be the parameter of the Bernoulli variable , which is supposed to be known before the auction happens.
A deterministic targeting corresponds to the case where we know certainly if the ad will be in the targeting when bidding in the auction, and we have in this case or . For example, targeting a website or an ad placement is a deterministic targeting.
A stochastic targeting corresponds to the case where we do not know if the ad will be in the targeting before the auction happens. In this case, is an estimation of the probability for the ad to be in the targeting. For example, targeting only clicked impressions is a stochastic targeting, as we do not know if the ad will be clicked or not before it has been displayed.
3.2.3. Penalty function
We introduce the variables , where corresponds to the difference between the goal and the volume delivered effectively on the targeting .
For campaign , the penalty function gives the penalty the publisher has to pay when it fails to deliver the goals. At the end of the campaign, this penalty should be deduced from the contractual revenue of the campaign to get the revenue the publisher got from the campaign.
A typical penalty function would be , corresponding to the case where the publisher pays for each non-delivered impression on targeting .
3.3. Auction mechanics
Let and be respectively the publisher bid and the RTB revenue in auction . We note also the expected RTB revenue. The auction mechanics is the following:
If , the publisher bidder wins the auction and the publisher chooses a direct campaign to display. We note the multinoulli variable with parameters such that campaign is chosen with probability . The RTB revenue for this auction is
If , the ad is sold in the RTB market. When the auction mechanism is a second-price auction, which is the most usual mechanism, we have 111This outcome corresponds to the case where no reserve price is set in the auction. The case where a reserve price is set is handled by replacing by the maximum value between the second highest bid and the reserve price.. When the auction is a first-price auction, we have
3.4. Volumes delivered on direct campaigns
Let be the volume delivered on the targeting after the auctions have occured. It can be written:
We note . We assume that for each , the random variables , and are independent, which gives:
Note that the independence hypothesis between , and is not a strong hypothesis: indeed, as these random variables are specific to auction , they include already all the information we may know about the auction.
3.5. Publisher revenue
Let be the total publisher revenue coming both from RTB and Direct, and let be its expected value. It depends on the publisher bids , and on the parameters used to select the winning campaign when the publisher bidder wins the auction.
To simplify the expression of the expected revenue, we make the following approximation:
This approximation tends to be well verified when the volumes delivered on all targetings are large.
The expected publisher revenue is finally written:
Note that the publisher bidder impacts the RTB revenue in two ways: by removing impressions from the RTB channel to deliver direct campaigns, and by modifying the RTB closing price (for example when the publisher bid acts as a second price in a second-price auction mechanism).
3.6. Optimization problem statement
In this section, we note the sequence of all (), the sequence of all ( and ), and the sequence of all ( and ).
The optimization problem is written the following way:
Note that the variables , which represent the volumes delivered on all targetings, are considered to be variables to optimize whereas they derive directly from and . This relation is introduced as an equality constraint. Stating the problem this way makes its analysis and the presentation of the optimization algorithm much simpler.
4. Optimality conditions
This section writes the Karush-Kuhn-Tucker conditions of the optimization problem stated in the previous section for two kinds of penalty functions: differentiable functions and rectified linear functions (which are not differentiable everywhere but which are widely used in practice). The lagrangian of the problem and the complete derivation of the Karush-Kuhn-Tucker conditions are given in the appendix.
4.1. Optimality conditions when the penalty functions are differentiable
We introduce the score of campaign in auction , defined by:
4.1.1. Optimality condition on
The optimality condition on writes:
In the particular case where there is one unique campaign maximizing , we have if and otherwise. If several campaigns maximize , the optimality conditions do not give an easy way to determine the corresponding .
4.1.2. Optimality condition on
The optimality condition on writes:
It is worth noting that the expected revenue from auction , should it bring if it is sold in a direct campaign, writes . Therefore, is the bid maximizing the expected revenue from auction if it brings if it is sold in a direct campaign.
The appendix gives a detailed analysis of the existence of verifying this condition.
4.2. Optimality conditions when some penalty functions are rectified linear functions
We assume in this section that the penalty function for campaign writes . This kind of penalty function is widely used in practice but is not differentiable.
The optimality conditions are similar to the ones obtained when the penalty functions are differentiable, but where the score of campaign is written the following way:
5. Optimization algorithm
This section proposes an algorithm to estimate an optimal strategy (i.e. and ) on a RTB auctions dataset. We assume that the dataset is processed in batches of auctions. In practice, the dataset is shuffled before processing it to ensure the batches are statistically equivalent.
5.1. Description of the algorithm when the penalty functions are differentiable
When the penalty functions are differentiable, the algorithm estimates the optimal volumes delivered on all targetings, which are the only variables needed to estimate in each auction the scores of the campaigns and then the optimal and .
We note the estimation of these volumes after having processed batches. A naive initialization of the algorithm is , but a better initialization can accelerate significantly the convergence of the algorithm. For example, it may be relevant to initialize the algorithm with .
The -th batch of auctions is processed with two steps.
5.1.1. Step 1: loop on the batch and apply the strategy determined using
In this section, we note the scores of the campaigns in auction computed when the volumes delivered on all targetings are .
We note and the publisher bid and the campaigns probabilities applied in auction . They are determined with the following equations, which are obtained by writing the optimality conditions when the volumes are given by :
When contains one element only, it is selected, otherwise a campaign is randomly chosen among those with the maximum score. We note the volumes delivered on all campaigns when this strategy is applied on the -th batch.
5.1.2. Step 2: determine
We introduce the ratio of the number of auctions in one batch to the number of auctions in the dataset, noted . We introduce also the learning rate of the algorithm . is determined with:
This update methodology is based on the fact that at the optimum, the volume delivered on a targeting on one batch of data is equal in average to .
The choice of has a major impact on the convergence speed of the algorithm. If is too high, the algorithm may not converge, whereas a too low value makes the convergence very slow. We use in practice a decreasing , for example .
The size of the batches is an important parameter of the algorithm. In practice, we notice that the smaller the batches, the faster converges the algorithm, where the convergence speed is defined as the number of auctions to process to attain a given level of convergence. However, the update step performed for each batch is computationally expensive as it requires to update the volumes for all campaigns. In practice, we use batches composed of a few hundreds of auctions.
5.2. Description of the algorithm when some penalty functions are rectified linear functions
We assume in this section that the penalty function for campaign writes . The score in auction writes in this case .
For this campaign, the algorithm estimates the values of , which are the variables needed to estimate the score of this campaign in each auction . We define the estimate of after having processed batches.
A natural way to initialize the algorithm is , which is consistent with the initialization . We could also initialize with .
Let be the estimated value of on the -th batch. It is estimated by comparing the volume delivered on the batch and the volume we expect to be delivered if the goal is reached:
Then, the update step is similar:
5.3. A regularization of the optimization problem
The application of the optimal strategy consists in picking in each auction the campaign with the highest score, where the score is computed using the volumes after the algorithm has converged.
Such a choice of the winning campaign may lead to an unwanted behaviour of the algorithm. For example, let’s consider the case of two campaigns with identical definitions. At the optimum, the volumes for these two campaigns should be identical, which means that in all auctions, both campaigns will be picked with equal probabilities. In practice, the optimal volumes estimated for these two campaigns using the algorithm may be slightly different, and in this case it is easy to see that one of the campaigns will always have a higher score than the other one, which will consequently never be picked.
This simple example illustrates that a minor error in the estimation of the optimal volumes of the campaigns may have a large impact on the estimated optimal strategy.
To overcome this issue, we have written a regularized version of the optimization problem and derived the corresponding optimality conditions. The complete derivation is given in the appendix. The regularized optimality conditions are:
where is the regularization parameter.
It is worth noting that using this regularization in the optimization phase may also accelerate the convergence of the optimization algorithm.
6. Experiments on an auctions dataset
We have used in this section a real RTB auctions dataset corresponding to one week of video advertising data. We have filtered auctions which led to an ad (i.e. auctions won by a buyer and for which a video ad has been delivered effectively). The dataset has been shuffled before running the optimization algorithm to ensure that the batches are statistically equivalent.
In each auction, we will use the following informations:
The impression id, which is a unique identifier of the impression
The placement id, which is a unique identifier of the ad slot. The dataset contains 4 different ad placements, called , , and
The highest bid of the auction (which is strictly positive because we filtered on auctions won by a buyer)
We know if the impression has been viewed222The IAB viewability measure is used: an ad is considered to be viewed if at least 50% of its surface has been within the visible area of the browser window during at least 1 second. or clicked
Two experiments have been performed:
The first experiment runs the optimization algorithm with a few campaigns defined manually, and gives an interpretation of the optimal strategy
The second experiment runs the optimization algorithm with many campaigns and shows that the algorithm scales with the number of campaigns
We assume that the auction mechanism is a first-price auction mechanism. This makes the optimality condition on easier to solve because it does not need any hypothesis on the highest RTB bid distribution.
The optimization algorithm is run with batches of size and with . The optimality conditions have been regularized with the temperature parameter , both when determining the optimal strategy and when applying the optimal strategy on the dataset. The algorithm is initialized with .
We define in this section the adjusted revenue as the RTB revenue reduced by the penalty the publisher has to pay on the direct campaigns in case of under-delivery. In both experiments, we plot the adjusted revenue of the publisher when applying the estimated optimal strategy on the whole dataset, as a function of the number of batches of size processed by the algorithm.
6.1. Descriptive analysis of the dataset
The table below gives the main characteristics of the dataset, per placement and globally.
|Nb Imps||Avg RTB bid||Avg view rate||Avg click rate|
|15.46 mlns||13.15 $CPM||77.0 %||0.13 %|
|5.56 mlns||15.36 $CPM||77.9 %||0.23 %|
|2.43 mlns||15.55 $CPM||59.7 %||2.03 %|
|0.14 mlns||25.37 $CPM||42.7 %||0.00 %|
|Total||23.59 mlns||13.99 $CPM||75.3 %||0.35 %|
6.2. First experiment: optimization with a few campaigns
In this section, we define a direct campaign by the following sextuplet:
, where and correspond to the goal and to a penalty parameter associated to the number of impressions, the number of impressions viewed and the number of impressions clicked. We assume that all campaigns target all the placements: this hypothesis makes the results simpler to interpret.
We assume that the penalty functions are rectified linear functions where is the penalty per non-delivered impression. If , and are the volumes of impressions, of impressions viewed and of impressions clicked which have been delivered, the penalty function writes:
This experiment runs the optimization algorithm with the following campaigns:
3 impressions campaigns:
3 viewability campaigns:
3 click campaigns:
To run the experiment, we have used simple viewability and click predictors which predict the average viewability rate and click rate of the placement. These predictors are used before each auction to compute the scores of the campaigns. Using more advanced predictors based on more features would clearly improve the results.
Figure 1 plots the adjusted revenue estimated on the whole dataset as a function of the number of batches processed by the optimization algorithm. The intialization point corresponds to the case where no impression is sold in a direct campaign and it is very far from the optimum in terms of adjusted revenue. The algorithm converges after having processed a few dozens of batches.
The table below has been built by applying the optimal strategy estimated after having processed batches of auctions. It gives, for each campaign, the number of impressions delivered on each placement, as a percentage of the goal of the campaign. It gives also the undelivered volume (i.e. the goal less the volume delivered) as a percentage of the goal of the campaign.
We see that the campaigns , , , , and are not fully delivered: for these campaigns, the penalty in case of under-delivery is low compared to the opportunity cost incurred by the publisher by not selling impressions on the RTB market. Also, the campaigns targeting clicks are mainly delivered on placement , which has a significantly higher click probability whereas the RTB bid is not significantly higher than on the other placements.
6.3. Second experiment: optimization with many campaigns
We note in this section
the uniform distribution on. In this experiment, we run the optimization with campaigns, and each campaign is generated the following way:
It targets all impressions where , where is a hash function, imp_id is the id of the impression, and is drawn from . This methodology ensures that the targetings of a pair of campaigns has most of the time a non-empty intersection (which makes probably the optimization harder, as each impression must be allocated efficiently among all the campaigns targeting it)
The impression goal is drawn from the distribution , where is chosen such that the sum of the goals on all campaigns represent in average of the total number of impressions available. Consequently, decreases when increases. Note that in this experiment, we do not define any goals on the number of impressions clicked or viewed
The penalty parameter associated to the impressions goal is drawn from the distribution
Figure 2 plots the adjusted revenue of the publisher on the whole dataset as a function of the number of batches processed by the optimization algorithm, for different values of the number of campaigns .
We see on the figure that increasing the number of campaigns does not slow the convergence of the algorithm, and a few dozens of batches are enough to converge to the optimum. In practice, the optimum is attained in less than a few minutes with campaigns.
7. Conclusions and future directions
In this paper, we study the problem of optimizing the revenue a web publisher gets through RTB and Direct. For each impression, the publisher bids in the RTB auction and chooses the direct campaign to deliver should it win the auction. We have been able to define an algorithm to determine an optimal strategy for the publisher in this setting.
The experiments performed on a RTB auctions dataset show that the algorithm scales with the number of campaigns, as increasing the number of campaigns does not slow the convergence. With campaigns, an optimal strategy can be estimated in a few minutes.
Also, applying the optimal strategy is particularly simple and the most computationally intensive part is to compute the score of each campaign, which is a weighted sum of the predictors . In practice, the publisher has a few milliseconds to determine its bid. Should the number of campaigns be too high to be able to compute all the scores, the publisher should define a strategy to sample the campaigns considered for a given impression.
In practice, a web publisher should estimate the optimal strategy on a long period of time (ideally on several months) to make it as stable as possible. Applying it in the future relies on the hypothesis that the future RTB market is similar to the past auctions used in the estimation, in terms of volume and bid level. The estimation can also be adjusted to reflect an expected change in the RTB market.
When the RTB market characteristics differ too much from the dataset used in the estimation, the publisher should define a methodology to adjust the strategy. Such a methodology can be based for example on a controller which updates the strategy from time to time. This is a different topic which has been studied specifically in the literature (see (Kitts et al., 2017), (Karlsson and Zhang, 2013)).
The algorithm presented in the paper combined with a controller form a system which is being developed to be deployed on thousands of publishers worldwide.
Finally, the hypothesis that the set of campaigns is known when estimating the strategy is well suited to the theoretical analysis of the problem and to the derivation of an estimation algorithm. In practice, the publisher should make hypotheses on the campaigns which will be agreed in the future.
8.1. Derivation of the optimality conditions
The lagrangian of the problem writes:
where , and are the lagrange multipliers corresponding to the constraints.
The Karush-Kuhn-Tucker optimality conditions are:
8.1.1. Conditions on
From the second equation we get for auction :
Let’s consider the set of -s maximizing :
If , we get:
As is the Lagrange multiplier corresponding to the constraint , we have . We know also that there must be at least one such that (otherwise all would be equal to and they couldn’t sum to ).
We deduce that , and finally: if , (and ), otherwise (and ).
We introduce the score of campaign in auction , defined by:
We obtain the following optimality condition:
8.1.2. Conditions on
The first equation gives the optimality condition on :
8.2. Particular case where some penalty functions are rectified linear functions
We assume in this section that the penalty function for campaign writes . This kind of penalty function is widely used in practice but is not differentiable.
Let be the sub-gradient of the lagrangian with respect to , which writes:
The Karush-Kuhn-Tucker optimality condition with respect to writes: . This gives similar optimality conditions where the score of campaign is written the following way:
8.3. Analysis of the optimality condition on
In this section, we note and the cumulative probability functions of the highest bid and the second highest bid , and and their probability density functions. We assume that these functions are continuous on and that .
The optimality condition on states that the optimal value verifies , where is the maximum score of the direct campaigns.
We show in this section that in a second-price auction mechanism, this optimality condition has at least one solution under a usual hypothesis on the distributions and . The solution is very simple in a first-price auction mechanism.
8.3.1. Second-price auction mechanism
Note that , because .
It is worth noting that this function depends on the marginal distributions and
only. It does not depend on the joint distribution, which makes the analysis much simpler.
It is easy to see that (as ). To analyze the behaviour of this function when , we write the function using the hazard rates of and , and :
We assume also that
. This hypothesis is quite usual and is verified by the normal and the log-normal distributions. Therefore, we have:
As is a continuous function on , we conclude that under the hypothesis , is a surjective function to . This means that the optimality condition has at least one solution for every . However, we have no guarantee on the unicity of the solution.
8.3.2. First-price auction mechanism
The first-price auction can be modeled with , and in this case we have . The optimality condition has one unique solution .
8.4. A regularization of the optimization problem
We introduce in this section a regularization of the optimization problem which makes the optimality conditions more stable. For each auction , we add the term to the objective function, where is the regularization parameter which does not depend on and is the entropy of the distribution . The case corresponds to the non regularized optimization problem, and in the case , the optimal distribution is the uniform distribution.
The objective function of the problem becomes:
The Karush-Kuhn-Tucker optimality conditions write:
The second equation gives:
And the constraint gives:
Finally, the optimality conditions of the problem are the following:
The define a Boltzmann distribution with the weights and a temperature
- Yield optimization of display advertising with ad exchange. In Proceedings of the 12th ACM Conference on Electronic Commerce, EC ’11, New York, NY, USA, pp. 27–28. External Links: Cited by: §2, §2.
- SHALE: an efficient algorithm for allocation of guaranteed display advertising. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’12, New York, NY, USA, pp. 1195–1203. External Links: Cited by: §2.
- Real-time optimization of web publisher rtb revenues. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, New York, NY, USA, pp. 1743–1751. External Links: Cited by: §2.
- PhD thesis, Financial Methods for Online Advertising. Cited by: §2.
- Risk-Aware Dynamic Reserve Prices of Programmatic Guarantee in Display Advertising. Cited by: §2, §2.
- A dynamic pricing model for unifying programmatic guarantee and real-time bidding in display advertising. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, ADKDD’14, New York, NY, USA, pp. 1:1–1:9. External Links: Cited by: §2, §2.
- Bidding for Representative Allocations for Display Advertising. Cited by: §2, §2.
- Applications of feedback control in online advertising. Cited by: §7.
- Ad serving with multiple kpis. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, New York, NY, USA, pp. 1853–1861. External Links: Cited by: §7.
- Real time bid optimization with smooth budget delivery in online advertising. In Proceedings of the Seventh International Workshop on Data Mining for Online Advertising, ADKDD ’13, New York, NY, USA, pp. 1:1–1:9. External Links: Cited by: §2.
- Optimal budget allocation strategies for real time bidding in display advertising. In Proceedings of the 2015 Winter Simulation Conference, WSC ’15, Piscataway, NJ, USA, pp. 3146–3147. External Links: Cited by: §2.
- Reserve prices in internet advertising auctions: a field experiment. In Proceedings of the 12th ACM Conference on Electronic Commerce, EC ’11, New York, NY, USA, pp. 59–60. External Links: Cited by: §2.
- Dynamic Revenue Management for Online Display Advertising. Cited by: §2.
- Selling futures online advertising slots via option contracts. In Proceedings of the 21st International Conference on World Wide Web, WWW ’12 Companion, New York, NY, USA, pp. 627–628. External Links: Cited by: §2.
- An empirical study of reserve price optimisation in real-time bidding. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, New York, NY, USA, pp. 1897–1906. External Links: Cited by: §2.