The campaign management system of Verizon Media DSP implements a feedback-based control system [Karlsson and Zhang, 2013], which adjusts the bid prices used to bid for impressions. Different aspects of how the optimization problem is formulated and solved as a control problem can be found in [Karlsson, 2016], [Guo and Karlsson, 2017], [Sang et al., 2018], [Karlsson and Guo, 2016]. The fact that the plant 111Plant is the term used to indicate the relation between an input signal and the output signal of a system without feedback, determined by the properties of the whole online advertising ecosystem is unknown, dynamic, nonlinear, and in general discontinuous is a characteristic property of online advertising processes and is a fundamental challenge in the development of feedback control solutions. In order to design feedback control algorithms for the optimization system, it is valuable to have a model of the input-output relationship, which is referred to as the control response curves in this paper. The key performance metrics for CPA lines include impressions, total advertiser spend, eCPA, etc. Besides, the problem of estimating the sensitivity of the output (mainly the total advertiser spend) to the control signal, which usually termed as the plant gain in control community, is also crucial to controller design and challenging due to the discontinuity of the plant.
Different approaches at indirectly estimating and controlling the plant are proposed in [Karlsson, 2014], [Karlsson, 2017], [Guo and Karlsson, 2017]. The former two of these papers make use of bid randomization, but all are based exclusively on local feedback information to estimate the plant gain. A systematic methodology for offline modeling of advertising plants is proposed in [Mardanlou et al., 2017]. The methodology is conveniently used to simulate realistic plants in a test bed for control algorithms. In this paper, we propose an analytical solution of advertiser spend as a function of control signal by modeling the event rate of available impressions using a Gaussian Mixture Model (GMM). In this way, the plant gain can be directly derived based on its definition.
To the advertisers, it is always of interest to know how different performance metrics interact with each other. For example, how does the eCPA change as the budget increase? How much incremental budget advertisers can spend without violating their eCPA constraint? The above questions can be answered by examining different control response curves, since they are all monotonically increasing functions of the control signal. Building the different control response curves will provide valuable and actionable information to the advertisers and help them managing their campaigns more efficiently and gradually draw more budget to Verizon Media DSP.
The rest of this paper is organized as follows. The problem of forecasting for CPA lines in the middle of the flight is formulated mathematically in Section 2. The detailed steps of the algorithm is described in Section 3. Experimental results are provided in Section 4 and implementation of the algorithm in production is provided in Section 5. At last, in Section 6, we conclude and propose future work.
2. Problem Formulation
In Verizon Media DSP, there are two auction stages, one is internal auction within the DSP and the other is external auction that happens in ad exchanges. In internal auction stage, multiple lines bid on the same impression by submitting a bid price, called score. The highest bidder wins the chance to submit a bid to the ad exchange. In the second stage, the winner submits a bid price to external ad exchanges. We assume that the auction is second-price [Krishna, 2002] cost model, which means that the winning bidder pays an amount that equals to the highest competing bid for the impression. In practice, partial of the traffic is under a first-price model where the winning bidder pays an amount that equals to its bid price. But for traffic under first price model, the DSP try to mimic the behavior of second price auctions by implementing a scheme called bid shading [Karlsson, 2019], which tries to reduce the bid as close to the highest competing bid as possible so that the winner bidder will still be able to win but only pay an amount that almost equals the highest competing price. In this work, we assume perfect shading so that all traffic can be viewed as under second price model. The bid price submitted to both internal and external auctions are equal, and is denoted by .
Suppose there is a set of impressions, denoted by , that meet the targeting of a line, we call the impressions in available impression for the line. For every impression , we use to denote the highest competing price. The event rate
, defined as the probability of an awarded impression converting to a conversion, is denoted by. The advertiser cost, defined as the amount that advertiser pays to impression , is denoted by
. We assume that for available impression of the line, the three variables are independently draw from the same underlying joint distribution, namelywhere is the Probability Density Function (PDF).
It is well known [Karlsson, 2016] that a value-maximizing bidding strategy is to bid proportional to the value of the impression, which is the even rate . We define a control based bid price, represented by , as follows
where is the line level control signal, is the goal amount defined by the advertiser, representing the maximum cost of a conversion. The bid price for impression is calculated based on , with some line level deterministic transformation as will be described in details in Section 3.1
For CPA lines, the eCPA, denoted by , is an important measurement of performance and is defined as the average cost per conversion. Based on the above model, the number of impressions, total advertiser spend, number of conversions and eCPA can be expressed as the following
where is the indicator function, represents the expectation of over the distribution of . The focus of this paper is to forecast the relationship between control signal with key performance metrics, defined in Eq. (2).
The datasets that we use are internal auction data 222which is located in /projects/kite/prod/internal/core/raw_internal_auction/5m on db along with some other line metadata from kite database. The internal auction data is 1:4 sampled from raw bid requests. For each bid request, it records the lines whose targeting criteria is met along with the predicted event rate, internal auction bid price by that line. Thus, for each line, we have a 1:4 sampled pool of its available impressions, denoted by . And for each available impression, we have the four tuples in which represents internal auction bid price for impression from the line.
Note that the highest competing bid is derived. If the line won, the is the maximum of second highest bid price in internal auction and the inventory cost for impression ; otherwise, is the highest internal auction bid price.
It also needs to be noted that, a pacing signal, representing the probability that a line responses to each impression and denoted by , is applied in the production, meaning that even an impression is considered to be available, with probability it appears in . Thus, to compensate for any , we proposed the normalization step in Section 3.3.1.
3.1. Bid Adjustment
Usually, advertisers will set a max bid representing the maximum value of bid price that they would pay to any impressions, denoted by . The control based bid price in Eq. (1) is capped by the max bid. The bid price is after further deducting fees and bid shaving from the capped value. We use to represent the multiplicative coefficient and use to represent the additive coefficient. Thus, throughout the paper we use the following to represent the overall bidding strategy in production
The line level parameters need to learn from historical data 333The parameters can also be queried from databases, but it adds challenge to building data pipeline. Thus, we choose to estimate directly from existing dataset which turns out to be more efficient.. The linear model parameterized by in Eq. (3) is to describe the procedures of deducting fees and bid shaving.
The parameters are learnt from the historical data pair event rate and internal auction score for all . The control signal in Eq. (3) corresponds to the daily average control value when the training set is collected. The parameter is obtained by solving the following constrained minimization problem
The constraints are interpreted from reasonable fees-deducting methods used in the DSP and are added to the optimization problem to avoid extremely abnormal estimates due to noise training dataset. Having an estimated , we are able to apply Eq. (3) to calculate bid price based on the event rate.
3.2. Event Rate Modeling
Through extensive experiments, we found that the correlationship (Pearson’s) between the and is very slight, with a median of for all 9968 active CPA lines in a single day. Similarly for the correlationship between and . Thus, we model the distribution of event rate independently from and ( and
are highly correlated based on the cost model). We choose GMM as a parametric model by observing the clustering behavior of data shown in Fig.1. The model is explicitly expressed as the following
where respectively represent the weight, mean and covariance of th Gaussian component and is the number of Gaussian components in the model. The parameters are estimated using Expectation Maximization (EM) algorithm, a well known algorithm for GMM parameter estimation. To choose the proper value of the hyper parameter , we use Bayesian Information Criterion (BIC), which avoids overfitting by adding a penalty term for the number of parameters in the model.
One important assumption that we made in this work is that the number of available impressions for a line is static over time given fixed line targeting, which is quite reasonable, especially considering the huge amount traffic in the DSP. Thus, we take the number of available impressions of today as the forecast for tomorrow. Let’s denote the 5-min available impressions in a day by , and denote the 5-min pacing signal by . We first adjust the available impressions at non-zero pacing signal to obtain a corresponding to . To facilitate the explanation, we define a set representing the index of all non-zero pacing rate, i.e., .
Then, we compensate for the Time of Day (TOD) pattern for where the pacing signal is zero due to either day parting, hard stop cap or any other reasons. The TOD pattern of Verizon Media DSP is modeled by a 2-term truncated Fourier series as follows [He and Karlsson, 2019]
where are model parameters that are estimated in a separate process. It can be easily verified that . Thus, the total number of available impressions , corresponding to the case when every 5-min pacing signal equals 1, would be expressed as the following
The above considers the 1:4 sampling rate of internal auction data, and a consistent winning rate of the DSP in external auction.
Once modeled the event rate distribution as a GMM with parameters obtained in Section 3.2, we are able to derive the number of impressions and total advertiser cost defined in Eq. (2). Assuming that event rate and highest competing price are independently distributed, the joint PDF can be written as the product of marginal distribution , where is the GMM PDF parameterized by . The number of impressions can be won under control signal can be written as follows
where represents the Cumulative Distribution Function (CDF). The last step transforms an integration into a summation, based on Monte Carlo theory, knowing that all are sampled from its distribution with probability . In a similar fashion, the total advertiser spend can be derived as follows
The major advantage of having an analytical form of advertiser spend as a function of control signal in Eq. (11) is to facilitate the calculation of plant gain, which is a significant challenge in the design of a control system for online advertising. It measures the sensitivity of delivery rate (the amount of budget spend in one hour) to the control signal and thus guide the adjustment of control signal to achieve targeted delivery. Based on the definition, which is the first derivative of with respective to , the plant gain can be obtained as follows
where the last step is by knowing as a function of control signal in the bid adjustment model defined in Eq. (3).
To calculate the number of conversions defined in Eq. (2), we used Monte Carlo method since no close form solution can be found as for and . In this method, we take samples from the GMM of event rate , represented by and the number events can be expressed as
The eCPA in Eq. (2) can be obtained by the following based on the definition
The internal auction data is a huge amount of data. Is it necessary to use all the data to make forecast? If not, how many samples we need for a desired confidence level? This subsection is to answer these questions.
We define a random variable
which is Bernoulli distributed
Basically, for a given control signal , if the bid price for impression is greater than the highest competing bid, then a realization of random variable , otherwise . So, for any value , an estimator of (we us as an abbreviation for in the remaining of this subsection, but knowing that it is a variable depending on the control signal ) would be the average values of , i.e.,
According to Central Limit Theory
(CLT), when independent random variables are added, their properly normalized sum tends to a normal distribution. Thus, we have
In our case, is the parameter to be estimated for every possible values of
. A confidence interval for the parameterwith confidence level , determined by , is an interval with the following property
Knowing the distribution of in Eq.(15), the left hand of the above can be written as follows
denotes the CDF of Gaussian distribution and the last step is by knowing thatachieves its maximum at for . To have the following confidence level satisfied
we can derive that
According to the above inequality, when choosing and , meaning that with probability , the estimated is within a very small vicinity of the ground true values of , i.e., , we need to take 10,000 samples from each line. Eq. (19) provides a guidance on sampling a sufficient number of samples from the raw internal auction data set based on desired confidence level of the impression vs. control curve, i.e., .
4. Experimental Results
In this section, we demonstrate how we applied the forecasting approach to generate the key performance metrics using one day historical data for eCAP lines. And we further validated our forecasting results.
4.1. Forecast Results
The actual control signal in production is in the range of , but the active range can be quite different from line to line. So for our forecast to be most informative, we first identify the active range of control signal for a line , where
corresponds to the control signal value above which no more impressions can be won.
The control response curve defined in Eq. (8) and (11) is shown in Fig 2. As can be seen from the equations, the response curves are monotonically increasing function of control signal. We also compared the curves calculated in Eq. (8) and (11) with numerical method, where a set of event rates are resampled from the GMM and number impressions and spend are counted numerically for every control signal. As shown in the figure, curves by the two methods are identical.
The plant gain is plotted out in Fig. 3, along with the plant gain calculated numerically which is a discrete representation of the function and can not be directly consumed by the controller and only serves as a reference for the analytical plant gain. It can be seen that the two curves, analytical and numerical, are almost overlap with each other. For control signal within , the plant gain is relatively high, corresponding to the steep changes of both impressions and advertiser spend in Fig. 2. Such information is used in the feedback control to determine how aggressive the control signal needs to change.
To advertisers, the plot in Fig. 4 gives insight on how much incremental budget they can spend with the DSP without eCPA exceeding the goal amount. According to the Optimization Mission Control 444yo/opmt-missioncontrol, there are around 39% CPA lines who has a delivery ratio and a performance ratio way below . Those lines can benefit from the results in Fig. 4 and suggestions can be made to the advertisers about much more budget we are able to spend safely.
To validate the forecasting results, we compared the forecasted number of impressions corresponding to the actual control values of day , denoted by , with the actual delivered impressions
. Note that, since the actual delivery corresponds to a pacing signal vector which may not be a all 1 vector, so we first normalize the actual delivery using the same method in Section3.3.1 and get a normalized actual delivery .We further define the forecasting bias, denoted by , as the ratio between the two, i.e., . The validation results is plotted in Fig. 5, where the bias is in logarithm to make over-forecast and under-forecast equally visible. It can be seen that the forecast is almost unbiased, and with probability 90%, the forecast is within times the actual delivery. And with probability 50%, the forecast is within times the actual delivery, which is a very good performance considering that such forecasting for eCPA lines is first available within Verizon Meidia DSP and the existing system can tens of times under-forecast or over-forecast for eCPM lines.
To implement the forecasting algorithm in production, we utilize Apache oozie workflow scheduler system to manage our job pipeline that can generate forecasting result for each line from GB raw data everyday. The workflow has three components: read and preprocess internal auction/controller data from kite and control report, run the forecasting algorithm and generate forecasting metrics, and finally validate last day’s forecasting result. Currently, the data pipeline is running and we are working refining the data-flow and enabling monitoring and visualization.
6. Conclusion and Future Work
In this paper, we proposed a novel approach for mid-flight forecasting for CPA lines by taking bidding strategy into account. We forecasted the number of impressions, total advertiser spend as analytical functions of the control signal, based on which we derived the plant gain, a key information metric for the design of feedback control for online advertising. The relationship between advertiser spend and eCPA performance is also investigated which provides advertisers actionable insights. The forecast results were validated against the actual delivery. In the future, we will generalize the method to other goal type lines. We will also expend our forecasting method to pre-flight which is a more challenging case due to the lack of line specific historical data.
- Model reference adaptive control of advertising systems. In 2017 American Control Conference (ACC), pp. 5482–5487. Cited by: §1, §1.
- Identification of seasonality in internet traffic to support control of online advertising. In 2019 American Control Conferece, Vol. . Note: External Links: Cited by: §3.3.1.
- Systems and methods for controlling online advertising campaigns. Google Patents. Note: US Patent App. 15/190,055 Cited by: §1.
- Applications of feedback control in online advertising. In 2013 American Control Conference, pp. 6008–6013. External Links: Cited by: §1.
- Adaptive control using heisenberg bidding. In 2014 American Control Conference, pp. 1304–1309. Cited by: §1.
- Control problems in online advertising and benefits of randomized bidding strategies. European Journal of Control 30, pp. 31–49. Cited by: §1, §2.
- Plant gain estimation in online advertising processes. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 2182–2187. Cited by: §1.
- Optimal real-time bidding mechanism for multi-objective ad campaigns. In The 15th Conference on Web and Internet Economics, pp. . Note: submitted Cited by: §2.
- Auction theory. Academic Press. Cited by: §2.
- Statistical plant modeling and simulation in online advertising. In 2017 American Control Conference (ACC), pp. 2176–2181. Cited by: §1.
- Feedback control of event rate in online advertising campaigns. Control Engineering Practice 75, pp. 126–136. Cited by: §1.