There are many reasons why a business may choose to outsource a particular task, job or process. Some of the recognized benefits of outsourcing include an improved focus on core business activities, increased efficiency due to the task being performed by specialists, and reduced costs. Outsourcing has gained wide-spread popularity in a wide range of application areas, including finance [Reddy2008], IT [LACITY2009], cloud computing [Motahari2009], supply chain management [Chli-old, Chli2015], and marketing [Kotabe2011]
. The focus of this paper is on outsourcing applications where both the outsourcer and the service providers are automated software agents and the task can be procured and delivered by an automated computer system. Although there are many benefits, outsourcing presents a number of challenges that must be addressed. First, when a task is outsourced, the outsourcing agent, the consumer, has no direct control over the process. Therefore, it may face some uncertainties in the task’s delivery time. Uncertain delivery times require a consumer with a strict deadline for task execution to deal with a tradeoff between the probability of completing the task on time and the overall cost. Recruiting faster and more reliable providers improves the task’s success probability but at the expense of an increased cost. The balance of this tradeoff varies from situation to situation and is greatly influenced by parameters such as the deadline for task execution, the task’s value for the consumer, and the diversity of service providers’ costs and speeds. For example, the higher the value of the task, the more money the consumer is willing to spend to reduce the risk of failure. On the other hand, when the task is more urgent, the consumer may place more weight on the providers’ speeds rather than their costs, when evaluating providers. Some of the information needed to find the optimal point of the success-cost tradeoff, such as the providers’ execution costs, is often unavailable to the consumer and must be extracted from the providers. In real-world applications, however, service providers are often self-interested and may misrepresent their private information upon request, if this promises to increase their profits. For instance, a self-interested provider, which only aims to maximize its own profit, may have an incentive to inflate its cost to earn higher revenue. Eliciting truthful information from self-interested providers is the second challenge faced by an outsourcing agent. The main goal of this paper is to design an outsourcing mechanism that optimally addresses the two challenges mentioned above. To address the first challenge, we introduce the following two outsourcing techniques that give the consumer sufficient degrees of freedom to adapt its strategy to situations:Redundant allocation and Gradual recruitment. Redundant allocation, which means outsourcing the same task to multiple service providers in parallel, is a powerful approach for improving the success probability [Ha2006]. The second technique, gradual recruitment, is an addition to redundant allocation, which enables the consumer to not recruit all providers at once, but gradually over time. Gradual recruitment allows the consumer to use the information revealed over time to adjust the success-cost tradeoff. Using this technique, the consumer can reduce its costs by first hiring some cheaper providers and then hiring more expensive and more reliable ones, if the deadline is approaching and the task is not completed yet. To address the challenge of information elicitation from selfish providers, we need to incorporate strong performance-based incentives into the outsourcing mechanism such that the providers find truthful information disclosure to be optimal. Existing incentive mechanisms that aim to balance the success-cost tradeoff in the presence of selfish providers restrict their attention to simultaneous deterministic outsourcing techniques, where a set of providers which are selected deterministically based on the reported costs are hired simultaneously to attempt the task in parallel [Zhang2018, Feldman2020]. This limited subset of outsourcing techniques does not include gradual recruitment where the recruitments happen not simultaneously but sequentially and are stochastic instead of deterministic, as each provider’s invocation depends on the uncertain behaviors of others hired earlier. In this paper, we show that sequential stochastic techniques such as gradual recruitment can help the consumer to significantly increase its revenue. However, to enable consumer to use such techniques in dealing with self-interested providers, the need for a novel incentive mechanism is heavily felt. To address this gap, we design a novel incentive mechanism, in the form of a procurement auction, that is flexible and adapts to the consumer’s needs by employing redundant allocation and gradual recruitment techniques. This auction, which we call WGPA, possesses the following desired properties at the same time: 1) Incentive compatibility: The auction provides sufficient incentives to providers to report their costs truthfully; 2) Interim- and ex-post individual rationality: It encourages providers to voluntarily participate in the auction (interim-individual rationality) and guarantees that they do not regret the decision they have made (ex-post individual rationality); and 3) Revenue-optimality: It provides the consumer with the highest possible revenue by achieving the optimal balance between the success probability and the total cost. The auction also guarantees that the consumer will never run into a deficit. An important point about our proposed revenue-optimal auction is that its payment function is different from that of the well-known threshold payment scheme which is proven to be optimal in auctions with deterministic allocation functions [Myerson1981]. We prove in this paper that the threshold payment scheme, which is also known as the critical payment scheme [Parkes2007]
, is not optimal for auctions with sequential stochastic allocation functions. For such auctions, we propose a “weighted” threshold payment scheme that guarantees incentive compatibility and individual rationality at the minimum cost. This payment scheme determines the price of recruiting each provider based on a weighted integral over the provider’s possible bid values. Using the weighted threshold payment scheme, we decompose the optimal auction design problem into several subproblems, each of which is a mixture of continuous and combinatorial optimization problems. We then give a branch-and-bound algorithm to solve each subproblem individually and accurately. We also present a low-complexity heuristic algorithm that can achieve performance very close to the optimal branch-and-bound algorithm. To test the effectiveness of the proposed auction, several benchmarks are adopted to demonstrate the performance in terms of revenue and social-welfare. The results show that employing the redundant allocation and gradual recruitment techniques can improve the revenue and social-welfare by up toand , respectively. Against this background, we advance the state-of-the-art in the following ways.
We propose the first payment scheme that can be used to design revenue-maximizing auctions in settings with stochastic and/or dynamic allocation.
We are the first to develop an incentive compatible and ex-post individually rational outsourcing mechanism that achieves the optimal tradeoff between success probability and overall cost. It achieves this by employing redundant allocation and gradual recruitment techniques.
Using the weighted threshold payment scheme, we present a low-complexity suboptimal incentive mechanism that retains incentive compatibility and individual rationality and achieves at least of the optimal expected revenue.
The rest of the paper is organized as follows. After a review of the main relevant literature (Section 2), a specification of our outsourcing problem is given in Section 3. In Section 4, we formally define redundant allocation and gradual recruitment as the two main outsourcing techniques used in this paper. In Section 5, we formulate the outsourcing problem with self-interested agents as an auction design problem and derive conditions that guarantee our desirable properties. Sections 6 and 7 are devoted to solving the optimal auction design problem introduced in Section 5. We first derive the payment function of the optimal auction in Section 6. We use this result in Section 7 to derive the optimal allocation function. The theoretical properties of the designed optimal auction are discussed in Section 8. In Section 9, we evaluate our proposed auction by simulations compared to several benchmarks. We conclude our paper in Section 10.
2 Related Work
Reaching a desired outcome when agents have private information can be achieved through some form of negotiation process [Sierra2001, Fatima2004, Jennings2005, Zheng2016]. In [Yao2010, Wang2021], the authors propose protocols that can be used for negotiation between the consumer and the service providers in an outsourcing application. These protocols provide a desired outcome when agents are truthful in information sharing. However, their performance significantly degrades when providers are selfish and may misreport their private information if it benefits them. Determining appropriate rules to achieve a desired outcome when providers are selfish is the subject of mechanism design [Myerson1988]. In a special case, when the consumer faces a single selfish provider, the goal could be achieved by designing a menu of contracts from which the provider can make a choice [Rogerson2003, Iyer2005, Chao2009]. However, when multiple providers exist and the outcome depends on all providers’ private information, a more general form of incentive mechanism is required [Borgers2015, Farhadi-faithful, Farhadi-interdependent]. Incentive mechanisms can be very diverse. However, according to the direct revelation principle [Myerson1979], the consumer can restrict attention to direct mechanisms where the providers are asked to reveal their private information in terms of bids, and the recruitment strategy and payments are determined based on these bids. Such mechanisms are often known as auctions and these are the basis for our work. Auctions are categorized into two groups based on the auction’s primary goal: efficient auctions or optimal auctions. The former are designed to maximize the social-welfare, while the latter aim to maximize the auctioneer’s expected revenue. Because optimality and efficiency usually cannot be achieved simultaneously, the auctioneer has to make the choice before starting. A cooperative auctioneer may prefer the efficient auctions, whereas a self-interested auctioneer may prefer the optimal auctions to gain more revenue. Most of the early works in the field have dealt with efficient auction design [Vickrey1961, Clarke1971, Groves1973]. The first insights into the design of optimal auctions have started with the works of Myerson [Myerson1981] and Riley and Samuelson [Riley1981] in 1981. Since then, much effort has been devoted to both issues in auction theory. In the following, we review the related works in each of these fields.
2.1 Social-Welfare Maximizing (Efficient) Auctions
The efficiency problem is theoretically solved by the Vickrey-Clarke-Groves (VCG) mechanism [Clarke1971, Groves1973] and its variants, including execution–contingent VCG [Ramchurn2009, Gerding2010], dynamic VCG [Bergemann2006], multi-stage VCG [Zhang2012], ad/position dependent cascade VCG (APDC-VCG) [Farina2017], and online VCG [Parkes2003]. Nevertheless, VCG-based mechanisms suffer from two drawbacks: (i) lack of ex-post individual rationality; and (ii) high computational complexity [Nisan2007-2]. To avoid such drawbacks, researchers started to develop some heuristic low-complexity mechanisms that approximate the optimal social-welfare while satisfying incentive compatibility and individual rationality. Examples of such approximations can be found in [Babaioff2009, Huang2012, Kraft2014, Jin2015, Stein-conference, STEIN2011]. In [Stein-conference, STEIN2011], the authors propose several service procurement mechanisms that vary in their information requirement. A pairing mechanism is the most general mechanism presented in [STEIN2011] that does not need tuning of the parameters. This mechanism first pairs providers randomly. Then, for each pair, it puts the provider with the lowest bid into a candidate set and assigns it a virtual cost equal to its pair’s bid. When this procedure ends, the mechanism restricts itself to providers in and computes the social-welfare maximizing recruitment strategy by assuming that the providers’ costs are equal to their virtual costs. Each provider will receive a payment equal to its virtual cost upon recruitment. The pairing mechanism is proved to be incentive compatible and ex-post individually rational and is currently the state-of-the-art in the class of approximate social-welfare maximizing mechanisms. The focus of this paper is not on designing efficient auctions, but rather on designing optimal auctions (whose related literature will be discussed in Section 2.2). However, we will show in Section 9 that our proposed auction is so well-designed that it also outperforms the current state-of-the-art (i.e. the pairing mechanism) in terms of social-welfare by up to .
2.2 Revenue Maximizing (Optimal) Auctions
The problem of designing an auction that maximizes the auctioneer’s revenue (optimal auction) is usually more challenging, as the ultimate goal depends not only on the allocations but also on the payments. Our work in this paper falls within the framework of optimal auction design, which is less studied due to its complexity. The study of optimal auctions started with the seminal work of Myerson in 1981 [Myerson1981]. In this work, Myerson introduced the first optimal auction for single-object problems. Since then, there has been some progress in extending Myerson’s fundamental result to multi-object environments, however, a general and analytical optimal auction in this framework has yet to be found [Nedelec2021]. The available literature on multi-object auction design can be categorized based on whether the objects are identical or not (homogeneous [Maskin1989, Malakhov2009, Pycia2021] vs. heterogeneous auctions [Vries2003, Ledyard2007, Xu2020]). The problem we investigate in this paper is a homogeneous multi-object auction design, as the auctioneer can create multiple copies of the task and assign them to different providers. However, there are two fundamental differences between our problem and the standard multi-object optimal auction design studied in the literature.
In available multi-object auctions, the decision is on the number or the set of objects (goods or tasks) assigned to each bidder. There is no time element in such auctions and it is often assumed that all assignments are made simultaneously. These auctions are often called simultaneous auctions [Zhang2018, Feldman2020]. There is a set of auctions called sequential auctions where the decisions about distinct sets of objects are made separately and sequentially in time [Tardos2012, Hosseinalipour2017, Donna2018, Narayan2019, Kong2021]. The goal of this approach is to simplify the optimal auction design problem by restricting attention to a subset of objects at each round. However, none of these works consider time as a decision factor. Our paper is the first work that considers allocation time as a deciding factor in the optimal auction design. Time adds a continuous aspect to the allocation design part of the problem and hence significantly increases its complexity.
Almost all of the revenue-maximizing auctions available in the literature, either single-object or multi-object auctions, utilize a deterministic allocation function. A deterministic allocation function determines a winning set of bidders for each object, based on the received bids, where bidders within the winning set receive and bidders out of the winning set do not receive the object for sure. In our problem, however, the recruitment process continues until the task is completed by at least one provider. Therefore, due to the uncertainty in delivery times, the recruitment of providers who are not first in the line is always conditional and probabilistic. The work presented in this paper is the first auction design that employs a non-deterministic allocation function.
The features mentioned above differentiate our work from the existing literature, making it the first work to address designing revenue-maximizing auctions when recruitments can take place at arbitrary points in time.
2.3 Robust Service Procurement
Our work is also related to the area of robust service procurement. There is a body of work that suggests the use of redundancy to overcome uncertainty. This is based on techniques that duplicate the critical components of a system in order to increase its reliability [Tillman1977, Coit1996]. The tools in this area are mainly focused on either maximizing the success probability or minimizing the cost. In this paper, instead, we concentrate on maximizing the service consumer’s revenue, which implicitly balances the success probability and cost. Redundancy can be achieved with either a parallel or a serial configuration. The parallel redundancy, where the providers attempt the task concurrently, has seen a large amount of research [Huhns2003, KOIDE2009, Zhang2009]. The serial redundancy, where a new provider is recruited when the previous service fails or takes too long, has also been studied in [Friese2005, Oinn2006, Erradi2006]. The serial redundancy is also known as gradual recruitment. Protocols that employ serial redundancy often use pre-defined deadlines to determine when to switch to an alternative provider [Oinn2006, Erradi2006]. A major drawback of the works within this area is that they rely on heuristic techniques to choose the number of redundant services and deadlines. The works of [Lukose2000] and [Glatard2007] tackle this drawback by studying when the current service should be timed out to invoke a new one. However, these studies are under the assumption that only one service provider could be active at any time. This shortcoming is addressed by [STEIN2011], where an algorithm for deriving the efficient combination of parallel and serial redundancy has been proposed. This algorithm gives a contingent plan for how different services should be invoked over time to maximize the social-welfare, when the providers’ cost information is publicly known. The authors also propose an incentive compatible and individually rational heuristic mechanism, called pairing, for settings with unknown cost information and selfish providers. However, as we show in Section 9 this heuristic mechanism is far from efficiency. Against the existing literature, our proposed approach is not heuristic, but rather systematic and guaranteed to produce the revenue-optimal redundancy-based procurement auction. Our proposed optimal auction is different from the first-past-the-post auctions, where the providers work on the task in parallel and then the first one that delivers the task is the only one that gets paid [Pandichi2011]. The main reason is that such redundancy-based auctions are not ex-post individually rational, as they cause regret to those that attempt the task but not win.
3 Problem Specification
A consumer would like a task to be completed before a deadline . The task has a value for the consumer if it is executed before the deadline. The consumer is no longer interested in the task when the deadline is passed. The consumer cannot accomplish the task itself and hence needs to outsource it to other agents that are capable of performing it. There are service providers (SPs) that can perform the task for the consumer. We denote the set of available SPs by . The consumer may be aware of this set in advance or obtain this information when it designs an auction and calls for bids. The time that each SP needs for executing the task is uncertain to the consumer, due to concurrent orders from other consumers, hardware or network problems and the provider’s scheduling policies. Therefore, the consumer may prefer to increase the chance of completing the task before the deadline, by procuring multiple SPs to attempt the task in parallel. However, executing the task has a cost for each provider, and the consumer must pay recruited providers a fee that covers at least their costs. Therefore, the consumer is looking for the optimal balance between the chance of success and the service invocation cost. This problem would be a standard optimization problem, if the costs are known to the consumer (as discussed in Section 1). However, each provider’s cost is its own private information and cannot be observed by either the consumer or other providers. The consumer cannot extract the providers’ costs by simply asking them, since the providers are strategic and have incentive to misreport their costs. Dealing with intelligent and strategic providers holding private information adds a new dimension to the problem, as the consumer needs to pay an extra cost to incentivize the providers to reveal truthful information. Our goal in this paper is to propose a flexible service procurement mechanism that can help a consumer to maximize its revenue when information is distributed among selfish providers. In the rest of this section, we model the main components of our problem precisely. A schematic of the problem is depicted in Fig. 1. We start the design process from Section 4.
3.1 Uncertainty in Service Provisioning
Providers may be different from each other in terms of the speed of performing the task. The time that each SP
requires to perform the task is uncertain. This uncertainty can be due to different reasons, e.g., the service lines of a SP might be less or more congested at different times; the resources required for performing the task could also be either available or unavailable at a given moment. We model this uncertainty by assuming that the timerequired by provider
to complete the task is not a deterministic, but a random variable obtained from a distribution with cumulative density function. We refer to function as provider ’s duration function and assume that for all . The providers’ duration functions are assumed to be independent and commonly known.
3.2 Providers’ Private Costs
Executing the task has a cost for each SP . The cost of each provider is drawn from an arbitrary distribution with support on the interval
. We denote the cumulative distribution function of the provider’s cost by and its corresponding density function by . The cost distribution of each SP is assumed to be regular, i.e., is increasing in . The regularity condition is satisfied by many distributions such as the uniform, normal, and Pareto distributions and is a common assumption in the auction literature [Salek2008, Feng2019]. The cost of each provider is only known to itself and is considered as provider ’s private information. The providers’ private information is not observed by either the other providers or the consumer. However, the consumer knows the distribution from which each is drawn. We assume that the costs of
providers are stochastically independent random variables. Thus, the joint density function for the cost vectoris . Of course, each provider considers its own cost to be a known quantity, not a random variable. However, we assume that SP
assesses the probability distribution for the other providers’ costs in the same way as the consumer does. That is, both the consumer and provider
assess the joint distribution function for the vectorof costs of all providers other than to be .
3.3 Providers’ Strategic Behavior
In order to invoke a service, the consumer has to make a payment to the provider at least equal to its cost. However, we discussed in Section 3.2, the consumer is not aware of the providers’ costs. In this paper, the providers are assumed to be self-interested and will act selfishly to maximize their own utilities. The utility of each provider is the difference between the payment it receives from the consumer and the cost it incurs for performing the task. If a self-interested provider is asked to declare a cost, it may have an incentive to either “mark up” the price to earn more profit or “mark down” the price to attract the consumer. In this situation, the consumer should design its recruitment strategy and the payments such that the providers cannot benefit by misreporting their costs. This can be done by designing an incentive compatible auction111We define the concepts of auctions and incentive compatibility formally in Section 5.[Brubaker1980]. The consumer announces the auction’s rules in advance and then each provider seeks an action that maximizes its own utility. The auction must be designed such that truth telling is the best strategy for each provider.
3.4 Consumer’s Objective
In most practical settings, the consumer is also self-interested and would like to maximize its own revenue. The consumer’s revenue is the difference between the value it gets from the task (if any) and all the payments it should make. The consumer gets a value from the task if it is executed successfully before the deadline. Thus, a self-interested consumer often faces a tradeoff between the success probability and the cost, as faster and more reliable SPs, which can increase the chance of getting the task done before the deadline, are often more expensive to hire. The goal of the consumer is to design auction rules such that it can lead the outcome to the optimum point of this tradeoff. In the next section, we present two outsourcing techniques that allow the consumer to better navigate this tradeoff. We incorporate these techniques into our auction design problem in Section 5.
4 Flexible Outsourcing Techniques
When the task’s delivery time is uncertain, a revenue-maximizing consumer with a strict deadline should look for effective ways to increase the success probability at a reasonable cost. In this section, we introduce two outsourcing techniques that can help the consumer to achieve this goal and lead the outcome to the optimal point of the success-cost tradeoff. Since the tipping point of this tradeoff depends on various factors, such as the task’s deadline and value along with the providers’ diversity in terms of reliability and cost, the outsourcing techniques introduced below provide the consumer with a high degree of freedom to adapt its strategy to situations.
4.1 Technique 1: Redundant Allocation
In this technique, the consumer increases the chance of task success by procuring multiple SPs to attempt the task in parallel. In this case, the consumer obtains value if at least one of the providers completes the task before the deadline . Completing the task by more than one SP does not provide any additional value to the consumer. However, the consumer cannot stop hired providers from performing the task when the task is completed by one SP or the deadline is reached. This means that the task is assumed to be non-interruptible and needs to run to completion once it started. In the redundant allocation technique, the consumer should make a payment to each hired provider to incentivize it to perform the task upon recruitment. This makes it apparent that recruiting more providers increases the consumer’s cost and could hurt the revenue if it is not managed properly. To avoid this negative effect, the consumer should decide wisely which set of providers leads to a better balance between the success probability and the total cost.
4.2 Technique 2: Gradual Recruitment
This technique is an extension to redundant allocation which aims to lower the cost while still retaining high success probability. Specifically, instead of hiring a set of providers all at the beginning, gradual recruitment lets the consumer make its decisions gradually over time. For example, the consumer has an option to hire some SPs at time and wait for a specific period of time (which is less than the deadline) to see whether any of the hired providers can complete the task. If they couldn’t, then the consumer may prefer to spend some money and recruit some new providers to increase the success likelihood. The gradual recruitment technique gives the consumer the opportunity to use the information revealed over time to better adjust the success-cost tradeoff. This ability can be illustrated by the following simple example.
Suppose that a high-valued task can be performed by two SPs. Provider is cheap to hire, but its delivery time varies based on its workload. Under a low workload, provider delivers the task after units of time. However, it needs at least time-units to deliver the task when working at a high workload. The second provider is expensive to hire, but it is totally reliable and always delivers the task by time . In this situation, if the consumer knew provider ’s workload, the optimal decision would be to recruit provider when the workload is low and to recruit provider when the workload is high. This information is not available to the consumer beforehand. However, the consumer can build a cheap experiment based on the gradual recruitment technique to first find out provider ’s workload and then make the decision accordingly. Specifically, the consumer can first recruit provider at a low cost and give it units of time to perform the task. Whether or not provider delivers the task by serves as a signal to the consumer to discover provider ’s workload. That is, provider ’s workload is low if and only if the job is delivered by time . If based on this information, the workload is revealed to be high, the consumer can spend more money to guarantee success by recruiting provider . This strategy provides the same value to the consumer as the one that recruits provider from the beginning (i.e., the probability of success is in both cases), however, the former incurred a much lower expected cost to the consumer than the latter.
Example 1 shows the ability of gradual recruitment technique to find and utilize new information over time. However, efficiently exploiting this potential can be challenging, as it presents the consumer with some new challenges. In addition to selecting a set of SPs to be hired, now the consumer also has to decide on the order of hiring sellers and the time gaps between them. We start facing this challenge in the next subsection, by embedding the concepts of redundant allocation and gradual recruitment into the consumer’s decision making problem.
4.3 Modeling Redundant Allocation and Gradual Recruitment
We define the consumer’s recruitment strategy as a contingent plan that specifies the providers that should be hired at specific times if the task is not completed yet.
Definition 1 (Recruitment strategy)
A recruitment strategy is a vector , where each element represents the time that provider is recruited and is an arbitrary integer number between and . The elements of this vector are all different in their first component (i.e., if ) and ordered increasingly based on their second component , i.e., , for all . Provider is recruited at time if and only if the task has not been completed so far.
A recruitment strategy specifies an ordered set of SPs that are candidates to be invoked, as well as the time of their invocation . The consumer starts by recruiting the provider with the first turn, i.e., , at time and waits up to time to see if the task is completed. If the task is not completed, the consumer increases the success probability by invoking provider , which is second in the line, to attempt the task in parallel with . This gradual recruitment continues until either the task is completed or all the candidate providers have been invoked. We denote the set of all recruitment strategies by . This set is completely general, it includes the extremes of “single strategies”, where the consumer recruits only one single provider (i.e., ), “simultaneous strategies”, where a set of providers are recruited simultaneously at time (i.e. ), and all conceivable intermediate strategies. The consumer would like to choose a recruitment strategy that maximizes its expected revenue. We denote the consumer’s expected revenue when it employs the recruitment strategy and makes payments to providers upon their recruitment, by
where is the expected value the consumer gets from task completion and is the expected cost of hiring providers. The effect of requirement strategy on the expected value and the expected cost is quite different and opposite. Recruiting more providers early on can raise the revenue by increasing the success probability and hence the expected value , but it can also reduce the revenue by increasing the total cost . Our goal in this paper is to propose an auction to find the optimal balance between these two conflicting factors. In the rest of this subsection, we quantify the expected value and the expected cost for any arbitrary recruitment strategy . Then, in Section 5, we design an auction to find the tipping point of the above-mentioned tradeoff.
4.3.1 Expected Value of a Recruitment Strategy
The consumer gets a value if the recruitment strategy is successful in delivering the task before deadline . A recruitment strategy is successful if at least one of the recruited SPs completes the task before the deadline. We define the completion time of the task when recruitment strategy is employed as the first completion time for providers and denote it by . The task completion time is a random variable whose distribution depends on the duration functions of the recruited providers. Since each , for , is drawn independently from distribution , the cumulative distribution function of can be derived as follows:
Based on (2), the success probability of strategy for a task with deadline can be formulated as
We can use (3) to derive the expected value of a recruitment strategy as
4.3.2 Expected Cost of a Recruitment Strategy
The consumer makes a payment to each provider upon recruitment. Therefore, the expected cost of a recruitment strategy when employed along with the payment strategy can be written as
where is the probability that provider will be invoked in the recruitment strategy . Providers out of the set of candidate providers have no chance for being hired, i.e., , where . However, each provider will be recruited at a specific time if all the providers with earlier turns fail to complete the task by then. To formalize this, we define the order of provider ’s invocation in the recruitment strategy as , where
Any provider will be invoked at time if all the providers with fail to complete the task by . Since the providers’ duration functions are independent random variables, we can derive provider ’s invocation probability as
where . It can be seen from (7) that the providers’ invocation probabilities are inversely related to their invocation orders; providers with earlier turns have higher chances of being recruited. By substituting (7) in (5), we can derive the consumer’s expected cost as
4.3.3 Expected Revenue of a Recruitment Strategy
The consumer’s utility (4.3.3) is decreasing in terms of the payments . Therefore, for each recruitment strategy , the consumer prefers to make the minimum possible payment to each provider. Each provider ’s received payment must be greater than or equal to its cost , as otherwise, the provider finds it unprofitable to perform the task and hence refuses to do so. If the consumer knew each provider’s cost , the payment could be simply equal to . However, as we mentioned earlier, the cost of each provider is its own private information and the consumer is not aware of it. To overcome this lack of knowledge, the consumer can conduct an auction and call for bids. However, since the providers are strategic, there is no guarantee that the submitted bids are equal to the actual costs. Receiving incorrect information from the providers may prevent the consumer from taking optimal decisions. Therefore, to avoid such fraudulent behaviors, the consumer would like to design the auction’s rules such that they leave no incentive for the providers to be untruthful. Based on the discussion above, the consumer’s ultimate goal is to design an auction that has the following three properties: 1) it is incentive compatible, i.e., it incentivizes providers to reveal the true information; 2) it is individually rational, i.e., it motivates providers to participate in the auction; and 3) it is revenue-maximizer, i.e., it finds the optimum point of the value-cost tradeoff faced by the consumer. We study the design of an auction with the above three features in the next section.
5 Optimal Auction Design Problem
In a procurement auction, the set of bids that a service provider may submit is equal to the set of private information it may have. In our problem, each provider’s private information consists of its cost for performing the task. Therefore, each provider ’s bid must belong to the set of possible costs and can be interpreted as provider ’s reported cost. Notice that each provider ’s reported cost is not necessarily equal to its actual cost . We denote the vector of all providers’ submitted bids by . A procurement auction is defined by an allocation function and a vector of payment functions . The allocation function determines the recruitment strategy that is selected by the consumer based on the received bids , and each payment function determines the amount of money that will be paid to provider , upon recruitment. As discussed in Section 4, the consumer’s goal is to design an auction that satisfies incentive compatibility, individual rationality, and revenue maximization. In the following subsections, we derive conditions on the auction’s allocation function and payment function that guarantee each of these properties. We then formulate the consumer’s auction design problem as an optimization problem.
5.1 Incentive Compatibility (IC)
Any procurement auction induces a non-cooperative game with incomplete information, where the set of players is the set of service providers , a bidding strategy of each player is a mapping from its true cost to the bid it submits, and each player ’s utility resulting from the action profile is
Provider ’s utility is the difference between the cost it incurs for performing the task and the payment it receives from the consumer if it gets hired (with prob. ), and zero otherwise. Procurement auction is said to be incentive compatible (IC) if truth-telling (i.e., ) is a BNE of the induced game [Borgers2015], i.e.,
holds for every , , and . For ease of notation in the future, we let denote the integral that appears in the right-hand side of (11). In fact, is provider ’s expected utility in an auction , when all providers except submit their costs truthfully, but provider with cost submits bid . Using this notation, the incentive compatibility constraint (11) can be rewritten as
5.2 Individual Rationality (IR)
An auction is individually rational (IR) for a provider, if the utility it gains from the auction is non-negative. There are different notions of IR in the literature; the two we focus on here are called interim IR and ex-post IR. The difference between these two notions is at the time the providers are allowed to drop out of the auction. In interim IR, providers should make their decisions before the auction starts. However, in ex-post IR, the providers are allowed to opt out of the auction even at the very end when the consumer decides to recruit them. The interim IR is guaranteed if the expected utility each provider with cost gains from participating in the auction is non-negative, i.e.
However, the ex-post IR requires that no provider has regrets regarding participation even if any bid vector is submitted by other providers, i.e.
Although ex-post IR is stronger and more applicable to our setting, it is also more difficult to deal with in the auction design problem. Therefore, in the design process, we relax this constraint to the interim IR constraint. However, at the end, we will theoretically show that the auction is sufficiently well-designed to satisfy the ex-post IR condition.
5.3 Revenue Maximization
In Section 4, we derived the consumer’s expected revenue when it employs the recruitment strategy and makes payments to providers upon their recruitment. In the truthful equilibrium of the game induced by auction , each bid vector is submitted with probability . When bid vector is received, the consumer employs the recruitment strategy and the payment strategy . Therefore, the consumer’s expected revenue in an incentive compatible auction is
The consumer would like to choose the allocation function and the payment function so as to maximize the expected revenue under the IC and IR constraints. This is equivalent to solving the following optimization problem:
6 Optimal Payment Function
Problem (14) is an optimization problem in the space of functions (which are infinite-dimensional vector spaces). Such problems are generally difficult to solve because of their high dimensionality. In this paper, we tackle optimization problem (14) by proceeding as follows. In Section 6.1, we derive a necessary and sufficient set of conditions for an auction to be incentive compatible and interim individually rational (i.e., satisfy constraints (14b)-(14c)). Replacing (14b)-(14c) with this set of constraints, we partially solve problem (14) and derive the optimal payment function in Section 6.2. The impact of this payment function on reducing the consumer’s cost will be discussed in Section 6.3. We finally substitute in (14) to express optimization problem (14) solely in terms of the allocation function in Section 6.4. Determining the optimal allocation function is the subject of the next section.
6.1 Simpler Conditions for IC and IR
as the conditional probability that service provider will be hired in the auction mechanism given that it submits bid .
An auction satisfies the individual rationality and incentive compatibility constraints if and only if the following conditions hold:
The conditional probability function defined in (15) is decreasing in its second argument, i.e.,
Each provider ’s expected utility when it tells the truth satisfies the two following conditions:
We present the proofs of all the lemmas and propositions in Appendices A-I. Condition 1 states that for a procurement auction to be incentive compatible, the allocation function must be monotone and give higher chances of selection to providers that report lower costs. This result is intuitively similar to Myerson’s monotonicity condition which is well-known in the mechanism design literature [Myerson1981, CARBAJAL2015]. However, there is a slight but important difference between the definition of monotonicity in our setting and that of Myerson. In most of the problems studied in the literature, the selection process is deterministic. That is, based on the received bids , the auctioneer selects a set of service providers and invokes all of them simultaneously. In such settings, monotonicity requires that if service provider gets invoked when it submits bid , it should also be invoked when it submits bid . However, in our problem, the selection process is gradual and stochastic. That is, each service provider that submits bid will be invoked with probability defined in (15). In this setting, monotonicity requires that a service provider that places a lower bid must have a greater probability of getting invoked. This is equivalent to the fact that is decreasing in its second argument. Condition 2 of Lemma 1 implies that in an incentive compatible and individually rational auction, each provider’s expected utility is positive and decreasing in its cost. This is a consequence of (17)-(18) and the fact that function is always positive.
6.2 Weighted Threshold Payment Scheme
The payment function of an optimal auction has the following form:
Lemma 2 gives the lowest amount of money that the consumer should pay to the service providers, based on their bids, to make the auction incentive compatible and individually rational. This payment is different from the well-known threshold payment that is widely used in the literature [Parkes2007, Wang2016]. The traditional threshold payment scheme which is also known as critical payment pays each provider the maximal cost that can bid and still gets invoked. This payment rule is designed for settings where the allocation function is deterministic and unconditional. However, in our auction design problem, the allocation function could be conditional and probabilistic. More precisely, based on the received bids, the consumer provides an ordered list of candidate providers along with their invocation times , and recruits each provider at time if the task has not completed by others yet. In this setting, increasing the declared cost may not completely remove a provider from the list of candidate providers, but could delay its turn and hence reduce the chance of being invoked. The payment function derived in (19) is the extension of the threshold payment scheme for auctions with probabilistic allocation functions. In our generalized payment scheme, which we call weighted threshold payment, the consumer considers the conditional probability that got hired if it reported , given that it is recruited when it submits bid , as the relative desirability of cost for provider . We denote this relative desirability by , where
It is clear from (20) that in monotone auctions the relative desirability is always between and . Based on (19), when the consumer recruits provider , it pays the provider its declared cost plus the integral of the relative desirability over all higher bids. Excess payment is an “information rent”, that is a payment made by the consumer to service providers in exchange for accurate disclosure of their private information.
6.3 Potential of Gradual Recruitment in Reducing the Consumer’s Cost
The optimal payment function (19) provides us a better insight on why gradual recruitment can reduce the consumer’s expected cost (see Section 4.3.2). The reason of this is two-fold: (i) reducing providers’ invocation probabilities and (ii) reducing the prices the consumer needs to pay to providers upon recruitment. The first reason is obvious from the discussion in Section 4. When the consumer employs a gradual recruitment strategy, it can stop the process as soon as one of the hired providers completes the task. This reduces the invocation probabilities of the providers with later turns, and eventually reduces the consumer’s expected cost. The second reason is less apparent and originates from the optimal payment function (19). We illustrate this by an example.
Consider a task with value and deadline . Suppose that there are two strategic service providers that can execute the task. The time needed by each of these providers to complete the task is exponentially distributed with mean , which is only known to itself, is drawn from uniform distribution over
. Suppose that there are two strategic service providers that can execute the task. The time needed by each of these providers to complete the task is exponentially distributed with mean, i.e., . Each provider ’s cost
, which is only known to itself, is drawn from uniform distribution over.
In this setting, we denote the best auction in the absence of gradual recruitment by . The allocation function , which can be obtained by using the techniques we will provide later in this paper, is shown in Fig. (a)a. This allocation function instructs the consumer to recruit both providers simultaneously at time if submitted bids and are both below . Otherwise, the consumer should only recruit the provider with the minimum submitted bid. The minimum price that the consumer should pay to the providers to make auction with allocation function incentive compatible and individually rational is derived based on (19), as follows:
In each pair returned by payment function , the first and second elements are the prices paid to providers 1 and 2, respectively, upon recruitment. Now, let’s study what will happen if the consumer takes advantage of gradual recruitment. In this case, it is optimal for the consumer to run auction with allocation function shown in Fig. (b)b222We discuss later in this paper how the optimal allocation function can be computed.. This allocation function divides the region with into three subregions below and provides a different recruitment plan for each subregion:
Subregion 1 (): Both providers get recruited simultaneously at time .
Subregion 2 (, ): Provider 1 gets recruited at time . Provider 2 is recruited in the future (i.e., at a time ), if the task is not completed by then.
Subregion 3 (, ): Provider 2 gets recruited at time . Provider 1 is recruited in the future (i.e., at a time ), if the task is not completed by then.
The optimal invocation time can be derived by techniques that we will develop in Section 7.1. However, for the sake of simplicity and since in this example our goal is just to show how gradual recruitment can reduce the payments, we consider a suboptimal invocation time , for all . With this invocation time, the invocation probability of the provider who is second in the queue is . For this allocation function, we can derive the optimal payment function according to (19), as follows:
Comparing (22) with (21), we can see that in each Subregion 1-3, the gradual recruitment reduces the total cost of incentivizing providers to submit true bids. This fact is more attractive and apparent in Subregion 1, as in this region, the allocation function is exactly the same as . Therefore, the gradual recruitment has not changed the providers’ invocation probabilities in this region. However, due to the changes it has made to the allocation functions of other regions, the payments of Subregion 1 are reduced as well. To see the reason, let’s take a closer look at for Subregion 1. In this subregion, the invocation probability of provider 1 is , i.e., . If provider 1 increases its bid , its invocation probability remains up until , and then drops to , for . Provider 1 will have no chance for being invoked if it submits a bid greater than . Based on these values, we can use (19) to compute payment for Subregion 1 as follows:
Comparing (23) with clarifies why payments of Subregion 1 are reduced when gradual recruitment is employed. In auction , if providers increase their bids, they will be at the risk of losing their first position in the recruitment queue and going to the second position. If this happens, the providers’ invocation probabilities and hence their expected utility decreases. Therefore, the providers have less incentive to submit higher bids and hence it is easier for the consumer to incentivize them to tell the truth.
Example 2 illustrates the impact of gradual recruitment and the weighted threshold payment scheme on lowering the consumer’s cost. In the next subsection, we use the result of this section to formulate the optimization problem we need to solve for computing the optimal allocation function.
6.4 Formulating Problem (14) in terms of the Allocation Function
Lemma 2 reduces the complexity of designing an optimal auction by expressing the optimal payment function in terms of the allocation function . This result helps us to focus on optimizing the allocation rule and then use equation (19) to derive the minimum payments required to make the auction with this allocation function incentive compatible and individually rational. Following this approach, in the next lemma, we state the optimal auction design problem solely in terms of the allocation function .
Suppose that the allocation function solves the following optimization problem:
Then, , where is derived based on (19), represents the optimal revenue-maximizing auction.
The objective function of problem (24) is similar in form to the expected social-welfare of an auction with allocation function , which is defined as
Social-welfare is the sum of the utilities of the consumer and the service providers in an auction. The expected social-welfare is derived by taking an expectation over the joint statistics of the providers’ costs. We can see that the objective function (24a) is similar to the expected social-welfare (25), except that the real cost of each provider is replaced by a virtual cost . We see by definition that the providers’ virtual costs are always greater than or equal to their actual costs, i.e., . This basically means that the cost that a revenue maximizing consumer evaluates for recruiting each provider is higher than its actual cost. The reason is simple. According to (19), the consumer should pay each recruited provider not only its actual cost , but also an information rent to induce the provider to disclose its private information. The difference between the virtual costs and the real costs of each provider is the expectation of the information rent that must be paid to the providers to reveal truthful information.
7 Optimal Allocation Function
The purpose of this section is to find an allocation function that solves optimization problem (24). For each and , the monotonicity constraint (24b) depends on the recruitment strategies that allocation function assigns not to a single cost vector, but rather to a set of cost vectors . This interaction among decision variables makes optimization problem (24) non-separable across cost vectors and hence difficult to solve. However, the next lemma proves that the monotonicity constraint (24b) is satisfied “for free” at the optimal allocation.
Lemma 4 allows us to relax the monotonicity constraint and be sure that this condition will be automatically satisfied at the final solution. If we drop the monotonicity constraint (24b), the problem becomes separable and can be decomposed into the following unconstrained subproblems , :
For each cost vector , problem aims to find the optimal recruitment strategy that specifies the optimal ordered set of providers and their optimal invocation times . This problem is a mixture of continuous and combinatorial optimization problems. Computing the optimal procurement times for each ordered set of providers is a continuous optimization problem. However, determining the optimal ordered set is a combinatorial optimization problem. Combinatorial optimization problems can be viewed as searching for the best element of some set of discrete items. Therefore, in principle, any sort of search algorithm can be used to solve them. However, the number of ordered subsets of providers can, in practice, become quite large (e.g. 1,956 for providers but 1,302,061,344 for providers). Exploring such a huge search space is computationally prohibitive and needs major accelerations in order to be used practically. We proceed as follows to solve optimization problem for each cost vector . In Subsection 7.1, we focus on the continuous part of and discuss how to derive the optimal invocation times for a fixed provider sequence . We make use of this result in Subsection 7.2 to develop a branch-and-bound algorithm for solving the combinatorial part of the problem.
7.1 Continuous Part: Optimal Invocation Times
In this part, we assume that the optimal subset of providers and their ordering is given. That is, we are given an ordered set of providers , where is invoked before . To compute the optimal invocation time, we must determine , where is the invocation time of , such that the objective function of problem gets maximized. Using (3) and (7), we can write the objective function of in terms of and , , as follows:
To derive the optimal invocation time, we must solve an optimization problem that maximizes under the following constraints: and
. This is a continuous non-convex optimization problem that can be solved numerically by existing methods such as stochastic gradient descent[Robbins1951] and saddle-free Newton [Dauphin2014]. We denote the function that returns the optimal invocation time for provider sequence by .
7.2 Combinatorial Part: Optimal Provider Sequence
One of the most universally applicable approaches for reducing the search space in combinatorial optimization problems is branch-and-bound algorithm. Such algorithms recursively split the search space into smaller groups; this splitting is called branching. Branching alone would amount to brute-force enumeration of candidate solutions and testing them all. To improve on the performance of brute-force search, a branch-and-bound algorithm keeps track of a lower bound and an upper bound for all solutions in given groups. The algorithm uses these bounds to prune the search space, eliminating groups that it can prove will not contain an optimal solution (i.e., its upper bound is less than the lower bound of another group). This systematic partitioning of the search space and removing the groups that cannot contain an optimal solution can potentially reduce the space of solutions that have to be searched. However, the performance of a branch-and-bound algorithm depends crucially upon the appropriate selection of branching and bounding techniques. In this paper, due to the similarity of the objective function of problem to the social-welfare (as discussed in Section 6.4), we adopt the branching and bounding techniques proposed in [STEIN2011] for deriving a social-welfare maximizing recruitment strategy . We briefly discuss each of these techniques below. Branching technique: The main idea is to partition the search space, which is the set of all ordered subsets of providers , based on their first elements. is a parameter that starts from and increases gradually over iterations. The pseudo-code of this algorithm is presented in Algorithm 1. In the first iteration, the search space is partitioned into groups (Line 3), where each represents the set of orderings that have as their first element. For example, for , we have . After this partitioning, the algorithm calculates a lower bound and an upper bound for the best consumer’s utility within each group (Lines 5-6) and keeps track of the highest lower bound found so far (Lines 7-10). We will discuss how the Lower and Upper functions work later. The highest lower bound serves as a touchstone for assessing the quality of the groups to be explored in the future. A group will be discarded if its upper bound is lower than or equal to (Lines 11-13). In the second iteration, the algorithm picks the group with the highest lower bound, as this group is most likely to contain the optimal ordering, and partitions it into smaller-sized groups based on the first two elements (Lines 16-18). The function in Line 18 sub-partitions group based on the next unfixed element. That is, if the orderings within match in their first elements, the function partitions them based on their - elements. The aim of this function is to create the next generation of groups to be explored more precisely. For example, if is picked at Line 16 when , the next generation of groups will be . These new groups will be evaluated based on the highest potential revenue they can provide for the consumer and will be discarded if their upper bound is below (Lines 19-29). The groups that survive from this generation along with the ones that survive from the first generation (Line 30) go for another round of branching and bounding. This process continues until there is no branchable group left.
Lower bounding technique: Deriving a lower bound for the highest expected utility the orderings within a group can provide to the consumer is simple. This is because the best expected utility provided by any member of the group can be interpreted as a lower bound for the maximum utility the whole group can provide to the consumer. The groups we consider in our branch-and-bound algorithm are of the form , where , and , for all . Each group is the set of all orderings that start with . We select ordering as a representative for group . In ordering , the consumer invokes in turn and does not invoke any other providers. The optimal invocation time for ordering is , where is the function derived in Section 7.1. Using this result, we can construct the best recruitment strategy with the ordering as . We consider the consumer’s expected utility when employing recruitment strategy as the lower bound for the highest expected utility members of the group can provide to the consumer. That is,
Upper bounding technique: Calculating an upper bound on the expected utility that members of a group can provide for the consumer is not immediately obvious. Obtaining a tight upper bound requires computing the utility provided by every member of the group, which is too cumbersome and probably impractical. However, we can obtain a looser (but still effective) upper bound by applying the following approach. For each group , we define
If , then is the only member of group . Therefore, the upper bound is equal to the lower bound discussed above. Otherwise, we create a virtual service provider that dominates all subsets of . This service provider has the minimum cost among all providers , i.e., , and can perform the task faster than any combination of providers , i.e.,