GLASSES: Relieving The Myopia Of Bayesian Optimisation

We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of future evaluations. Our novel algorithm, GLASSES, permits the consideration of dozens of evaluations into the future. This is done by approximating the ideal look-ahead loss function, which is expensive to evaluate, by a cheaper alternative in which the future steps of the algorithm are simulated beforehand. An Expectation Propagation algorithm is used to compute the expected value of the loss.We show that the far-horizon planning thus enabled leads to substantive performance gains in empirical tests.


page 4

page 6


Investigating the Decoders of Maximum Likelihood Sequence Models: A Look-ahead Approach

We demonstrate how we can practically incorporate multi-step future info...

Learning to Search with MCTSnets

Planning problems are among the most important and well-studied problems...

Lexicographic Optimisation of Conditional Value at Risk and Expected Value for Risk-Averse Planning in MDPs

Planning in Markov decision processes (MDPs) typically optimises the exp...

Incorporating Expert Prior in Bayesian Optimisation via Space Warping

Bayesian optimisation is a well-known sample-efficient method for the op...

On the Impact of the Cutoff Time on the Performance of Algorithm Configurators

Algorithm configurators are automated methods to optimise the parameters...

Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?

Bayesian optimisation presents a sample-efficient methodology for global...

Can Global Optimization Strategy Outperform Myopic Strategy for Bayesian Parameter Estimation?

Bayesian adaptive inference is widely used in psychophysics to estimate ...

1 Introduction

Figure 1:

A Bayesian network describing the

-step lookahead problem. The shaded node () is known, and the diamond node () is the current decision variable. All nodes are correlated with one another under the gp model. Note that the nested maximisation problems required for and integration problems required for and (in either case for ) render inference in this model prohibitively computationally expensive.

Global optimisation is core to any complex problem where design and choice play a role. Within Machine Learning, such problems are found in the tuning of hyperparameters

(Snoek et al., 2012), sensor selection (Garnett et al., 2010) or experimental design (González et al., 2014; Martinez-Cantin et al., 2009). Most global optimisation techniques are myopic, in considering no more than a single step into the future. Relieving this myopia requires solving the multi-step lookahead problem: the global optimisation of an function by considering the significance of the next function evaluation on function evaluations (steps) further into the future. It is clear that a solution to the problem would offer performance gains. For example, consider the case in which we have a budget of two evaluations with which to optimise a function over the domain . If we are strictly myopic, our first evaluation will likely be at , and our second then at only one of and . This myopic strategy will thereby result in ignoring half of the domain , regardless of the second choice. If we adopt a two-step lookahead approach, we will select function evaluations that will be more evenly distributed across the domain by the time the budget is exhausted. We will consequently be better informed about and its optimum.

There is a limited literature on the multi-step lookahead problem. Osborne et al. (2009) perform multi-step lookahead by optimising future evaluation locations, and sampling over future function values. This approach scales poorly with the number of future evaluations considered, and the authors present results for no more than two-step lookahead. (Marchant et al., 2014)

reframe the multi-step lookahead problem as a partially observed Markov decision process, and adopt a Monte Carlo tree search approach in solving it. Again, the scaling of the approach permits the authors to consider no more than six steps into the future. In the past, the multi-step look ahead problem was studied by

Streltsov and Vakili (1999) proposing a utility function that maximizes the total ‘reward’ of the algorithm by taking into account the cost of future computations, rather than trying to find the optimum after a fixed number of evaluations.

Interestingly, there exists a link between the multi-step lookahead problem and batch Bayesian optimisation (Ginsbourger et al., 2009; Azimi et al., 2012). In this later case, batches of locations rather than individual observations are selected in each iteration of the algorithm and evaluated in parallel. When such locations are selected greedily, that is, one after the other, the key to selecting good batches relies on the ability of the batch criterion of predicting future steps of the algorithm. In this work we will exploit this parallelism to compute a non-myopic loss for Bayesian optimisation.

This paper is organised as follows. In Section 2 we formalise the problem and describe the contributions of this work. Section 3 describe the details of the proposed algorithm. Section 4 illustrates the superior performance of glasses in a variety of test functions and we conclude in Section 5 with a discussion about the most interesting results observed in this work.

2 Background and challenge

2.1 Bayesian optimisation with one step look-ahead

Let be well behaved function defined on a compact subset . We are interested in solving the global optimization problem of finding

We assume that is a black-box from which only perturbed evaluations of the type , with , are available. Bayesian Optimization (bo

) is an heuristic strategy to make a series of evaluations

of , typically very limited in number, such that the the minimum of is evaluated as soon as possible (Lizotte, 2008; Jones, 2001; Snoek et al., 2012; Brochu et al., 2010).

Assume that points have been gathered so far, having a dataset . Before collecting any new point, a surrogate probabilistic model for is calculated. This is typically a Gaussian Process (gp) with mean function and a covariance function , and whose parameters will be denoted by . Let be the current available information: the conjunction of , the model parameters and the model likelihood type. Under Gaussian likelihoods, the predictive distribution for at

is also Gaussian with mean posterior mean and variance

where is the matrix such that , (Rasmussen and Williams, 2005).

Given the gp model, we now need to determine the best location to sample. Imagine that we only have one remaining evaluation () before we need to report our inferred location about the minimum of . Denote by , the current best found value. We can define the loss of evaluating this last time at assuming it is returning as

Its expectation is

where the subscript in refers to the fact that we are considering one future evaluation. Giving the properties of the gp, can be computed in closed form for any . In particular, for

the usual Gaussian cumulative distribution function, we have that

where we have abbreviated as and as . Finally, the next evaluation is located where gives the minimum value. This point can be obtained by any gradient descent algorithm since analytical expressions for the gradient and Hessian of exist (Osborne, 2010).

Figure 2: A Bayesian network describing our approximation to the -step lookahead problem. The shaded node () is known, and the diamond node () is the current decision variable, which is now directly connected with all future steps of the algorithm. Compare with Figure 1: the sparser structure renders our approximation computationally tractable.

2.2 Looking many steps ahead

Expression (2.1) can also be used as a myopic approximation to the optimal decision when evaluations of remain available. Indeed, most bo methods are myopic and ignore the future decisions that will be made by the algorithm in the future steps.

Let for be the remaining available evaluations and by the available information after the data set has been augmented with and the parameters of the model updated. We use to denote the expected loss of selecting given and considering future evaluations. A proper Bayesian formulation allows us to define this long-sight loss (Osborne, 2010) as111We assume that .



is the predictive distribution of the gp at and

reflects the optimization step required to obtain after all previous the evaluations have been iteratively optimized and marginalized. The graphical probabilistic model underlying (2) is illustrated in Figure 1.

To evaluate Eq. (2) we can successively sample from to and optimize for the appropriate . This is in done in (Osborne, 2010) for only two steps look ahead. The reason why further horizons remain unexplored is the computational burden required to compute this loss for many steps ahead. Note that analytical expression are only available in the myopic case .

2.3 Contributions of this work

The goal of this work is to propose a computationally efficient approximation to Eq. (2) for many steps ahead able to relieve the myopia of classical Bayesian optimization. The contributions of this paper are:

  • A new algorithm, glasses, to relieve the myopia of Bayesian optimisation that is able to efficiently take into account dozens of steps ahead. The method is based on the prediction of the future steps of the myopic loss to efficiently integrate out a long-side loss.

  • The key aspect of our approach is to split the recursive optimization marginalization loop in Eq. (2) into two independent optimisation-marginalization steps that jointly act on all the future steps. We propose an Expectation-Propagation formulation for the joint marginalisation and we discuss different strategies to carry out the optimisation step.

  • Together with this work, we deliver a open source Python code framework222 containing a fully functional implementation of the method useful to reproduce the results of this work and applicable in general global optimisation problems. As we mentioned in the introduction of this work, there exist a limited literature in bo non-myopic methods and, to our knowledge, none available generic bo package can be used with myopic loss functions.

  • Simulations: New practical experiments and insights that show that non-myopic methods outperform myopic approaches in a benchmark of optimisation problems.

3 The GLASSES algorithm

Figure 3: Top row: (left) Myopic expected loss computed after 10 observations of the Six-Hump Camel function; (center, case 1) predicted steps ahead when the putative point, grey star, is close to the global optimum of the acquisition; (right, case 2) predicted steps ahead when the putative input is located far from the optimum of the acquisition. Bottom row: iterative decision process that predicts that 5 steps look ahead of case 2. Every time a point is selected, the loss is penalised in a neighbourhood, encouraging the next location to be selected far from any previous location point but still in a region where the value of the loss is low.

As detailed in the previous section, a proper multi-step look ahead loss function requires the iterative optimization-marginalization of the future steps, which is computationally intractable. A possible way of dealing with this issue is to jointly model our epistemic uncertainty over the future locations

with a joint probability distribution

and to consider the expected loss



the vector of future evaluations of

and X the dimensional matrix whose rows are the future evaluations . is multivariate Gaussian, since it corresponds to the predictive distribution of the gp at X. The graphical probabilistic model underlying (2) is illustrated in Figure 2. differs from in the fact that all future evaluations are modelled jointly rather then sequentially. A proper choice of is crucial here. An interesting option would be to choose to be some determinantal point process (dpp)333 A determinantal point process is a probability measure over sets that is entirely characterised by the determinant of some (kernel) function. defined on (Affandi et al., 2014) and integrate Eq. (3) with respect to by averaging over multiple samples (Kulesza and Taskar, 2012, 2011). DPPs provide a density over sample locations that induces them to be dissimilar to each other (as well-spaced samples should), but that can be concentrated in chosen regions (such as regions of low myopic expected loss). However, although DPPs have nice computational properties in discrete sets, here we would need to take samples from the dpp by conditioning on and the number of steps ahead. Although this is possible in theory, the computational burden of generating these samples will make this strategy impractical.

An alternative and more efficient approach, that we explore here, is to work with a fixed X, which we assume it is computed beforehand. As we show in this section, although this approach does not make use of our epistemic uncertainty on the future steps, it drastically reduces the computational burden of approximating .

3.1 Oracle multiple steps look-ahead expected loss

Suppose that we had access to an oracle function able to predict the future locations that the loss would suggest if we started evaluating at . In other words, takes the putative location as input and it returns and the predicted future locations . We work here under the assumption that the oracle has perfect information about the future locations, in the same way we have have perfect information about the locations that the algorithm already visited. This is an unrealistic assumption in practice, but it will help us to set-up our algorithm. We leave for the next section the details of how to marginalise over .

Assume, for now, that exists and that we have access to it. We it and denote by the vector of future locations evaluations of at . Assuming that is known, it is possible to rewrite the expected loss in Eq. (2) as


where the expectation is taken over the multivariate Gaussian distribution, with mean vector

and covariance matrix , that gives rise after marginalizing the posterior distribution of the GP at . Note that under a fixed , it also holds that . See supplementary materials for details.

The intuition behind Eq. (4) is as follows: the expected loss at is the best possible function value that we expect to find in the next steps, conditional on the first evaluation being made at . Rather than merely quantifying the benefit provided by the next evaluation, this loss function accounts for the expected gain in the whole future sequence of evaluations of . As we analyse in the experimental section of this work, the effect of this is an adaptive loss that tends to be more explorative when more remaining evaluations are available and more exploitative as soon as we approach the final evaluations of .

  Input: dataset , number of remaining evaluations (), look-ahead predictor .
  for  to  do
     1. Fit a gp with kernel to .
     2. Build a predictor of the future evaluations: .
     3. Select next location by taking .
     4. Evaluate at and obtain .
     5. Augment the dataset .
  end for
  Fit a gp with kernel to
  Returns: Propose final location at .
Algorithm 1 Decision process of the glasses algorithm.
Figure 4: Expected loss for different number of steps ahead in an example with 10 data points and the Six-hump Camel function. Increasing the number of steps-ahead flattens down the loss since it is likely for the algorithm to hit a good location irrespective of the initial point (all candidate points look better because of the future chances of the algorithm to be in a good location).

To compute Eq. (4) we use Expectation Propagation, ep, (Minka, 2001). This turns out to be a natural operation by observing that

where and

See supplementary materials for details. The first term in Eq. (3.1) is a Gaussian probability on an unbounded polyhedron in which the limits are aligned with the axis. The second term is the sum of the Gaussian expectations on different non-axis-aligned different polyhedra defined by the indicator functions. Both terms can be computed with ep using the approach proposed in (Cunningham et al., 2011). In a nutshell, to compute the integrals it is possible to replace the indicator functions with univariate Gaussians that play the role of soft-indicators in the ep iterations. This method is computationally efficient and scales well for high dimensions. Note that when , Eq. (4) reduces to Eq. (2.1).

Under the hypothesis of this section, the next evaluation is located where gives the minimum value. We still need, however, to propose a way to approximate the oracle . We do in next section.

3.2 Local Penalisation to Predicting the Steps Ahead

This section proposes an empirical surrogate for . A sensible option would be to use the maximum a posteriori probability, map, of the above-mentioned dpp. However, as it is the generation of dpp samples, to compute the map of a dpp is an expensive operation (Gillenwater et al., 2012). Alternatively, here we use some ideas that have been recently developed in the batch Bayesian optimisation literature. In a nutshell, batch bo methods aim to define sets of points in where should be evaluated in parallel, rather than sequentially. In essence, a key aspect to building a ‘good’ batch is the same as to computing a good approximation for : to find a set of good locations at which to evaluate the objective.

In this work we adapt to our context the batch bo idea proposed by González et al. (2015). Inspired by the repulsion properties of dpp, González et al. (2015) propose to build each batch by iteratively optimising and penalising the acquisition function in a neighbourhood of the already collected points by means of some local penalisers . Note that any other batch method could be used here, but we consider this approach since it is computationally tractable and scales well with the size of the batches (steps ahead in our context) and the dimensionality of the problem.

More formally, assume that the objective function is -Lipschitz continuous, that is, it satisfies that , . Take and valid Lipschitz constant . It is possible to show that the ball


where , doesn’t contain the minimum of . Probabilistic versions of these balls are used to define the above mentioned penalisers by noting that . In particular, is chosen as the probability that , any point in that is a potential candidate to be a minimum, does not belong to : As detailed in González et al. (2015), these functions have a closed form and create an exclusion zone around the point . The predicted k-th location when looking at step ahead and using as putative point is defined as


for and where are local local penalizers centered at and is the soft-plus transformation .

To illustrate how computes the steps ahead we include Figure (3). We show the myopic loss in a bo experiment together with predicted locations by for two different putative points. In case 1, the putative point (grey star) is close to the location of the minimum of the myopic loss (blue region). In case 2, is located in an uninteresting region. In both cases the future locations explore the interesting region determined for the myopic loss while the locations of the points are conditioned to the first location. In the bottom row of the figure we show how the points are selected in Case 1. The first putative point creates an exclusion zone that shifts the minimum of the acquisition, where the next location is selected. Iteratively, new locations are found by balancing diversity (due to the effect of the exclusion areas) and quality (exploring locations where the loss is low), similarly to the way samples the probabilities over subsets can be characterised in a dpp (Kulesza and Taskar, 2012).

3.3 Algorithm and computational costs

All the steps of glasses are detailed in Algorithm 1. The main computational cost is the calculation of the steps ahead, which is done using a sequence of l-bfgs optimizers at complexity for , the maximum number of l-bfgs updates. The use of ep to compute the value of the expected loss at each location requires a quadratic run time factor update in the dimensionality of each Gaussian factor.

4 Experiments

4.1 Interpreting the non-myopic loss

The goal of this experiment is to visualise the effect on the expected loss of considering multiple steps ahead. To this end, we use six-hump camel function (see Table 4 for details). We fit a gp with a square exponential kernel and we plot the myopic expected loss together with 5 variants that consider 2, 3, 5, 10 and 20 steps ahead. Increasing the steps ahead decreases the optimum value of the loss: the algorithm can visit more locations and the expected minimum is lower. Also, increasing the steps ahead flattens down the loss: it is likely to hit a good location irrespective of the initial point so all candidate looks better because of the future chances of the algorithm to be in a good location. In practice this behaviour translates into an acquisition function that becomes more explorative as we look further ahead.

4.2 Testing the effect of considering multiple steps ahead

Name Function domain
SinCos 1
Cosines 2
Branin 2
Sixhumpcamel 2
McCormick 2
Dropwave 2
Beale 2
Powers 2
Alpine2- 2, 5, 10
Ackley- 2, 5
Table 1: Details of the functions used in the experiments. The explicit form of these functions can be found at, (Molga and Smutnicki, 1995) and the supplementary materials of this work.
SinCos 0.7147 0.6058 0.7645 0.8656 0.6027 0.4881 0.8274 0.9000
Cosines 0.8637 0.8704 0.8161 0.8423 0.8118 0.7946 0.7477 0.8722
Branin 0.9854 0.9616 0.9900 0.9856 0.9673 0.9824 0.9887 0.9811
Sixhumpcamel 0.8983 0.9346 0.9299 0.9115 0.9067 0.8970 0.9123 0.8880
Mccormick 0.9514 0.9326 0.9055 0.9139 0.9189 0.9283 0.9389 0.9424
Dropwave 0.7308 0.7413 0.7667 0.7237 0.7555 0.7293 0.6860 0.7740
Powers 0.2177 0.2167 0.2216 0.2428 0.2372 0.2390 0.2339 0.3670
Ackley-2 0.8230 0.8975 0.7333 0.6382 0.5864 0.6864 0.6293 0.7001
Ackley-5 0.1832 0.2082 0.5473 0.6694 0.3582 0.3744 0.6700 0.4348
Ackley-10 0.9893 0.9864 0.8178 0.9900 0.9912 0.9916 0.8340 0.8567
Alpine2-2 0.8628 0.8482 0.7902 0.7467 0.5988 0.6699 0.6393 0.7807
Alpine2-5 0.5221 0.6151 0.7797 0.6740 0.6431 0.6592 0.6747 0.7123
Table 2: Results for the average ‘gap’ measure (5 replicates) across different functions. el-k is the expect loss function computed with steps ahead at each iteration. glasses is the glasses algorithm, mpi is the maximum probability of improvement and gp-lcb is the lower confidence bound criterion. The best result for each function is bolded. In italic, the cases in which a non-myopic loss outperforms the myopic loss are highlighted.

To study the validity of our approximation we choose a variety of functions with a range of dimensions and domain sizes. See Table 1 for details. We use the full glasses algorithm (in which at each iteration the number of remaining iterations is used as the number of steps-ahead) and we show the results when 2, 3, 5, and 10 steps look-ahead are used to compute the loss function. Each problem is solved 5 times with different random initialisations of 5 data points. The number of allows evaluations is 10 times the dimensionality of the problem. This allows us to compare the average performance of each method on each problem. As baseline we use the myopic expected loss, el. For comparative purposes we used other two loss functions are commonly used in the literature. In particular, we use the Maximum Probability of Improvement, mpi, and the gp lower confidence bound, gp-lcb. In this later case we set the parameter that balances exploration and exploitation to 1. See (Snoek et al., 2012) for details on these loss functions. All acquisition functions are optimised using the dividing rectangles algorithm direct (Jones et al., 1993). As surrogate model for the functions we used a gp with a squared exponential kernel plus a bias kernel (Rasmussen and Williams, 2005). The hyper-parameters of the model were optimised by the standard method of maximising the marginal likelihood, using l-bfgs (Nocedal, 1980) for 1,000 iterations and selected the best of 5 random restarts. To compare the methods we used the ‘gap’ measure of performance (Huang et al., 2006), which is defined as

where represents the evaluation of the objective function, is the global minimum, and and are the first and best evaluated point, respectively. To avoid gap measures larger that one due to the noise in the data, the measures for each experiment are normalized across all methods. The initial point was chosen to be the best of the original points used to initialise the models.

Table 2 shows the comparative across different functions and methods. None of the methods used is universally the best but a non myopic loss is the best in 6 of the 11 cases. In 3 cases the full glasses approach is the best of all methods. Specially interesting is the case of the McCormick and the Powers function. In these two cases, to increase the number of steps ahead used to compute the loss consistently improve the obtained results. Note as well that when the glasses algorithm is not the best global method it typically performs closely to the best alternative which makes it a good ‘default’ choice if the function to optimise is expensive to evaluate.

5 Conclusions

In this paper we have explored the myopia in Bayesian optimisation methods. For the first time in the literature, we have proposed an non-myopic loss that allows taking into account dozens of future evaluations before making the decision of where to sample the objective function. The key idea is to jointly model all future evaluations of the algorithm with a probability distribution and to compute the expected loss by marginalising them out. Because this is an expensive step, we avoid it by proposing a fixed prediction of the future steps. Although this doesn’t make use of the epistemic uncertainty on the steps ahead, it drastically reduces the computation burden of approximating the loss. We made use of the connection of the multiple steps ahead problem with some methods proposed in the batch Bayesian optimisation to solve this issue. The final computation of the loss for each point in the domain is carried out by adapting ep to our context. As previously suggested in Osborne et al. (2009), our results confirm that using a non-myopic loss helps, in practice, to solve global optimisation problems. Interestingly, and as happens with any comparison of loss functions across many objective functions, there is not a universal best method. However, in cases in which glasses is not superior, it performs very closely to the myopic loss, which makes it an interesting default choice in most scenarios.

Some interesting challenges will be addressed in the future such as making the optimisation of the loss more efficient (for which direct is employed in this work): although the smoothness of the loss is guaranteed if the steps ahead are consistently predicted for points close in the space, if the optimization of the steps ahead fails, the optimization of the loss may be challenging. Also, the use of non-stationary kernels, extensions to deal with to high dimensional problems and finding efficient was of sampling many steps ahead will also be analysed.


  • Affandi et al. [2014] Raja H. Affandi, Emily .B. Fox, and Ben Taskar. Approximate inference in continuous determinantal processes. In Neural Information Processing Systems 26. MIT Press, 2014.
  • Azimi et al. [2012] Javad Azimi, Ali Jalali, and Xiaoli Zhang Fern. Hybrid batch Bayesian optimization. In Proceedings of the 29th International Conference on Machine Learning, 2012.
  • Brochu et al. [2010] Eric Brochu, Vlad M. Cora, and Nando De Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010.
  • Cunningham et al. [2011] John P. Cunningham, Philipp Hennig, and Simon Lacoste-Julien. Gaussian probabilities and expectation propagation. arXiv:1111.6832 [stat], Nov 2011. arXiv: 1111.6832.
  • Garnett et al. [2010] Roman. Garnett, Michael. A. Osborne, and Stephan. J. Roberts. Bayesian optimization for sensor set selection, page 209–219. ACM, 2010. ISBN 1605589888. doi: 10.1145/1791212.1791238.
  • Gillenwater et al. [2012] Jennifer Gillenwater, Alex Kulesza, and Ben Taskar. Near-optimal MAP inference for determinantal point processes. In F. Pereira, C.J.C. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 2735–2743. Curran Associates, Inc., 2012.
  • Ginsbourger et al. [2009] David Ginsbourger, Rodolphe Le Riche, and Laurent Carraro. A multi-points criterion for deterministic parallel global optimization based on Gaussian processes. HAL: hal-00260579, 2009.
  • González et al. [2014] Javier González, Joseph Longworth, David James, and Neil Lawrence. Bayesian optimisation for synthetic gene design. NIPS Workshop on Bayesian Optimization in Academia and Industry, 2014.
  • González et al. [2015] Javier González, Zhenwen Dai, Philipp Hennig, and Neil D Lawrence. Batch Bayesian optimization via local penalization. arXiv preprint arXiv:1505.08052, 2015.
  • Huang et al. [2006] D. Huang, T. T. Allen, W. I. Notz, and N. Zeng. Global optimization of stochastic black-box systems via sequential kriging meta-models. J. of Global Optimization, 34(3):441–466, March 2006. ISSN 0925-5001.
  • Jones et al. [1993] D. R. Jones, C. D. Perttunen, and B. E. Stuckman. Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl., 79(1):157–181, October 1993. ISSN 0022-3239.
  • Jones [2001] Donald R. Jones. A taxonomy of global optimization methods based on response surfaces. Journal of global optimization, 21(4):345–383, 2001.
  • Kulesza and Taskar [2011] Alex Kulesza and Ben Taskar. k-dpps: Fixed-size determinantal point processes. In Lise Getoor and Scheffe, editors, ICML, 2011.
  • Kulesza and Taskar [2012] Alex Kulesza and Ben Taskar. Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning, 5(2–3):123–286, 2012. ISSN 1935-8237. doi: 10.1561/2200000044.
  • Lizotte [2008] Daniel James Lizotte. Practical Bayesian Optimization. PhD thesis, University of Alberta, 2008. AAINR46365.
  • Marchant et al. [2014] Roman Marchant, Fabio Ramos, and Scott Sanner. Sequential Bayesian optimisation for spatial-temporal monitoring. In

    Proceedings of the International Conference on Uncertainty in Artificial Intelligence

    , 2014.
  • Martinez-Cantin et al. [2009] Ruben Martinez-Cantin, Nando de Freitas, Eric Brochu, José Castellanos, and Arnaud Doucet. A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot. Autonomous Robots, 27(2):93–103, August 2009. ISSN 0929-5593, 1573-7527. doi: 10.1007/s10514-009-9130-2.
  • Minka [2001] Thomas P. Minka.

    Expectation propagation for approximate Bayesian inference.

    In Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, UAI ’01, pages 362–369, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. ISBN 1-55860-800-1.
  • Molga and Smutnicki [1995] Marcin Molga and Czesław Smutnicki. Test functions for optimization needs. NIPS Workshop on Bayesian Optimization in Academia and Industry, 1995. URL
  • Nocedal [1980] Jorge Nocedal. Updating quasi-Newton matrices with limited storage. Mathematics of Computation, 35(151):773–782, 1980.
  • Osborne [2010] Michael Osborne. Bayesian Gaussian Processes for Sequential Prediction, Optimisation and Quadrature. PhD thesis, University of Oxford, 2010.
  • Osborne et al. [2009] Michael A. Osborne, Roman Garnett, and Stephen J. Roberts. Gaussian processes for global optimization. In 3rd international conference on learning and intelligent optimization (LION3), pages 1–15, 2009.
  • Rasmussen and Williams [2005] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, 2005.
  • Snoek et al. [2012] Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, page 2951–2959, 2012.
  • Streltsov and Vakili [1999] Simon Streltsov and Pirooz Vakili. A non-myopic utility function for statistical global optimization algorithms. J. Global Optim., 14(3):283–298, 1999.

S1 Oracle Multiple Steps look-ahead Expected Loss

Denote by the value of the best visited location when looking at evaluations in the future. Note that reduces to the current best lost in the one step-ahead case. It is straightforward to see that

It holds hat

where the integrals with respect to are , since we don’t need to optimize for any location and . Notice that

and therefore

S2 Formulation of the Oracle Multiple Steps loook-ahead Expected Loss to be computed using Expectation Propagation

Assume that . Then we have that

The first term can be written as follows:

where . We can do this because the regions are disjoint and it holds that . Also, note that the can be replaced within the integrals since within each it holds that . Rewriting the integral in terms of indicator functions we have that


where if and otherwise.

The second term can be written as


where . Merge (S.1) and (S.2) to obtain Eq. (3.1).

s2.1 Synthetic functions

In this section we include the formulation of the objective functions used in the experiments that are not available in the references provided.

Name Function
Cosines .
Table 3: Functions used in the experimental section.