1 Introduction
Bayesian inference provides the fundamental basis for modeling uncertainty. The posterior distribution provides a complete summary of what is known about the parameters of a model given some observed data . In particular, the posterior is necessary and sufficient information for making optimal decisions under uncertainty [1].
. Our approach learns to correct for this mistake, by replacing the theoretically optimal decision with parametric decisionmaking module – a neural network – that maps quantiles of the incorrect predictive distribution into optimal decisions.
For predictive models , such as regression or classification, the decisions correspond to committing to individual decision that minimizes the risk , the expectation of a loss over the uncertainty in the model parameters. For example, to minimize the squared loss we should commit to the mean of the predictive distribution , whereas for a loss that penalizes for overestimation of the true value the optimal decision is to report a suitably chosen lower quantile of the predictive distribution. For many losses the Bayes optimal decision is a simple statistic of the predictive distribution (Table 1).
Unfortunately, the decisions obtained by minimizing the risk are optimal only if we have access to the true posterior . For most models we can only obtain an approximation
. The almost universally adopted practice is to plug in the approximation in lieu of the posterior in the predictive distribution and proceed as though the result were the true predictive distribution. Even though this works relatively well for good approximations (e.g., wellconverged Markov Chain Monte Carlo sampler), it may fail miserably for bad approximations. Furthermore, even for good approximations it would make sense to account for common biases; e.g., meanfield variational approximations are known to underestimate the posterior variation and hence also the variation in the predictive distribution
[2]. Consequently, using decision rules optimal for the true posterior will have systematic bias as well.Even though the problem persists in virtually all use of approximate Bayesian inference in decisionmaking problems, the literature addressing the issue is limited. The best examples build on the concept of losscalibrated inference. Originally proposed in the context of variational approximation by Lacoste–Julien et al. [3], the core idea is to take into account the eventual decision problem (characterized by some loss) already during inference by altering the learning objective for approximate inference. Recently Cobb et al. [4] and Kuśmierczyk et al. [5] have presented practical algorithms building on this principle, but despite the elegant theory the procedure runs into several practical difficulties: It is computationally heavy, requires knowing the decision loss already during inference, and the empirical improvements are typically meagre. A similar approach has been proposed also for MCMC [6], with similar drawbacks.
We propose an alternative, novel approach for making optimal decisions based on a probabilistic model characterized by an approximate posterior. Instead of modifying the inference procedure as in losscalibrated inference, we modify the decision making phase by replacing the analytic Bayes optimal decisions with a parametric function tuned to minimize the risk for the approximate posterior at hand. The proposed approach is conceptually simple, can be applied on top of any approximate posterior irrespective of the manner it was obtained, does not require recomputing the posterior approximation for new losses, and it is demonstrated empirically to clearly outperform the losscalibrated inference approach [5] in some scenarios. When applied on the true posterior the procedure reverts back to the standard Bayes optimal decision.
Our key idea is to couple the Bayesian model with a decisionmaking module, in practice a neural network, which takes as input a characterization of the predictive distribution and outputs the decision . It mimics optimal Bayesian decision rules by using the same input sufficient for optimal decisions, but is optimized to minimize a particular risk. The principle is illustrated in Figure 1.
Training a flexible mapping from the predictive distribution to the decisions by direct optimization of the empirical risk can, naturally, overfit to the specific data collection. Furthermore, a flexible mapping may, under some conditions, overrule the underlying model, by making predictions not supported by the assumed model. We overcome these issues by presenting a generalized Bayesian inference strategy, building on [7], over the decisionmaking modules. We provide a justified prior distribution that controls the decisionmaking module, by regularizing it towards the standard Bayes optimal decision, and demonstrate automatic prior specification using a light bootstrap procedure [8].
Our result is a plugin tool that is applicable for every Bayesian model designed for predictive tasks. One can use any modeling framework or inference strategy, as long as we have access to samples from the predictive distribution. We demonstrate the approach for standard Bayesian models ranging from matrix factorization to sparse regression, and show improved decisions for poor approximations.
2 Background
To facilitate understanding the rest of the paper, we briefly summarize Bayesian decision theory, the concept of approximate posterior inference, and the standard approach for incorporating approximate posteriors into Bayesian decision making.
2.1 Bayesian Decision Theory
Loss  Expression  Decision rule 

squared  
absolute  
imbalanced absolute  percentile  
tilted  percentile 
Bayesian decision theory [1]
defines a rigorous framework for decisionmaking under uncertainty. Given a loss function
and a posterior distributionof a parametric model conditioned on the dataset
, the Bayes optimal decision minimizes expected posterior lossIn supervised settings, decisions are made for pairs , and the risk is an expectation over unknown data generating distribution . For setups where we do not explicitly model but do assume a model for , we can define conditional risk for using
(1) 
where . The optimal decisions for individual data points are then , denoted as optimal to indicate they are optimal for the true predictive distribution. Similarly, we call (1) the risk or simply risk. Table 1 presents optimal decisions for several example losses.
In practice the risk needs to be approximated by empirical risk , based on some data collection, where the distribution of and
is purely empirical. More detailed discussion of Bayesian decision theory for supervised learning can be found in
[3].2.2 Approximate Inference
Typically, the posterior distribution is analytically intractable, and we need to resort to approximate inference techniques to replace it with a computationally tractable proxy . A wide array of inference tools are available with different tradeoffs between computational time and efficiency. Monte Carlo Markov Chain (MCMC) methods approximate with samples from the posterior, with stateoftheart algorithms using gradientbased information to improve convergence and speed [9]. Distributional approximations, in turn, assume a parametric approximation family and optimize for its parameters to minimize some discrepancy measure between the approximation and the true posterior. This can be done in multitude of ways, the most common strategies being variational approximation [2] and expectation propagation [10, 11]. Distributional approximations often have computational advantage over MCMC, but are biased or inaccurate especially when using poor approximation family [12]
. For example, variational approximation minimizes a KullbackLeibler divergence between the approximation and the true posterior, and hence underestimates posterior variation
[2].Probabilistic programming tools, such as Stan [13] and Edward [14] have recently made practical Bayesian modeling easier, by coupling automatic inference engines based on Hamiltonian Monte Carlo [9] or gradientbased variational approximation [15, 16] with easy model specification language. The typical mode of operation today is to specify the model in such a language, and the practitioner may not even care about the approximation strategy being used. This has made Bayesian modeling possible for wider audience, which implies more research is needed on how the models are being used in downstream applications.
2.3 Decisionmaking for Approximate Inference
We often do not have access to the true posterior to evaluate the risk in Eq. 1. We instead use the approximation as a proxy for the posterior. Following the nomenclature of Lacoste–Julien et al. [3], we define the posterior predictive distribution and the risk
The optimal decisions minimize the risk. Despite the name, the optimal decisions are not optimal with respect to the risk. Instead, we typically have . Section 3 describes how the discrepancy between and can be mitigated, when only having access to the approximation .
3 Predictive Decision Theory
Our goal is to make good – possibly even optimal – decisions, as measured by the risk under a probabilistic model , when only having access to an approximation of the true posterior . We do this by introducing a parametric decisionmaking module , a function that takes as input a characterization of the predictive distribution under the approximate posterior and produces as an output the decision . This module can be applied on top of arbitrary probabilistic models.
In the following, we describe the necessary technical elements required for implementing the proposed Predictive Decision Theory (PDT). We first formulate the problem as generalized Bayesian inference over the decisionmakers in Section 3.1, and then explain how practical decisionmaking modules can be implemented in Section 3.2.
3.1 Decision Belief Distributions
To provide justified uncertainty quantification for the decisions, we build on the generalized Bayesian inference framework by Bissiri et al. [7]. They present a coherent procedure for updating beliefs for scenarios where the parameter of interest is connected to observations via arbitrary loss functions, instead of likelihood functions. Their key result is that (using generic notation to avoid confusion with the symbols used in main derivations of our results)
is a valid posterior distribution for any loss that depends on the parameters and the observation , and any constant allowing for calibration of relative impact of the prior and the loss. In other words, we can use exponentiated negative losses in place of likelihoods and still characterize the uncertainty on in a coherent manner.
The framework provides direct basis for decisionproblems involving the model parameters itself. We use it for predictive problems instead, for problems characterized by loss defined in terms of predicted quantities. The decision is made based on the predictive distribution that is obtained via marginalization of model parameters . One could think of plugging in the predictive distribution along with the procedure for obtaining in place of loss, but that would make model the impossible to train because of the marginalization required over . Instead, we consider decision belief distributions
and tie decisions for different with parameteric decisionmaking module (DM) . DM makes decisions on the observed data , is parameterized by , and takes as input the predictive distribution (or an approximation ) from already trained model. This allows us to define the Bayesian update rule for belief distribution of a decisionmaking module
(2) 
where and is some prior defined over the parameters of the DM. The constant allows tuning the compromise, but importantly all values result in valid posterior.
3.2 Decision Maker
We replace the standard decisionmaking procedure explained in Section 2.3 with parametric function (decision maker) , for which posterior inference is characterized by Eq. (2). For a collection of data instances the logposterior is
(3) 
where is a normalization constant that has no effect on inference and . Empirical risk
appears naturally as a part of the objective. Suitable priors, however, help us to address the issue of generalization, to make sure the decisions are good for the true risk instead of just the empirical one. The formulation defines the basis for full posterior inference, but we focus on finding a single good decisionmaker and hence resort to MAP estimation of Eq. (
3) that is less computationally expensive.Regularization with optimal Decisions
We start by specifying the prior used for controlling the flexibility of the DM, by building on the assumption that is reasonably good approximation of . When they are identical the optimal decision corresponds exactly to the optimal decision , and for small approximation errors we would still expect to find the optimal decision in the neighborhood of the theoretical optimum.
Building on the above principle we define an implicit prior. Instead of placing a prior on the parameters directly, we place a prior on the decisions , which induces a prior on . For the decisions we assume the prior
(4) 
where is optimal decision for
th data point and the variance is set to unit value as it can be subsumed to the
parameter in Eq. (3). Small allows DM to deviate arbitrary far from , whereas large forces DM to emulate optimal decisions. A good value can be selected, e.g., by crossvalidation.A more flexible prior is obtained by allowing pointspecific variation with
This prior requires an automatic procedure for setting and , which we do based on the expected variation caused by the underlying distribution . The decisions are made conditional on the covariates and the total risk averages over the data generating distribution , as briefly explained in Section 2.1. The natural variation in samples produced by the generating distribution provides a reasonable basis for estimating the allowed variation in decisions as well, and to estimate this we apply a bootstrap procedure [8].
With bootstrapping, we draw several new datasets (), using in our empirical experiments. For each dataset, we obtain the posterior approximation and the optimal decisions , and estimate the parameters for each data point using , . Finally, under independence assumption on , we can define the prior for as
(5) 
where for a simpler case and for bootstrapbased prior . The overall strength of the prior is in both cases controlled by .
Representation of the Predictive Distribution
Bayesian decision theory states that the predictive distribution is necessary and sufficient information for making optimal decisions. We retain this assumption when switching to parametric decisionmakers, by using the predictive distribution as the sole input for the decisionmaker. However, it is rarely available as analytic expression.
A practical and modelindependent representation is obtained by sampling a collection of data points from the model under the approximate posterior . Such a collection can be summarized with suitable finite statistic that can then be passed as input for the decisionmaker; we use evenlyspaced empirical quantiles, but for example a histogram would work as well. This representation only requires the ability to sample from the predictive distribution, covering essentially all probabilistic models of interest. It could even generalize for simulatorbased models that lack closedform expression for likelihood but still enable sampling from the model [17].
The parameters and influence the richness and accuracy of the input representation, but as illustrated empirically in Section 5.1 the procedure is not very sensitive to the choices. Since computing the representation is cheap we can safely use large , and small
is enough because the decisionmakers can implicitly interpolate between the available quantiles if needing more granularity.
Parametric Decision Makers
Most Bayesian optimal decisions are simple summary statistics of the predictive distributions (Table 1), and simplified illustrations of the procedure like Figure 1 intuitively suggest that it may be enough to modify the specific threshold, e.g., by lowering or increasing the quantile for tilted loss to account for the discrepancy between the predictive distributions. Such a decisionmaker is easy to implement and has limited room for overfitting, but we will later show empirically (Section 5.1) that it is not sufficient for notably improving the predictions. Instead, we need more flexible functions.
A natural choice for a flexible modelindependent decision maker is to use a neural network, interpreted as arbitrary mapping from the predictive distribution to the decision . The decisionmaker is parameterized by
, which denotes collectively the set of all network weights. Such modules are easy to implement in modern machine learning platforms, allowing for flexible choice of network architectures. In our experiments, we use a simple feedforward network with 3 hidden layers (with 20, 20 and 10 nodes) with ReLU activation and Adam optimizer with learning rate
, but note that the specific network details are not important. Instead, we simply need a network that is able to relatively flexibly convert a fixeddimensional quantile representation of the predictive distribution into a scalar decision.) loss, showing how the regularization completely removes overfitting. (Right:) Effects of number of samples and predictive distribution representation (number of quantiles) on test data performance for tilted (
) loss.4 Related Work
Our work lies in the intersection of Bayesian modeling and learning theory. The general principle of minimizing the regularized risk for making decisions that generalize for the datagenerating distribution is the cornerstone of standard learning theory (see, e.g., [18]). The current work does not utilize or require results in recent learning theory literature, but we note that there may be interesting connections useful for future work. Our method also loosely relates to work on quantile regression [19] and the training of decisionmakers to generate prediction intervals [20]. However, we build on existing Bayesian models and merely use a neural network to correct for approximation error, whereas they use blackbox methods for the predictive distribution itself.
The most closely related work is on direct losscalibration of posterior approximations [3, 6]. We propose an alternative for that approach and will empirically compare the proposed solution against the latest losscalibration method [5], and hence describe here their approach briefly.
The core idea of losscalibration is to replace the original learning objective, in case of variational approximation, a lower bound to the marginal likelihood , with an augmented objective that involves a separate term accounting for the loss
The function is the loss transformed to utility [1]. This augmented objective is maximized, typically with an alternating algorithm, with respect to both the parameters of the approximation and the decisions . The decisions influence the approximation and the approximation influence the decisions, and hence the optimization is tightly coupled. This implies that the procedure needs to be carried out separately for all losses of interest and the algorithmic details need to be derived for each approximation strategy separately. Our approach is applied on top of existing approximation and hence does not suffer from these limitations: Once the approximation is available, we can easily optimize for any loss without needing to reiterate the inference.
5 Experiments
To illustrate and validate the procedure we conduct a series of experiments, using variational approximation as . We first compare the method against the alternative of calibrating the posterior inference to account for the loss for a matrix factorization model, and then demonstrate improved decisions for a sparse regression model and a multilevel model for cases with approximations. We also study practical decisions regarding the input representation and regularization.
5.1 Predictive Performance
We compare our approach to Losscalibrated Variational Inference (LCVI) by Kuśmierczyk et al. [5], the closest alternative to our method, using the same simplified probabilistic matrix factorization model
they used for demonstrating losscalibration improves predictions over standard variational inference on a subset of last.fm data [21]. We use their code^{1}^{1}1https://github.com/tkusmierczyk/lcvi to precisely reproduce the experiment, matching all of their modeling and approximation choices (, , meanfield Gaussian approximation, and logtransformation for the count data).
Figure 2 (left) compares relative reduction of empirical risk (, where is the risk of optimal decision on standard approximation) on the test data for the same four losses they used. For tilted losses the proposed approach using neural network as decisionmaker dramatically outperforms LCVI, reaching risk reduction depending on the parameter of the loss. This demonstrates the proposed approach has clear practical value over the alternative of finetuning the approximation itself. The simpler decisionmaker that only optimizes for the quantile is unable to improve the decisions, demonstrating that it is preferable to learn a more flexible mapping.
For squared loss even the neural decisionmaker performs poorly, due to overfitting. Figure 2 (middle) illustrates how regularizing the decisionmaker towards the optimal decisions using the prior (5) elegantly removes the overfitting. The method never achieves risk reduction, but instead converges to the standard optimal estimator. Consequently, there are scenarios for which the computationally more expensive LCVI is able to improve the predictions, whereas the proposed method is not.
5.2 Posterior Representation
In practical computation the posterior predictive distribution is presented using empirical quantiles of samples drawn from the predictive distribution (Section 3.2). The choice of the sample size and the number of quantiles naturally influences the representation accuracy. We study the effect of these parameters on the same data and model as above.
Figure 2 (right) compares test data empirical risk averaged of 10 different random initializations for various representations. The main observations are that it is beneficial to use large sample size, and since the computational cost of drawing the samples is negligible compared to other stages of the procedure (fitting the approximation, training the decisionmaker) we use in other experiments. The risk improves also when increasing the representation size, and for the other experiments we use quantiles. However, it is worth noting that already very few quantiles are sufficient for improving the risk compared to optimal decisions (risk ).
5.3 Handling VI Failures
The main use for the method is in correcting for inaccurate posterior approximations, and hence we conduct two experiments on two separate regression models to validate this. At the same time, we illustrate the bootstrapbased regularization strategy.
Multilevel Models with Poor Approximation
Since variational approximations are trained with iterative algorithms, we can construct a controlled experiment by terminating the inference algorithm early and varying the termination point. In general, approximations trained for shorter time are further away from the final approximation and the true posterior.
We use the radon data and the multilevel model
for modeling it [22]. The model is implemented using the publicly available Stan code^{2}^{2}2http://mcstan.org/users/documentation/casestudies/radon.html, using the Minnesota subset of the data and also otherwise conforming to their details. The inference is carried out using automatic differentiation variational inference [16], and we split the data randomly into equally sized training and test set.
Figure 3 (top) compares performance of different decisionmaking strategies (vertical axis) w.r.t quality of the approximation fit controlled by number of training iterations (horizontal axis). Hamiltonian Monte Carlo [9] provides here sufficiently accurate posterior to act as the baseline, and we see that variational approximation converges sufficiently close to the true posterior around iterations, reaching the risk. The proposed method without regularization achieves relatively good risk already for very poor posterior approximations, but never converges to the optimal decision due to overfitting. The variant regularized with the bootstrapbased prior in Eq. (5) achieves the best of both worlds, reducing the risk for poor approximations but converging to the optimal decisions when the approximation becomes good. Figure 3 (bottom) plots the risk at iterations (still incorrect approximation) as a function of the regularization parameter , showing that the method is robust to the choice of ; all values within the range improve compared to the optimal baseline, and for the risk is virtually identical.
Sparse Models with Failure of Convergence
Variational approximations have trouble with multimodal or otherwise complicated posteriors. Yao et al. [12] use models with horseshoe priors [23] for demonstrating this, and we use the same model family to demonstrate how we can still recover good decisions even when this happens. We use the sparse regression model
following closely a classification model proposed by Piironen and Vehtari [24] with public Stan code^{3}^{3}3https://github.com/yaoyl/EvaluatingVariationalInference/blob/master/R_code/glm_bernoulli_rhs.stan, but change to Gaussian likelihood suitable for regression problems. We apply the model on the corn data [25], that we randomly split into equally sized training and test subsets.
For this model the quality of the variational approximation is sensitive to random initialization and the stochastic variation during the optimization, so that occasionally the posterior is reasonable whereas for some runs it converges to a very bad solution. Figure 4 illustrates the risk for 10 independent runs with different random seeds, showing that for 7 runs all decisionmaking strategies are sufficient. However, for 3 of the runs the optimal decision fails miserably, yet the parametric decisionmaker is able to recover essentially optimal decisions. In other words, we show that the proposed strategy is able to correct for the mistake in the posterior.
5.4 Model Faithfulness
The main goal of our work is to improve the decisions made given a specific probabilistic model, which means the decisions should remain faithful to the underlying model: The decisionmaker should correct for mistakes in the posterior approximation but not in model missspecification, since the users needs to be able to rely on interperations of the model. Next we show the regularization strategy proposed in Section 3.2 achieves this.
Figure 5 shows the predictive distribution of intentionally incorrect model, a linear model for highly nonlinear data. Even though the decisionmaker only takes as input the predictive distribution (red shaded area), a flexible enough DM learns to ignore the model and returns the upper quantile of the data distribution that minimizes the risk. This can be done because there exists a mapping from the predictive quantiles back to the covariates, and hence the neural network can implicitly use the covariates themselves for predictions. In other words, it can learn to directly map the nonlinear data, without conforming to the linear model.
Regularizing with (4) smoothly transitions the decisions towards the optimal decision, here a linear function of the covariates. Sufficiently strong regularization prevents the flexible DM module from overriding the model.
6 Conclusion
Even though Bayesian decision theory [1] elegantly separates inference from decisions, this separation is constantly being misused by applying decision rules derived for the true posterior to approximations. This may dramatically affect the decisions, typically in a manner that is difficult to notice since there is no access to the true risk.
Some effort has been made to remedy this, based on directly modifying the inference process to account for the eventual decision loss [3, 4, 5, 6], but these works are computationally costly and have demonstrated only marginal improvements. We proposed the alternative of retaining the original approximation and changing the decisionmaking process instead, by replacing the analytic decision with parametric decisionmaking module. This has the significant advantage of being agnostic to the inference strategy being used, making the approach applicable to, e.g., improving predictive accuracy of models for which inference has already been carried out. To demonstrate the generality of our method, we ran several experiments on top of posterior approximations computed by offtheshelf probabilistic programming system Stan [13], demonstrating improved decisions for cases where the posterior approximation is inaccurate.
While the notion of directly optimizing for empirical risk may feel somewhat unorthodox in a strict Bayesian sense, we presented several technical arguments supporting the validity of the approach. By building on generalized Bayesian inference [7]
we can reformulate reasoning over the decisionmakers as justified Bayesian inference, and we provide a prior distribution that allows controlling for how faithful we remain to the underlying model, with automatic procedure for selecting parameters that does not allow significant deviation. While some theoretical questions of interest still remain, we have demonstrated that the quantitative value of predictive Bayesian models can be improved by incorporating flexible decisionmakers borrowed from the deep learning literature without compromising the interpretability of Bayesian modeling.
Acknowledgements
The work was supported by Academy of Finland, under grant 1313125, as well as the Finnish Center for Artificial Intelligence (FCAI), a Flagship of the Academy of Finland. We also thank the Finnish Grid and Cloud Infrastructure (urn:nbn:fi:researchinfras2016072533) for providing computational resources.
References
 [1] James O Berger. Statistical Decision Theory and Bayesian Analysis; 2nd edition. Springer Series in Statistics. Springer, New York, 1985.
 [2] David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational Inference: A Review for Statisticians. Journal of the American Statistical Association, 112(518), 2017.
 [3] Simon LacosteJulien, Ferenc Huszár, and Zoubin Ghahramani. Approximate inference for the losscalibrated Bayesian. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, 2011.
 [4] Adam D Cobb, Stephen J Roberts, and Yarin Gal. LossCalibrated Approximate Inference in Bayesian Neural Networks. In Theory of Deep Learning workshop, ICML, 2018.
 [5] Tomasz Kuśmierczyk, Joseph Sakaya, and Arto Klami. Variational Bayesian Decisionmaking for Continuous Utilities. arXiv preprint arXiv:1902.00792, 2019.
 [6] Ehsan Abbasnejad, Justin Domke, and Scott Sanner. Losscalibrated Monte Carlo Action Selection. In Proceedings of the TwentyNinth AAAI Conference on Artificial Intelligence, 2015.
 [7] Pier Giovanni Bissiri, Chris C Holmes, and Stephen G Walker. A general framework for updating belief distributions. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78(5):1103–1130, 2016.
 [8] Bradley Efron and Robert J Tibshirani. An introduction to the bootstrap. CRC press, 1994.
 [9] Matthew D. Hoffman and Andrew Gelman. The nouturn sampler: Adaptively setting path lengths in hamiltonian monte carlo. Journal of Machine Learning Research, 15:1593–1623, 2014.
 [10] Andrew Gelman, Aki Vehtari, Pasi Jylänki, Tuomas Sivula, Dustin Tran, Swupnil Sahai, Paul Blomstedt, John P Cunningham, David Schiminovich, and Christian Robert. Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data. arXiv preprint arXiv:1412.4869, 2017.
 [11] Thomas P. Minka. Expectation Propagation for Approximate Bayesian Inference. In Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence, 2001.
 [12] Yuling Yao, Aki Vehtari, Daniel Simpson, and Andrew Gelman. Yes, but did it work?: Evaluating variational inference. In Proceedings of the 35th International Conference on Machine Learning, 2018.
 [13] Bob Carpenter, Andrew Gelman, Matthew D Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell. Stan: A Probabilistic Programming Language. Journal of Statistical Software, 76(1), 2017.
 [14] Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, and David M. Blei. Edward: A Library for Probabilistic Modeling, Inference, and Criticism. arXiv:1610.09787, 2016.
 [15] Michalis Titsias and Miguel LázaroGredilla. Doubly Stochastic Variational Bayes for nonConjugate Inference. In Proceedings of 31st International Conference on Machine Learning, 2014.
 [16] Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and David M Blei. Automatic Differentiation Variational Inference. The Journal of Machine Learning Research, 18(1), 2017.
 [17] Michael U. Gutmann and Jukka Corander. Bayesian optimization for likelihoodfree inference of simulatorbased statistical models. Journal of Machine Learning Research, 17(125):1–47, 2016.
 [18] Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data mining, inference and prediction. Springer, 2nd edition edition, 2009.
 [19] Roger Koenker and Kevin F. Hallock. Quantile regression. Journal of Economic Perspectives, 15(4):143–156, December 2001.
 [20] Tim Pearce, Alexandra Brintrup, Mohamed Zaki, and Andy Neely. Highquality prediction intervals for deep learning: A distributionfree, ensembled approach. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4075–4084, 2018.
 [21] Thierry BertinMahieux, Daniel P.W. Ellis, Brian Whitman, and Paul Lamere. The Million Song Dataset. In Proceedings of the 12th International Conference on Music Information Retrieval, 2011.
 [22] Andrew Gelman and Jennifer Hill. Data analysis using regression and multilevel/hierarchical models. Cambridge university press, 2006.
 [23] Carlos M. Carvalho, Nicholas G. Polson, and James G. Scott. The horseshoe estimator for sparse signals. Biometrika, 97:465–480, 2010.

[24]
Juho Piironen and Aki Vehtari.
On the hyperprior choice for the global shrinkage parameter in the horseshoe prior.
In Artificial Intelligence and Statistics, pages 905–913, 2017. 
[25]
Tao Chen and Elaine Martin.
Bayesian linear regression and variable selection for spectroscopic calibration.
Analytica chimica acta, 631(1):13–21, 2009.
Comments
There are no comments yet.