Ian Dewancker

is this you? claim profile


  • A Stratified Analysis of Bayesian Optimization Methods

    Empirical analysis serves as an important complement to theoretical analysis for studying practical Bayesian optimization. Often empirical insights expose strengths and weaknesses inaccessible to theoretical analysis. We define two metrics for comparing the performance of Bayesian optimization methods and propose a ranking mechanism for summarizing performance within various genres or strata of test functions. These test functions serve to mimic the complexity of hyperparameter optimization problems, the most prominent application of Bayesian optimization, but with a closed form which allows for rapid evaluation and more predictable behavior. This offers a flexible and efficient way to investigate functions with specific properties of interest, such as oscillatory behavior or an optimum on the domain boundary.

    03/31/2016 ∙ by Ian Dewancker, et al. ∙ 0 share

    read it

  • Active Preference Learning for Personalized Portfolio Construction

    In financial asset management, choosing a portfolio requires balancing returns, risk, exposure, liquidity, volatility and other factors. These concerns are difficult to compare explicitly, with many asset managers using an intuitive or implicit sense of their interaction. We propose a mechanism for learning someone's sense of distinctness between portfolios with the goal of being able to identify portfolios which are predicted to perform well but are distinct from the perspective of the user. This identification occurs, e.g., in the context of Bayesian optimization of a backtested performance metric. Numerical experiments are presented which show the impact of personal beliefs in informing the development of a diverse and high-performing portfolio.

    08/24/2017 ∙ by Kevin Tee, et al. ∙ 0 share

    read it

  • Sequential Preference-Based Optimization

    Many real-world engineering problems rely on human preferences to guide their design and optimization. We present PrefOpt, an open source package to simplify sequential optimization tasks that incorporate human preference feedback. Our approach extends an existing latent variable model for binary preferences to allow for observations of equivalent preference from users.

    01/09/2018 ∙ by Ian Dewancker, et al. ∙ 0 share

    read it

  • Sampling Humans for Optimizing Preferences in Coloring Artwork

    Many circumstances of practical importance have performance or success metrics which exist implicitly---in the eye of the beholder, so to speak. Tuning aspects of such problems requires working without defined metrics and only considering pairwise comparisons or rankings. In this paper, we review an existing Bayesian optimization strategy for determining most-preferred outcomes, and identify an adaptation to allow it to handle ties. We then discuss some of the issues we have encountered when humans use this optimization strategy to optimize coloring a piece of abstract artwork. We hope that, by participating in this workshop, we can learn how other researchers encounter difficulties unique to working with humans in the loop.

    06/10/2019 ∙ by Michael McCourt, et al. ∙ 0 share

    read it