On Sequential Bayesian Optimization with Pairwise Comparison

by   Tanya Ignatenko, et al.

In this work, we study the problem of user preference learning on the example of parameter setting for a hearing aid (HA). We propose to use an agent that interacts with a HA user, in order to collect the most informative data, and learns user preferences for HA parameter settings, based on these data. We model the HA system as two interacting sub-systems, one representing a user with his/her preferences and another one representing an agent. In this system, the user responses to HA settings, proposed by the agent. In our user model, the responses are driven by a parametric user preference function. The agent comprises the sequential mechanisms for user model inference and HA parameter proposal generation. To infer the user model (preference function), Bayesian approximate inference is used in the agent. Here we propose the normalized weighted Kullback-Leibler (KL) divergence between true and agent-assigned predictive user response distributions as a metric to assess the quality of learned preferences. Moreover, our agent strategy for generating HA parameter proposals is to generate HA settings, responses to which help resolving uncertainty associated with prediction of the user responses the most. The resulting data, consequently, allows for efficient user model learning. The normalized weighted KL-divergence plays an important role here as well, since it characterizes the informativeness of the data to be used for probing the user. The efficiency of our approach is validated by numerical simulations.


Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes

We consider Bayesian optimization of expensive-to-evaluate experiments t...

Bayesian preference elicitation for multiobjective combinatorial optimization

We introduce a new incremental preference elicitation procedure able to ...

Pseudo-Mallows for Efficient Probabilistic Preference Learning

We propose the Pseudo-Mallows distribution over the set of all permutati...

Learning User Preferences for Trajectories from Brain Signals

Robot motions in the presence of humans should not only be feasible and ...

Learnable Strategies for Bilateral Agent Negotiation over Multiple Issues

We present a novel bilateral negotiation model that allows a self-intere...

Construction and Adaptability Analysis of User's Preference Models Based on Check-in Data in LBSN

With the widespread use of mobile phones, users can share their location...

POLAR: Preference Optimization and Learning Algorithms for Robotics

Parameter tuning for robotic systems is a time-consuming and challenging...