Refined approachability algorithms and application to regret minimization with global costs

09/08/2020
by   Joon Kwon, et al.
0

Blackwell's approachability is a framework where two players, the Decision Maker and the Environment, play a repeated game with vector-valued payoffs. The goal of the Decision Maker is to make the average payoff converge to a given set called the target. When this is indeed possible, simple algorithms which guarantee the convergence are known. This abstract tool was successfully used for the construction of optimal strategies in various repeated games, but also found several applications in online learning. By extending an approach proposed by Abernethy et al. (2011), we construct and analyze a class of Follow the Regularized Leader algorithms (FTRL) for Blackwell's approachability which are able to minimize not only the Euclidean distance to the target set (as it is often the case in the context of Blackwell's approachability) but a wide range of distance-like quantities. This flexibility enables us to apply these algorithms to minimize the exact quantity of interest in various online learning problems. In particular, for regret minimization with global costs, we obtain novel guarantees for general norm cost functions, and for the case of ℓ_p cost functions, we obtain the first regret bounds with explicit dependence in p and the dimension d.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/10/2014

Approachability in unknown games: Online learning meets multi-objective optimization

In the standard setting of approachability there are two players and a t...
research
03/16/2016

Regret Minimization in Repeated Games: A Set-Valued Dynamic Programming Approach

The regret-minimization paradigm has emerged as an effective technique f...
research
02/03/2023

Pseudonorm Approachability and Applications to Regret Minimization

Blackwell's celebrated approachability theory provides a general framewo...
research
01/27/2023

Online Learning in Stackelberg Games with an Omniscient Follower

We study the problem of online learning in a two-player decentralized co...
research
02/12/2013

Competing With Strategies

We study the problem of online learning with a notion of regret defined ...
research
11/14/2010

Online Learning: Beyond Regret

We study online learnability of a wide class of problems, extending the ...
research
03/09/2023

Blackwell's Approachability with Time-Dependent Outcome Functions and Dot Products. Application to the Big Match

Blackwell's approachability is a very general sequential decision framew...

Please sign up or login with your details

Forgot password? Click here to reset