Near-Optimal No-Regret Learning for General Convex Games

06/17/2022
by   Gabriele Farina, et al.
0

A recent line of work has established uncoupled learning dynamics such that, when employed by all players in a game, each player's regret after T repetitions grows polylogarithmically in T, an exponential improvement over the traditional guarantees within the no-regret framework. However, so far these results have only been limited to certain classes of games with structured strategy spaces – such as normal-form and extensive-form games. The question as to whether O(polylog T) regret bounds can be obtained for general convex and compact strategy sets – which occur in many fundamental models in economics and multiagent systems – while retaining efficient strategy updates is an important question. In this paper, we answer this in the positive by establishing the first uncoupled learning algorithm with O(log T) per-player regret in general convex games, that is, games with concave utility functions supported on arbitrary convex and compact strategy sets. Our learning dynamics are based on an instantiation of optimistic follow-the-regularized-leader over an appropriately lifted space using a self-concordant regularizer that is, peculiarly, not a barrier for the feasible region. Further, our learning dynamics are efficiently implementable given access to a proximal oracle for the convex strategy set, leading to O(loglog T) per-iteration complexity; we also give extensions when access to only a linear optimization oracle is assumed. Finally, we adapt our dynamics to guarantee O(√(T)) regret in the adversarial regime. Even in those special cases where prior results apply, our algorithm improves over the state-of-the-art regret bounds either in terms of the dependence on the number of iterations or on the dimension of the strategy sets.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/25/2022

Uncoupled Learning Dynamics with O(log T) Swap Regret in Multiplayer Games

In this paper we establish efficient and uncoupled learning dynamics so ...
08/16/2021

Near-Optimal No-Regret Learning in General Games

We show that Optimistic Hedge – a common variant of multiplicative-weigh...
11/22/2021

No-Regret Dynamics in the Fenchel Game: A Unified Framework for Algorithmic Convex Optimization

We develop an algorithmic framework for solving convex optimization prob...
05/20/2018

Transitions, Losses, and Re-parameterizations: Elements of Prediction Games

This thesis presents some geometric insights into three different types ...
06/13/2020

Follow the Perturbed Leader: Optimism and Fast Parallel Algorithms for Smooth Minimax Games

We consider the problem of online learning and its application to solvin...
06/09/2021

Online Optimization in Games via Control Theory: Connecting Regret, Passivity and Poincaré Recurrence

We present a novel control-theoretic understanding of online optimizatio...