Learning in Stackelberg Games with Non-myopic Agents

by   Nika Haghtalab, et al.

We study Stackelberg games where a principal repeatedly interacts with a long-lived, non-myopic agent, without knowing the agent's payoff function. Although learning in Stackelberg games is well-understood when the agent is myopic, non-myopic agents pose additional complications. In particular, non-myopic agents may strategically select actions that are inferior in the present to mislead the principal's learning algorithm and obtain better outcomes in the future. We provide a general framework that reduces learning in presence of non-myopic agents to robust bandit optimization in the presence of myopic agents. Through the design and analysis of minimally reactive bandit algorithms, our reduction trades off the statistical efficiency of the principal's learning algorithm against its effectiveness in inducing near-best-responses. We apply this framework to Stackelberg security games (SSGs), pricing with unknown demand curve, strategic classification, and general finite Stackelberg games. In each setting, we characterize the type and impact of misspecifications present in near-best-responses and develop a learning algorithm robust to such misspecifications. Along the way, we improve the query complexity of learning in SSGs with n targets from the state-of-the-art O(n^3) to a near-optimal O(n) by uncovering a fundamental structural property of such games. This result is of independent interest beyond learning with non-myopic agents.


page 1

page 2

page 3

page 4


On the Complexity of Multi-Agent Decision Making: From Learning in Games to Partial Monitoring

A central problem in the theory of multi-agent reinforcement learning (M...

Calibrated Stackelberg Games: Learning Optimal Commitments Against Calibrated Agents

In this paper, we introduce a generalization of the standard Stackelberg...

Corrupted Multidimensional Binary Search: Learning in the Presence of Irrational Agents

Standard game-theoretic formulations for settings like contextual pricin...

Bandit learning in concave N-person games

This paper examines the long-run behavior of learning with bandit feedba...

Optimally Deceiving a Learning Leader in Stackelberg Games

Recent results in the ML community have revealed that learning algorithm...

Strategic Decision-Making in the Presence of Information Asymmetry: Provably Efficient RL with Algorithmic Instruments

We study offline reinforcement learning under a novel model called strat...

Altruism Design in Networked Public Goods Games

Many collective decision-making settings feature a strategic tension bet...

Please sign up or login with your details

Forgot password? Click here to reset