Log In Sign Up

Subgame solving without common knowledge

by   Brian Hu Zhang, et al.

In imperfect-information games, subgame solving is significantly more challenging than in perfect-information games, but in the last few years, such techniques have been developed. They were the key ingredient to the milestone of superhuman play in no-limit Texas hold'em poker. Current subgame-solving techniques analyze the entire common-knowledge closure of the player's current information set, that is, the smallest set of nodes within which it is common knowledge that the current node lies. However, this set is too large to handle in many games. We introduce an approach that overcomes this obstacle, by instead working with only low-order knowledge. Our approach allows an agent, upon arriving at an infoset, to basically prune any node that is no longer reachable, thereby massively reducing the game tree size relative to the common-knowledge subgame. We prove that, as is, our approach can increase exploitability compared to the blueprint strategy. However, we develop three avenues by which safety can be guaranteed. First, safety is guaranteed if the results of subgame solves are incorporated back into the blueprint. Second, we provide a method where safety is achieved by limiting the infosets at which subgame solving is performed. Third, we prove that our approach, when applied at every infoset reached during play, achieves a weaker notion of equilibrium, which we coin affine equilibrium, and which may be of independent interest. We show that affine equilibria cannot be exploited by any Nash strategy of the opponent, so an opponent who wishes to exploit must open herself to counter-exploitation. Even without the safety-guaranteeing additions, experiments on medium-sized games show that our approach always reduced exploitability even when applied at every infoset, and a depth-limited version of it led to–to our knowledge–the first strong AI for the massive challenge problem dark chess.


page 1

page 2

page 3

page 4


Safe and Nested Subgame Solving for Imperfect-Information Games

In imperfect-information games, the optimal strategy in a subgame may de...

Combining Deep Reinforcement Learning and Search for Imperfect-Information Games

The combination of deep reinforcement learning and search at both traini...

Depth-Limited Solving for Imperfect-Information Games

A fundamental challenge in imperfect-information games is that states do...

DecisionHoldem: Safe Depth-Limited Solving With Diverse Opponents for Imperfect-Information Games

An imperfect-information game is a type of game with asymmetric informat...

Value Functions for Depth-Limited Solving in Zero-Sum Imperfect-Information Games

Depth-limited look-ahead search is an essential tool for agents playing ...

Safe Search for Stackelberg Equilibria in Extensive-Form Games

Stackelberg equilibrium is a solution concept in two-player games where ...

Continual Depth-limited Responses for Computing Counter-strategies in Extensive-form Games

In real-world applications, game-theoretic algorithms often interact wit...