DeepAI AI Chat
Log In Sign Up

Solving Large-Scale Extensive-Form Network Security Games via Neural Fictitious Self-Play

by   Wanqi Xue, et al.

Securing networked infrastructures is important in the real world. The problem of deploying security resources to protect against an attacker in networked domains can be modeled as Network Security Games (NSGs). Unfortunately, existing approaches, including the deep learning-based approaches, are inefficient to solve large-scale extensive-form NSGs. In this paper, we propose a novel learning paradigm, NSG-NFSP, to solve large-scale extensive-form NSGs based on Neural Fictitious Self-Play (NFSP). Our main contributions include: i) reforming the best response (BR) policy network in NFSP to be a mapping from action-state pair to action-value, to make the calculation of BR possible in NSGs; ii) converting the average policy network of an NFSP agent into a metric-based classifier, helping the agent to assign distributions only on legal actions rather than all actions; iii) enabling NFSP with high-level actions, which can benefit training efficiency and stability in NSGs; and iv) leveraging information contained in graphs of NSGs by learning efficient graph node embeddings. Our algorithm significantly outperforms state-of-the-art algorithms in both scalability and solution quality.


page 1

page 2

page 3

page 4


No-Press Diplomacy from Scratch

Prior AI successes in complex games have largely focused on settings wit...

Evolutionary Approach to Security Games with Signaling

Green Security Games have become a popular way to model scenarios involv...

Temporal Induced Self-Play for Stochastic Bayesian Games

One practical requirement in solving dynamic games is to ensure that the...

A Scalable Graph-Theoretic Distributed Framework for Cooperative Multi-Agent Reinforcement Learning

The main challenge of large-scale cooperative multi-agent reinforcement ...

A Unified Perspective on Deep Equilibrium Finding

Extensive-form games provide a versatile framework for modeling interact...

Distributed Node Covering Optimization for Large Scale Networks and Its Application on Social Advertising

Combinatorial optimizations are usually complex and inefficient, which l...