Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient

10/18/2021
by   Shanchao Yang, et al.
0

Improving the resilience of a network protects the system from natural disasters and malicious attacks. This is typically achieved by introducing new edges, which however may reach beyond the maximum number of connections a node could sustain. Many studies then resort to the degree-preserving operation of rewiring, which swaps existing edges AC, BD to new edges AB, CD. A significant line of studies focuses on this technique for theoretical and practical results while leaving three limitations: network utility loss, local optimality, and transductivity. In this paper, we propose ResiNet, a reinforcement learning (RL)-based framework to discover resilient network topologies against various disasters and attacks. ResiNet is objective agnostic which allows the utility to be balanced by incorporating it into the objective function. The local optimality, typically seen in greedy algorithms, is addressed by casting the cumulative resilience gain into a sequential decision process of step-wise rewiring. The transductivity, which refers to the necessity to run a computationally intensive optimization for each input graph, is lifted by our variant of RL with auto-regressive permutation-invariant variable action space. ResiNet is armed by our technical innovation, Filtration enhanced GNN (FireGNN), which distinguishes graphs with minor differences. It is thus possible for ResiNet to capture local structure changes and adapt its decision among consecutive graphs, which is known to be infeasible for GNN. Extensive experiments demonstrate that with a small number of rewiring operations, ResiNet achieves a near-optimal resilience gain on multiple graphs while balancing the utility, with a large margin compared to existing approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/26/2020

Graph neural network based approximation of Node Resiliency in complex networks

The emphasis on optimal operations and efficiency has led to increased c...
research
04/24/2023

Policy Resilience to Environment Poisoning Attacks on Reinforcement Learning

This paper investigates policy resilience to training-environment poison...
research
10/03/2020

Policy Gradient with Expected Quadratic Utility Maximization: A New Mean-Variance Approach in Reinforcement Learning

In real-world decision-making problems, risk management is critical. Amo...
research
03/17/2021

Near Optimal Policy Optimization via REPS

Since its introduction a decade ago, relative entropy policy search (REP...
research
06/06/2019

Classical Policy Gradient: Preserving Bellman's Principle of Optimality

We propose a new objective function for finite-horizon episodic Markov d...
research
05/27/2022

Learning to Solve Combinatorial Graph Partitioning Problems via Efficient Exploration

From logistics to the natural sciences, combinatorial optimisation on gr...
research
09/25/2018

Resilient Computing with Reinforcement Learning on a Dynamical System: Case Study in Sorting

Robots and autonomous agents often complete goal-based tasks with limite...

Please sign up or login with your details

Forgot password? Click here to reset