A Game Theoretical Framework for the Evaluation of Unmanned Aircraft Systems Airspace Integration Concepts

by   Negin Musavi, et al.

Predicting the outcomes of integrating Unmanned Aerial Systems (UAS) into the National Aerospace (NAS) is a complex problem which is required to be addressed by simulation studies before allowing the routine access of UAS into the NAS. This thesis focuses on providing 2D and 3D simulation frameworks using a game theoretical methodology to evaluate integration concepts in scenarios where manned and unmanned air vehicles co-exist. The fundamental gap in the literature is that the models of interaction between manned and unmanned vehicles are insufficient: a) they assume that pilot behavior is known a priori and b) they disregard decision making processes. The contribution of this work is to propose a modeling framework, in which, human pilot reactions are modeled using reinforcement learning and a game theoretical concept called level-k reasoning to fill this gap. The level-k reasoning concept is based on the assumption that humans have various levels of decision making. Reinforcement learning is a mathematical learning method that is rooted in human learning. In this work, a classical and an approximate reinforcement learning (Neural Fitted Q Iteration) methods are used to model time-extended decisions of pilots with 2D and 3D maneuvers. An analysis of UAS integration is conducted using example scenarios in the presence of manned aircraft and fully autonomous UAS equipped with sense and avoid algorithms.


A 3D Game Theoretical Framework for the Evaluation of Unmanned Aircraft Systems Airspace Integration Concepts

Predicting the outcomes of integrating Unmanned Aerial Systems (UAS) int...

UAS in the Airspace: A Review on Integration, Simulation, Optimization, and Open Challenges

Air transportation is essential for society, and it is increasing gradua...

A Way to Facilitate Decision Making in a Mixed Group of Manned and Unmanned Aerial Vehicles

A mixed group of manned and unmanned aerial vehicles is considered as a ...

Modeling Cyber-Physical Human Systems via an Interplay Between Reinforcement Learning and Game Theory

Predicting the outcomes of cyber-physical systems with multiple human in...

Equivariant Reinforcement Learning for Quadrotor UAV

This paper presents an equivariant reinforcement learning framework for ...

Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions

Maneuver decision-making is the core of unmanned combat aerial vehicle f...

Modeling Human Driver Interactions Using an Infinite Policy Space Through Gaussian Processes

This paper proposes a method for modeling human driver interactions that...

Please sign up or login with your details

Forgot password? Click here to reset