DeepAI AI Chat
Log In Sign Up

Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design

by   Yuji Takubo, et al.

This paper develops a hierarchical reinforcement learning architecture for multi-mission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportation scheduling. This problem involves a high-dimensional design space and is challenging especially with uncertainty present. To tackle this challenge, the developed framework has a hierarchical structure with reinforcement learning (RL) and network-based mixed-integer linear programming (MILP), where the former optimizes campaign-level decisions (e.g., design of the vehicle used throughout the campaign, destination demand assigned to each mission in the campaign), whereas the latter optimizes the detailed mission-level decisions (e.g., when to launch what from where to where). The framework is applied to a set of human lunar exploration campaign scenarios with uncertain in-situ resource utilization (ISRU) performance as a case study. The main value of this work is its integration of the rapidly growing RL research and the existing MILP-based space logistics methods through a hierarchical framework to handle the otherwise intractable complexity of space mission design under uncertainty. We expect this unique framework to be a critical steppingstone for the emerging research direction of artificial intelligence for space mission design.


page 1

page 2

page 3

page 4


Bridging adaptive management and reinforcement learning for more robust decisions

From out-competing grandmasters in chess to informing high-stakes health...

A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications

Transportation is the backbone of the economy and urban development. Imp...

Model-based Reinforcement Learning: A Survey

Sequential decision making, commonly formalized as Markov Decision Proce...

Distributional Reinforcement Learning for Scheduling of (Bio)chemical Production Processes

Reinforcement Learning (RL) has recently received significant attention ...

Amplifying the Imitation Effect for Reinforcement Learning of UCAV's Mission Execution

This paper proposes a new reinforcement learning (RL) algorithm that enh...

Advanced Scaling Methods for VNF deployment with Reinforcement Learning

Network function virtualization (NFV) and software-defined network (SDN)...

Innovations in the field of on-board scheduling technologies

Space missions are characterized by long distances, difficult or unavail...