Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design

by   Yuji Takubo, et al.

This paper develops a hierarchical reinforcement learning architecture for multi-mission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportation scheduling. This problem involves a high-dimensional design space and is challenging especially with uncertainty present. To tackle this challenge, the developed framework has a hierarchical structure with reinforcement learning (RL) and network-based mixed-integer linear programming (MILP), where the former optimizes campaign-level decisions (e.g., design of the vehicle used throughout the campaign, destination demand assigned to each mission in the campaign), whereas the latter optimizes the detailed mission-level decisions (e.g., when to launch what from where to where). The framework is applied to a set of human lunar exploration campaign scenarios with uncertain in-situ resource utilization (ISRU) performance as a case study. The main value of this work is its integration of the rapidly growing RL research and the existing MILP-based space logistics methods through a hierarchical framework to handle the otherwise intractable complexity of space mission design under uncertainty. We expect this unique framework to be a critical steppingstone for the emerging research direction of artificial intelligence for space mission design.



There are no comments yet.


page 1

page 2

page 3

page 4


Model-based Reinforcement Learning: A Survey

Sequential decision making, commonly formalized as Markov Decision Proce...

Distributional Reinforcement Learning for Scheduling of (Bio)chemical Production Processes

Reinforcement Learning (RL) has recently received significant attention ...

Amplifying the Imitation Effect for Reinforcement Learning of UCAV's Mission Execution

This paper proposes a new reinforcement learning (RL) algorithm that enh...

Innovations in the field of on-board scheduling technologies

Space missions are characterized by long distances, difficult or unavail...

Behavior Planning at Urban Intersections through Hierarchical Reinforcement Learning

For autonomous vehicles, effective behavior planning is crucial to ensur...

Online Multimodal Transportation Planning using Deep Reinforcement Learning

In this paper we propose a Deep Reinforcement Learning approach to solve...

MISSION AWARE: Evidence-Based, Mission-Centric Cybersecurity Analysis

Currently, perimeter-based approaches are the mainstay of cybersecurity....
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.