Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design

03/16/2021
by   Yuji Takubo, et al.
0

This paper develops a hierarchical reinforcement learning architecture for multi-mission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportation scheduling. This problem involves a high-dimensional design space and is challenging especially with uncertainty present. To tackle this challenge, the developed framework has a hierarchical structure with reinforcement learning (RL) and network-based mixed-integer linear programming (MILP), where the former optimizes campaign-level decisions (e.g., design of the vehicle used throughout the campaign, destination demand assigned to each mission in the campaign), whereas the latter optimizes the detailed mission-level decisions (e.g., when to launch what from where to where). The framework is applied to a set of human lunar exploration campaign scenarios with uncertain in-situ resource utilization (ISRU) performance as a case study. The main value of this work is its integration of the rapidly growing RL research and the existing MILP-based space logistics methods through a hierarchical framework to handle the otherwise intractable complexity of space mission design under uncertainty. We expect this unique framework to be a critical steppingstone for the emerging research direction of artificial intelligence for space mission design.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/30/2020

Model-based Reinforcement Learning: A Survey

Sequential decision making, commonly formalized as Markov Decision Proce...
03/01/2022

Distributional Reinforcement Learning for Scheduling of (Bio)chemical Production Processes

Reinforcement Learning (RL) has recently received significant attention ...
01/17/2019

Amplifying the Imitation Effect for Reinforcement Learning of UCAV's Mission Execution

This paper proposes a new reinforcement learning (RL) algorithm that enh...
05/04/2022

Innovations in the field of on-board scheduling technologies

Space missions are characterized by long distances, difficult or unavail...
11/09/2020

Behavior Planning at Urban Intersections through Hierarchical Reinforcement Learning

For autonomous vehicles, effective behavior planning is crucial to ensur...
05/18/2021

Online Multimodal Transportation Planning using Deep Reinforcement Learning

In this paper we propose a Deep Reinforcement Learning approach to solve...
12/05/2017

MISSION AWARE: Evidence-Based, Mission-Centric Cybersecurity Analysis

Currently, perimeter-based approaches are the mainstay of cybersecurity....
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.