Koopman Spectrum Nonlinear Regulator and Provably Efficient Online Learning

06/30/2021
by   Motoya Ohnishi, et al.
0

Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, behaviors with sudden accelerations that waste energy and lack predictability. In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics. This induces a broader class of dynamical behaviors that evolve over stable manifolds such as nonlinear oscillators, closed loops, and smooth movements. We demonstrate that some dynamics realizations that are not possible with a cumulative cost are feasible in this paradigm. Moreover, we present a provably efficient online learning algorithm for our problem that enjoys a sub-linear regret bound under some structural assumptions.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 21

page 25

06/19/2018

Online Linear Quadratic Control

We study the problem of controlling linear time-invariant systems with k...
03/25/2020

Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems

We study the problem of adaptive control in partially observable linear ...
06/09/2020

Online Learning in Iterated Prisoner's Dilemma to Mimic Human Behavior

Prisoner's Dilemma mainly treat the choice to cooperate or defect as an ...
03/07/2017

Online Learning to Rank in Stochastic Click Models

Online learning to rank is a core problem in information retrieval and m...
11/14/2019

A Reduction from Reinforcement Learning to No-Regret Online Learning

We present a reduction from reinforcement learning (RL) to no-regret onl...
10/21/2019

Robust Online Learning for Resource Allocation – Beyond Euclidean Projection and Dynamic Fit

Online-learning literature has focused on designing algorithms that ensu...
06/21/2018

Using Nonlinear Normal Modes for Execution of Efficient Cyclic Motions in Soft Robots

With the aim of getting closer to the performance of the animal musclesk...

Code Repositories

KSNR

Koopman Spectrum Nonlinear Regulator and Provably Efficient Online Learning


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.