Learning to Optimize via Wasserstein Deep Inverse Optimal Control

05/22/2018
by   Yichen Wang, et al.
0

We study the inverse optimal control problem in social sciences: we aim at learning a user's true cost function from the observed temporal behavior. In contrast to traditional phenomenological works that aim to learn a generative model to fit the behavioral data, we propose a novel variational principle and treat user as a reinforcement learning algorithm, which acts by optimizing his cost function. We first propose a unified KL framework that generalizes existing maximum entropy inverse optimal control methods. We further propose a two-step Wasserstein inverse optimal control framework. In the first step, we compute the optimal measure with a novel mass transport equation. In the second step, we formulate the learning problem as a generative adversarial network. In two real world experiments - recommender systems and social networks, we show that our framework obtains significant performance gains over both existing inverse optimal control methods and point process based generative models.

READ FULL TEXT
research
03/01/2016

Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization

Reinforcement learning can acquire complex behaviors from high-level spe...
research
09/27/2016

Task Specific Adversarial Cost Function

The cost function used to train a generative model should fit the purpos...
research
11/11/2016

A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models

Generative adversarial networks (GANs) are a recently proposed class of ...
research
04/10/2019

Learning Trajectory Prediction with Continuous Inverse Optimal Control via Langevin Sampling of Energy-Based Models

Autonomous driving is a challenging multiagent domain which requires opt...
research
11/18/2019

Inverse Cooperative and Non-Cooperative Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning

Dynamic game theory provides mathematical means for modeling the interac...
research
03/21/2018

Inverse Optimal Control with Incomplete Observations

In this article, we consider the inverse optimal control problem given i...
research
12/13/2017

Inverse Reinforcement Learning for Marketing

Learning customer preferences from an observed behaviour is an important...

Please sign up or login with your details

Forgot password? Click here to reset