Conditional Importance Sampling for Off-Policy Learning

10/16/2019

∙

The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This framework yields new perspectives and understanding of existing off-policy algorithms, and reveals a broad space of unexplored algorithms. We theoretically analyse this space, and concretely investigate several algorithms that arise from this framework.

READ FULL TEXT

Conditional Importance Sampling for Off-Policy Learning

Importance Sampling Placement in Off-Policy Temporal-Difference Methods

Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling

Value-aware Importance Weighting for Off-policy Reinforcement Learning

Sample Dropout: A Simple yet Effective Variance Reduction Technique in Deep Policy Optimization

Asymptotic optimality of adaptive importance sampling

An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment

Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning

Conditional Importance Sampling for Off-Policy Learning

Related Research

Importance Sampling Placement in Off-Policy Temporal-Difference Methods

Understanding the Curse of Horizon in Off-Policy Evaluation via Conditional Importance Sampling

Value-aware Importance Weighting for Off-policy Reinforcement Learning

Sample Dropout: A Simple yet Effective Variance Reduction Technique in Deep Policy Optimization

Asymptotic optimality of adaptive importance sampling

An Empirical Comparison of Off-policy Prediction Learning Algorithms in the Four Rooms Environment

Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning