Quantum Algorithms for Solving Dynamic Programming Problems

06/05/2019
by   Pooya Ronagh, et al.
0

We present quantum algorithms for solving finite-horizon and infinite-horizon dynamic programming problems. The infinite-horizon problems are studied using the framework of Markov decision processes. We prove query complexity lower bounds for classical randomized algorithms for the same tasks and consequently demonstrate a polynomial separation between the query complexity of our quantum algorithms and best-case query complexity of classical randomized algorithms. Up to polylogarithmic factors, our quantum algorithms provide quadratic advantage in terms of the number of states |S|, and the number of actions |A|, in the Markov decision process when the transition kernels are deterministic. This covers all discrete and combinatorial optimization problems solved classically using dynamic programming techniques. In particular, we show that our quantum algorithm solves the travelling salesperson problem in O^*(c^4 √(2^n)) where n is the number of nodes of the underlying graph and c is the maximum edge-weight of it. For stochastic transition kernels the quantum advantage is again quadratic in terms of the numbers of actions but less than quadratic (from |S|^2 to |S|^3/2) in terms of the numbers of states. In all cases, the speed-up achieved is at the expense of appearance of other polynomial factors in the scaling of the algorithm. Finally we prove lower bounds for the query complexity of our quantum algorithms and show that no more-than-quadratic speed-up in either of |S| or |A| can be achieved for solving dynamic programming and Markov decision problems using quantum algorithms.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/11/2014

Quantum POMDPs

We present quantum observable Markov decision processes (QOMDPs), the qu...
research
11/05/2019

A Note on Quantum Markov Models

The study of Markov models is central to control theory and machine lear...
research
04/13/2020

K-spin Hamiltonian for quantum-resolvable Markov decision processes

The Markov decision process is the mathematical formalization underlying...
research
06/07/2023

Dynamic Programming on a Quantum Annealer: Solving the RBC Model

We introduce a novel approach to solving dynamic programming problems, s...
research
01/16/2014

An Investigation into Mathematical Programming for Finite Horizon Decentralized POMDPs

Decentralized planning in uncertain environments is a complex task gener...
research
02/23/2021

Blending Dynamic Programming with Monte Carlo Simulation for Bounding the Running Time of Evolutionary Algorithms

With the goal to provide absolute lower bounds for the best possible run...
research
08/31/2022

Partial Counterfactual Identification for Infinite Horizon Partially Observable Markov Decision Process

This paper investigates the problem of bounding possible output from a c...

Please sign up or login with your details

Forgot password? Click here to reset