Quantum Algorithms for Solving Dynamic Programming Problems

by   Pooya Ronagh, et al.

We present quantum algorithms for solving finite-horizon and infinite-horizon dynamic programming problems. The infinite-horizon problems are studied using the framework of Markov decision processes. We prove query complexity lower bounds for classical randomized algorithms for the same tasks and consequently demonstrate a polynomial separation between the query complexity of our quantum algorithms and best-case query complexity of classical randomized algorithms. Up to polylogarithmic factors, our quantum algorithms provide quadratic advantage in terms of the number of states |S|, and the number of actions |A|, in the Markov decision process when the transition kernels are deterministic. This covers all discrete and combinatorial optimization problems solved classically using dynamic programming techniques. In particular, we show that our quantum algorithm solves the travelling salesperson problem in O^*(c^4 √(2^n)) where n is the number of nodes of the underlying graph and c is the maximum edge-weight of it. For stochastic transition kernels the quantum advantage is again quadratic in terms of the numbers of actions but less than quadratic (from |S|^2 to |S|^3/2) in terms of the numbers of states. In all cases, the speed-up achieved is at the expense of appearance of other polynomial factors in the scaling of the algorithm. Finally we prove lower bounds for the query complexity of our quantum algorithms and show that no more-than-quadratic speed-up in either of |S| or |A| can be achieved for solving dynamic programming and Markov decision problems using quantum algorithms.



There are no comments yet.


page 1

page 2

page 3

page 4


Quantum POMDPs

We present quantum observable Markov decision processes (QOMDPs), the qu...

A Note on Quantum Markov Models

The study of Markov models is central to control theory and machine lear...

K-spin Hamiltonian for quantum-resolvable Markov decision processes

The Markov decision process is the mathematical formalization underlying...

An Investigation into Mathematical Programming for Finite Horizon Decentralized POMDPs

Decentralized planning in uncertain environments is a complex task gener...

Quantum Dynamic Programming Algorithm for DAGs. Applications for AND-OR DAG Evaluation and DAG's Diameter Search

In this paper, we present Quantum Dynamic Programming approach for probl...

From Bit-Parallelism to Quantum: Breaking the Quadratic Barrier

Many problems that can be solved in quadratic time have bit-parallel spe...

Blending Dynamic Programming with Monte Carlo Simulation for Bounding the Running Time of Evolutionary Algorithms

With the goal to provide absolute lower bounds for the best possible run...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.