A Simple Algorithm for Online Decision Making

by   Rui Chen, et al.

Motivated by recent progress on online linear programming (OLP), we study the online decision making problem (ODMP) as a natural generalization of OLP. In ODMP, there exists a single decision maker who makes a series of decisions spread out over a total of T time stages. At each time stage, the decision maker makes a decision based on information obtained up to that point without seeing into the future. The task of the decision maker is to maximize the accumulated reward while overall meeting some predetermined m-dimensional long-term goal (linking) constraints. ODMP significantly broadens the modeling framework of OLP by allowing more general feasible regions (for local and goal constraints) potentially involving both discreteness and nonlinearity in each local decision making problem. We propose a Fenchel dual-based online algorithm for ODMP. At each time stage, the proposed algorithm requires solving a potentially nonconvex optimization problem over the local feasible set and a convex optimization problem over the goal set. Under the uniform random permutation model, we show that our algorithm achieves O(√(mT)) constraint violation deterministically in meeting the long-term goals, and O(√(mlog m)√(T)) competitive difference in expected reward with respect to the optimal offline decisions. We also extend our results to the grouped random permutation model.


page 1

page 2

page 3

page 4


Online Convex Optimization for Cumulative Constraints

We propose an algorithm for online convex optimization which examines a ...

A Unifying Framework for Online Optimization with Long-Term Constraints

We study online learning problems in which a decision maker has to take ...

Achieving Long-Term Fairness in Sequential Decision Making

In this paper, we propose a framework for achieving long-term fair seque...

Simple and Fast Algorithm for Binary Integer and Online Linear Programming

In this paper, we develop a simple and fast online algorithm for solving...

Optimal design of lottery with cumulative prospect theory

A lottery is a popular form of gambling between a seller and multiple bu...

Note on Thompson sampling for large decision problems

There is increasing interest in using streaming data to inform decision ...

Global-Local Metamodel Assisted Two-Stage Optimization via Simulation

To integrate strategic, tactical and operational decisions, the two-stag...

Please sign up or login with your details

Forgot password? Click here to reset