A Data-Driven State Aggregation Approach for Dynamic Discrete Choice Models

by   Sinong Geng, et al.

We study dynamic discrete choice models, where a commonly studied problem involves estimating parameters of agent reward functions (also known as "structural" parameters), using agent behavioral data. Maximum likelihood estimation for such models requires dynamic programming, which is limited by the curse of dimensionality. In this work, we present a novel algorithm that provides a data-driven method for selecting and aggregating states, which lowers the computational and sample complexity of estimation. Our method works in two stages. In the first stage, we use a flexible inverse reinforcement learning approach to estimate agent Q-functions. We use these estimated Q-functions, along with a clustering algorithm, to select a subset of states that are the most pivotal for driving changes in Q-functions. In the second stage, with these selected "aggregated" states, we conduct maximum likelihood estimation using a commonly used nested fixed-point algorithm. The proposed two-stage approach mitigates the curse of dimensionality by reducing the problem dimension. Theoretically, we derive finite-sample bounds on the associated estimation error, which also characterize the trade-off of computational complexity, estimation error, and sample complexity. We demonstrate the empirical performance of the algorithm in two classic dynamic discrete choice estimation applications.


page 1

page 2

page 3

page 4


Finite-sample risk bounds for maximum likelihood estimation with arbitrary penalties

The MDL two-part coding index of resolvability provides a finite-sampl...

Constrained maximum likelihood estimation of clusterwise linear regression models with unknown number of components

We consider an equivariant approach imposing data-driven bounds for the ...

Robust estimation in beta regression via maximum Lq-likelihood

Beta regression models are widely used for modeling continuous data limi...

Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism

In this paper, we study offline Reinforcement Learning with Human Feedba...

Process monitoring based on orthogonal locality preserving projection with maximum likelihood estimation

By integrating two powerful methods of density reduction and intrinsic d...

Estimation of Discrete Choice Models: A Machine Learning Approach

In this paper we propose a new method of estimation for discrete choice ...

Passenger Path Choice Estimation Using Smart Card Data: A Latent Class Approach with Panel Effects Across Days

Understanding passengers' path choice behavior in urban rail systems is ...

Please sign up or login with your details

Forgot password? Click here to reset