Reduction of Markov Chains using a Value-of-Information-Based Approach

03/21/2019
by   Isaac J. Sledge, et al.
0

In this paper, we propose an approach to obtain reduced-order models of Markov chains. Our approach is composed of two information-theoretic processes. The first is a means of comparing pairs of stationary chains on different state spaces, which is done via the negative Kullback-Leibler divergence defined on a model joint space. Model reduction is achieved by solving a value-of-information criterion with respect to this divergence. Optimizing the criterion leads to a probabilistic partitioning of the states in the high-order Markov chain. A single free parameter that emerges through the optimization process dictates both the partition uncertainty and the number of state groups. We provide a data-driven means of choosing the `optimal' value of this free parameter, which sidesteps needing to a priori know the number of state groups in an arbitrary chain.

READ FULL TEXT

page 5

page 8

page 11

page 14

page 15

page 17

research
07/05/2021

An Information-Theoretic Approach for Automatically Determining the Number of States when Aggregating Markov Chains

A fundamental problem when aggregating Markov chains is the specificatio...
research
04/21/2016

Markov models for ocular fixation locations in the presence and absence of colour

We propose to model the fixation locations of the human eye when observi...
research
04/29/2022

Information-Theoretic Reduction of Markov Chains

We survey information-theoretic approaches to the reduction of Markov ch...
research
06/13/2018

Path-entropy maximized Markov chains for dimensionality reduction

Stochastic kernel based dimensionality reduction methods have become pop...
research
02/05/2018

Guided Policy Exploration for Markov Decision Processes using an Uncertainty-Based Value-of-Information Criterion

Reinforcement learning in environments with many action-state pairs is c...
research
10/28/2017

Partitioning Relational Matrices of Similarities or Dissimilarities using the Value of Information

In this paper, we provide an approach to clustering relational matrices ...
research
02/09/2023

Learning Mixtures of Markov Chains with Quality Guarantees

A large number of modern applications ranging from listening songs onlin...

Please sign up or login with your details

Forgot password? Click here to reset