Nearly Maximally Predictive Features and Their Dimensions

02/27/2017
by   Sarah E. Marzen, et al.
0

Scientific explanation often requires inferring maximally predictive features from a given data set. Unfortunately, the collection of minimal maximally predictive features for most stochastic processes is uncountably infinite. In such cases, one compromises and instead seeks nearly maximally predictive features. Here, we derive upper-bounds on the rates at which the number and the coding cost of nearly maximally predictive features scales with desired predictive power. The rates are determined by the fractal dimensions of a process' mixed-state distribution. These results, in turn, show how widely-used finite-order Markov models can fail as predictors and that mixed-state predictive features offer a substantial improvement.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/29/2020

Shannon Entropy Rate of Hidden Markov Processes

Hidden Markov chains are widely applied statistical models of stochastic...
research
02/21/2021

Divergent Predictive States: The Statistical Complexity Dimension of Stationary, Ergodic Hidden Markov Processes

Even simply-defined, finite-state generators produce stochastic processe...
research
05/07/2022

Accuracy Convergent Field Predictors

Several predictive algorithms are described. Highlighted are variants th...
research
07/26/2023

Using Markov Boundary Approach for Interpretable and Generalizable Feature Selection

Predictive power and generalizability of models depend on the quality of...
research
12/09/2014

Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes

Predictive rate-distortion analysis suffers from the curse of dimensiona...
research
08/01/2017

Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse?

Understanding the generative mechanism of a natural system is a vital co...
research
09/19/2021

Topology, Convergence, and Reconstruction of Predictive States

Predictive equivalence in discrete stochastic processes have been applie...

Please sign up or login with your details

Forgot password? Click here to reset