Streaming Generalized Cross Entropy
share
We propose a new method to combine adaptive processes with a class of entropy estimators for the case of streams of data. Starting from a first estimation obtained from a batch of initial data, at each step the parameters of the model are estimated combining the prior knowledge and the new observation (or a block of observations). This allows extending the maximum entropy technique to a dynamical setting distinguishing between entropic contributions of the signal and the error. Furthermore, it gives a suitable approximation of standard GME problems when exact solutions are hard to evaluate. We test this method performing numerical simulations at various sample sizes and batch dimensions. Moreover, we explore intermediate cases between streaming GCE and standard GCE, namely, when the update of estimations involves blocks of observations of different sizes, and we include collinearity effects. Finally, we discuss the results to highlight the main characteristics of this method, the range of application, and future perspectives.
READ FULL TEXT