Solution Path of Time-varying Markov Random Fields with Discrete Regularization

by   Salar Fattahi, et al.

We study the problem of inferring sparse time-varying Markov random fields (MRFs) with different discrete and temporal regularizations on the parameters. Due to the intractability of discrete regularization, most approaches for solving this problem rely on the so-called maximum-likelihood estimation (MLE) with relaxed regularization, which neither results in ideal statistical properties nor scale to the dimensions encountered in realistic settings. In this paper, we address these challenges by departing from the MLE paradigm and resorting to a new class of constrained optimization problems with exact, discrete regularization to promote sparsity in the estimated parameters. Despite the nonconvex and discrete nature of our formulation, we show that it can be solved efficiently and parametrically for all sparsity levels. More specifically, we show that the entire solution path of the time-varying MRF for all sparsity levels can be obtained in ๐’ช(pT^3), where T is the number of time steps and p is the number of unknown parameters at any given time. The efficient and parametric characterization of the solution path renders our approach highly suitable for cross-validation, where parameter estimation is required for varying regularization values. Despite its simplicity and efficiency, we show that our proposed approach achieves provably small estimation error for different classes of time-varying MRFs, namely Gaussian and discrete MRFs, with as few as one sample per time. Utilizing our algorithm, we can recover the complete solution path for instances of time-varying MRFs featuring over 30 million variables in less than 12 minutes on a standard laptop computer. Our code is available at <>.

โˆ™ 02/06/2021

Scalable Inference of Sparsely-changing Markov Random Fields with Strong Statistical Guarantees

In this paper, we study the problem of inferring time-varying Markov ran...
โˆ™ 03/30/2021

A New Algorithm for Discrete-Time Parameter Estimation

We propose a new discrete-time adaptive algorithm for parameter estimati...
โˆ™ 06/21/2022

Efficient Inference of Spatially-varying Gaussian Markov Random Fields with Applications in Gene Regulatory Networks

In this paper, we study the problem of inferring spatially-varying Gauss...
โˆ™ 09/25/2020

Towards the interpretation of time-varying regularization parameters in streaming penalized regression models

High-dimensional, streaming datasets are ubiquitous in modern applicatio...
โˆ™ 05/12/2020

Stochastic Learning for Sparse Discrete Markov Random Fields with Controlled Gradient Approximation Error

We study the L_1-regularized maximum likelihood estimator/estimation (ML...
โˆ™ 06/22/2020

The space of sections of a smooth function

Given a compact manifold X with boundary and a submersion f : X โ†’ Y whos...
โˆ™ 09/01/2020

Time-Varying Parameters as Ridge Regressions

Time-varying parameters (TVPs) models are frequently used in economics t...

Please sign up or login with your details

Forgot password? Click here to reset