From graphs to DAGs: a low-complexity model and a scalable algorithm

by   Shuyu Dong, et al.

Learning directed acyclic graphs (DAGs) is long known a critical challenge at the core of probabilistic and causal modeling. The NoTears approach of (Zheng et al., 2018), through a differentiable function involving the matrix exponential trace tr(exp(·)), opens up a way to learning DAGs via continuous optimization, though with a O(d^3) complexity in the number d of nodes. This paper presents a low-complexity model, called LoRAM for Low-Rank Additive Model, which combines low-rank matrix factorization with a sparsification mechanism for the continuous optimization of DAGs. The main contribution of the approach lies in an efficient gradient approximation method leveraging the low-rank property of the model, and its straightforward application to the computation of projections from graph matrices onto the DAG matrix space. The proposed method achieves a reduction from a cubic complexity to quadratic complexity while handling the same DAG characteristic function as NoTears, and scales easily up to thousands of nodes for the projection problem. The experiments show that the LoRAM achieves efficiency gains of orders of magnitude compared to the state-of-the-art at the expense of a very moderate accuracy loss in the considered range of sparse matrices, and with a low sensitivity to the rank choice of the model's low-rank component.


page 1

page 2

page 3

page 4


Low Rank Pure Quaternion Approximation for Pure Quaternion Matrices

Quaternion matrices are employed successfully in many color image proces...

Asymmetric Multiresolution Matrix Factorization

Multiresolution Matrix Factorization (MMF) was recently introduced as an...

Sketching sparse low-rank matrices with near-optimal sample- and time-complexity

We consider the problem of recovering an n_1 × n_2 low-rank matrix with ...

SKI to go Faster: Accelerating Toeplitz Neural Networks via Asymmetric Kernels

Toeplitz Neural Networks (TNNs) (Qin et. al. 2023) are a recent sequence...

Low Rank Directed Acyclic Graphs and Causal Structure Learning

Despite several important advances in recent years, learning causal stru...

Kissing to Find a Match: Efficient Low-Rank Permutation Representation

Permutation matrices play a key role in matching and assignment problems...

DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization

The combinatorial problem of learning directed acyclic graphs (DAGs) fro...

Please sign up or login with your details

Forgot password? Click here to reset