DeepAI AI Chat
Log In Sign Up

On the Convergence of Continuous Constrained Optimization for Structure Learning

by   Ignavier Ng, et al.

Structure learning of directed acyclic graphs (DAGs) is a fundamental problem in many scientific endeavors. A new line of work, based on NOTEARS (Zheng et al., 2018), reformulates the structure learning problem as a continuous optimization one by leveraging an algebraic characterization of DAG constraint. The constrained problem is typically solved using the augmented Lagrangian method (ALM) which is often preferred to the quadratic penalty method (QPM) by virtue of its convergence result that does not require the penalty coefficient to go to infinity, hence avoiding ill-conditioning. In this work, we review the standard convergence result of the ALM and show that the required conditions are not satisfied in the recent continuous constrained formulation for learning DAGs. We demonstrate empirically that its behavior is akin to that of the QPM which is prone to ill-conditioning, thus motivating the use of second-order method in this setting. We also establish the convergence guarantee of QPM to a DAG solution, under mild conditions, based on a property of the DAG constraint term.


page 1

page 2

page 3

page 4


Efficient Neural Causal Discovery without Acyclicity Constraints

Learning the structure of a causal graphical model using both observatio...

Global convergence rates of augmented Lagrangian methods for constrained convex programming

Augmented Lagrangian method (ALM) has been popularly used for solving co...

Penalty Augmented Kaczmarz Methods For Linear Systems Linear Feasibility Problems

In this work, we shed light on the so-called Kaczmarz method for solving...

On the Sparse DAG Structure Learning Based on Adaptive Lasso

Learning the underlying casual structure, represented by Directed Acycli...

Conditioning of linear systems arising from penalty methods

Penalizing incompressibility in the Stokes problem leads, under mild ass...

DAGs with NO TEARS: Smooth Optimization for Structure Learning

Estimating the structure of directed acyclic graphs (DAGs, also known as...