Log In Sign Up

On the Role of Entropy-based Loss for Learning Causal Structures with Continuous Optimization

by   Ruichu Cai, et al.

Causal discovery from observational data is an important but challenging task in many scientific fields. Recently, NOTEARS [Zheng et al., 2018] formulates the causal structure learning problem as a continuous optimization problem using least-square loss with an acyclicity constraint. Though the least-square loss function is well justified under the standard Gaussian noise assumption, it is limited if the assumption does not hold. In this work, we theoretically show that the violation of the Gaussian noise assumption will hinder the causal direction identification, making the causal orientation fully determined by the causal strength as well as the variances of noises in the linear case and the noises of strong non-Gaussianity in the nonlinear case. Consequently, we propose a more general entropy-based loss that is theoretically consistent with the likelihood score under any noise distribution. We run extensive empirical evaluations on both synthetic data and real-world data to validate the effectiveness of the proposed method and show that our method achieves the best in Structure Hamming Distance, False Discovery Rate, and True Positive Rate matrices.


page 1

page 2

page 3

page 4


A new data fitting method for stretched Gaussian noise: stretched least square method

Stretched Gaussian distribution is the fundamental solution of the Hausd...

Score matching enables causal discovery of nonlinear additive noise models

This paper demonstrates how to recover causal graphs from the score of t...

The Effect of Noise Level on Causal Identification with Additive Noise Models

In recent years a lot of research has been conducted within the area of ...

CCSL: A Causal Structure Learning Method from Multiple Unknown Environments

Most existing causal structure learning methods require data to be indep...

Federated Causal Discovery

Causal discovery aims to learn a causal graph from observational data. T...

Information-Theoretic Approximation to Causal Models

Inferring the causal direction and causal effect between two discrete ra...