BELIEF in Dependence: Leveraging Atomic Linearity in Data Bits for Rethinking Generalized Linear Models

10/19/2022
by   Benjamin Brown, et al.
0

Two linearly uncorrelated binary variables must be also independent because non-linear dependence cannot manifest with only two possible states. This inherent linearity is the atom of dependency constituting any complex form of relationship. Inspired by this observation, we develop a framework called binary expansion linear effect (BELIEF) for assessing and understanding arbitrary relationships with a binary outcome. Models from the BELIEF framework are easily interpretable because they describe the association of binary variables in the language of linear models, yielding convenient theoretical insight and striking parallels with the Gaussian world. In particular, an algebraic structure on the predictors with nonzero slopes governs conditional independence properties. With BELIEF, one may study generalized linear models (GLM) through transparent linear models, providing insight into how modeling is affected by the choice of link. For example, setting a GLM interaction coefficient to zero does not necessarily lead to the kind of no-interaction model assumption as understood under their linear model counterparts. Furthermore, for a binary response, maximum likelihood estimation for GLMs paradoxically fails under complete separation, when the data are most discriminative, whereas BELIEF estimation automatically reveals the perfect predictor in the data that is responsible for complete separation. We explore these phenomena and provide a host of related theoretical results. We also provide preliminary empirical demonstration and verification of some theoretical results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/02/2020

Hierarchical Aitchison-Silvey models for incomplete binary sample spaces

Multivariate sample spaces may be incomplete Cartesian products, when ce...
research
03/17/2018

A two-stage estimation procedure for non-linear structural equation models

Applications of structural equation models (SEMs) are often restricted t...
research
10/12/2017

Asymptotic theory for maximum likelihood estimates in reduced-rank multivariate generalised linear models

Reduced-rank regression is a dimensionality reduction method with many a...
research
07/14/2015

Robust Estimation of Self-Exciting Generalized Linear Models with Application to Neuronal Modeling

We consider the problem of estimating self-exciting generalized linear m...
research
08/25/2022

On the maximum likelihood estimation in general log-linear models

General log-linear models specified by non-negative integer design matri...
research
09/01/2023

Optimal Scaling transformations to model non-linear relations in GLMs with ordered and unordered predictors

In Generalized Linear Models (GLMs) it is assumed that there is a linear...
research
02/01/2021

The problem of perfect predictors in statistical spike train models

Generalized Linear Models (GLMs) have been used extensively in statistic...

Please sign up or login with your details

Forgot password? Click here to reset