Adversarially-regularized mixed effects deep learning (ARMED) models for improved interpretability, performance, and generalization on clustered data

02/23/2022
by   Kevin P. Nguyen, et al.
17

Data in the natural sciences frequently violate assumptions of independence. Such datasets have samples with inherent clustering (e.g. by study site, subject, experimental batch), which may lead to spurious associations, poor model fitting, and confounded analyses. While largely unaddressed in deep learning, mixed effects models have been used in traditional statistics for clustered data. Mixed effects models separate cluster-invariant, population-level fixed effects from cluster-specific random effects. We propose a general-purpose framework for building Adversarially-Regularized Mixed Effects Deep learning (ARMED) models through 3 non-intrusive additions to existing networks: 1) a domain adversarial classifier constraining the original model to learn only cluster-invariant features, 2) a random effects subnetwork capturing cluster-specific features, and 3) a cluster-inferencing approach to predict on clusters unseen during training. We apply this framework to dense feedforward neural networks (DFNNs), convolutional neural networks, and autoencoders on 4 applications including simulations, dementia prognosis and diagnosis, and cell microscopy. We compare to conventional models, domain adversarial-only models, and the naive inclusion of cluster membership as a covariate. Our models better distinguish confounded from true associations in simulations and emphasize more biologically plausible features in clinical applications. ARMED DFNNs quantify inter-cluster variance in clinical data while ARMED autoencoders visualize batch effects in cell images. Finally, ARMED improves accuracy on data from clusters seen during training (up to 28 conventional models) and generalizes better to unseen clusters (up to 9 conventional models). By incorporating powerful mixed effects modeling into deep learning, ARMED increases performance, interpretability, and generalization on clustered data.

READ FULL TEXT

page 4

page 9

page 11

page 12

page 21

page 22

page 23

page 26

research
07/13/2019

Inference for high-dimensional linear mixed-effects models: A quasi-likelihood approach

Linear mixed-effects models are widely used in analyzing clustered or re...
research
07/12/2019

Multilevel models for continuous outcomes

Multilevel models (mixed-effect models or hierarchical linear models) ar...
research
11/29/2022

UQ-ARMED: Uncertainty quantification of adversarially-regularized mixed effects deep learning for clustered non-iid data

This work demonstrates the ability to produce readily interpretable stat...
research
02/06/2023

A distribution-free mixed-integer optimization approach to hierarchical modelling of clustered and longitudinal data

We create a mixed-integer optimization (MIO) approach for doing cluster-...
research
05/09/2023

Mixed effects models for large sized clustered extremes

Extreme value theory (EVT) provides an elegant mathematical tool for the...
research
09/02/2022

Marginal Regression on Transient State Occupation Probabilities with Clustered Multistate Process Data

Clustered multistate process data are commonly encountered in multicente...

Please sign up or login with your details

Forgot password? Click here to reset