Measuring the Stability of Learned Features

02/20/2021
by   Kris Sankaran, et al.
0

Many modern datasets don't fit neatly into n × p matrices, but most techniques for measuring statistical stability expect rectangular data. We study methods for stability assessment on non-rectangular data, using statistical learning algorithms to extract rectangular latent features. We design controlled simulations to characterize the power and practicality of competing approaches. This motivates new strategies for visualizing feature stability. Our stability curves supplement the direct analysis, providing information about the reliability of inferences based on learned features. Finally, we illustrate our approach using a spatial proteomics dataset, where machine learning tools can augment the scientist's workflow, but where guarantees of statistical reproducibility are still central. Our raw data, packaged code, and experimental outputs are publicly available.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 12

page 29

page 30

page 32

page 35

page 36

01/23/2019

Three principles of data science: predictability, computability, and stability (PCS)

We propose the predictability, computability, and stability (PCS) framew...
01/31/2020

Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models

Nowadays we are witnessing a transformation of the business processes to...
11/14/2020

Deep Spatial Learning with Molecular Vibration

Machine learning over-fitting caused by data scarcity greatly limits the...
08/13/2020

Learning Stability Certificates from Data

Many existing tools in nonlinear control theory for establishing stabili...
07/28/2017

Review of Machine Learning Algorithms in Differential Expression Analysis

In biological research machine learning algorithms are part of nearly ev...
01/28/2017

A Study of FOSS'2013 Survey Data Using Clustering Techniques

FOSS is an acronym for Free and Open Source Software. The FOSS 2013 surv...
05/09/2021

Statistical Assessment of Replicability via Bayesian Model Criticism

Assessment of replicability is critical to ensure the quality and rigor ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.