DeepAI AI Chat
Log In Sign Up

A Fast Divide-and-Conquer Sparse Cox Regression

by   Yan Wang, et al.
Harvard University

We propose a computationally and statistically efficient divide-and-conquer (DAC) algorithm to fit sparse Cox regression to massive datasets where the sample size n_0 is exceedingly large and the covariate dimension p is not small but n_0≫ p. The proposed algorithm achieves computational efficiency through a one-step linear approximation followed by a least square approximation to the partial likelihood (PL). These sequences of linearization enable us to maximize the PL with only a small subset and perform penalized estimation via a fast approximation to the PL. The algorithm is applicable for the analysis of both time-independent and time-dependent survival data. Simulations suggest that the proposed DAC algorithm substantially outperforms the full sample-based estimators and the existing DAC algorithm with respect to the computational speed, while it achieves similar statistical efficiency as the full sample-based estimators. The proposed algorithm was applied to an extraordinarily large time-independent survival dataset and an extraordinarily large time-dependent survival dataset for the prediction of heart failure-specific readmission within 30 days among Medicare heart failure patients.


page 1

page 2

page 3

page 4


Scalable Sparse Cox's Regression for Large-Scale Survival Data via Broken Adaptive Ridge

This paper develops a new sparse Cox regression method for high-dimensio...

Optimal Cox Regression Subsampling Procedure with Rare Events

Massive sized survival datasets are becoming increasingly prevalent with...

Matching methods for obtaining survival functions to estimate the effect of a time-dependent treatment

In observational studies of survival time featuring a binary time-depend...

ROC-Guided Survival Trees and Forests

Tree-based methods are popular nonparametric tools in studying time-to-e...

Data-driven sparse polynomial chaos expansion for models with dependent inputs

Polynomial chaos expansions (PCEs) have been used in many real-world eng...

Scalable Algorithms for Large Competing Risks Data

This paper develops two orthogonal contributions to scalable sparse regr...