Journal article
Sample size for binary logistic prediction models: Beyond events per variable criteria
Statistical methods in medical research, Vol.28(8), pp.2455-2474
08/2019
Handle:
https://hdl.handle.net/2376/105115
PMCID: PMC6710621
PMID: 29966490
Abstract
Binary logistic regression is one of the most frequently applied statistical approaches for developing clinical prediction models. Developers of such models often rely on an Events Per Variable criterion (EPV), notably EPV ≥10, to determine the minimal sample size required and the maximum number of candidate predictors that can be examined. We present an extensive simulation study in which we studied the influence of EPV, events fraction, number of candidate predictors, the correlations and distributions of candidate predictor variables, area under the ROC curve, and predictor effects on out-of-sample predictive performance of prediction models. The out-of-sample performance (calibration, discrimination and probability prediction error) of developed prediction models was studied before and after regression shrinkage and variable selection. The results indicate that EPV does not have a strong relation with metrics of predictive performance, and is not an appropriate criterion for (binary) prediction model development studies. We show that out-of-sample predictive performance can better be approximated by considering the number of predictors, the total sample size and the events fraction. We propose that the development of new sample size criteria for prediction models should be based on these three parameters, and provide suggestions for improving sample size determination.
Metrics
18 Record Views
Details
- Title
- Sample size for binary logistic prediction models: Beyond events per variable criteria
- Creators
- Maarten van Smeden - Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The NetherlandsKarel GM Moons - Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The NetherlandsJoris AH de Groot - Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The NetherlandsGary S Collins - Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Oxford, UKDouglas G Altman - Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Oxford, UKMarinus JC Eijkemans - Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The NetherlandsJohannes B Reitsma - Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
- Publication Details
- Statistical methods in medical research, Vol.28(8), pp.2455-2474
- Identifiers
- 99900546900901842
- Language
- English
- Resource Type
- Journal article