Minimum sample size for developing a multivariable prediction model: PART II ‐ binary and time‐to‐event outcomes

Richard D Riley; Kym IE Snell; Joie Ensor; Danielle L Burke; Frank E Harrell Jr; Karel GM Moons; Gary S Collins

doi:10.1002/sim.7992

Back

Minimum sample size for developing a multivariable prediction model: PART II ‐ binary and time‐to‐event outcomes

Journal article

Open access

Peer reviewed

Minimum sample size for developing a multivariable prediction model: PART II ‐ binary and time‐to‐event outcomes

Richard D Riley, Kym IE Snell, Joie Ensor, Danielle L Burke, Frank E Harrell Jr, Karel GM Moons and Gary S Collins

Statistics in medicine, Vol.38(7), pp.1276-1296

03/30/2019

DOI: https://doi.org/10.1002/sim.7992

Handle:

https://hdl.handle.net/2376/100858

PMCID: PMC6519266

PMID: 30357870

Files and links (2)

pdf

Statistics in Medicine - 20181.61 MBDownload View

CC BY-NC V4.0, Open Access

url

https://doi.org/10.1002/sim.7992View

Published (Version of record) Open

Abstract

multivariable prediction model

logistic and Cox regression

sample size

shrinkage

binary and time‐to‐event outcomes

pseudo R‐squared

When designing a study to develop a new prediction model with binary or time‐to‐event outcomes, researchers should ensure their sample size is adequate in terms of the number of participants (n) and outcome events (E) relative to the number of predictor parameters (p) considered for inclusion. We propose that the minimum values of n and E (and subsequently the minimum number of events per predictor parameter, EPP) should be calculated to meet the following three criteria: (i) small optimism in predictor effect estimates as defined by a global shrinkage factor of ≥0.9, (ii) small absolute difference of ≤ 0.05 in the model's apparent and adjusted Nagelkerke's R2, and (iii) precise estimation of the overall risk in the population. Criteria (i) and (ii) aim to reduce overfitting conditional on a chosen p, and require prespecification of the model's anticipated Cox‐Snell R2, which we show can be obtained from previous studies. The values of n and E that meet all three criteria provides the minimum sample size required for model development. Upon application of our approach, a new diagnostic model for Chagas disease requires an EPP of at least 4.8 and a new prognostic model for recurrent venous thromboembolism requires an EPP of at least 23. This reinforces why rules of thumb (eg, 10 EPP) should be avoided. Researchers might additionally ensure the sample size gives precise estimates of key predictor effects; this is especially important when key categorical predictors have few events in some categories, as this may substantially increase the numbers required.

Metrics

22 Record Views

See more details

Details

Title: Minimum sample size for developing a multivariable prediction model: PART II ‐ binary and time‐to‐event outcomes
Creators: Richard D Riley - Keele University
Kym IE Snell - Keele University
Joie Ensor - Keele University
Danielle L Burke - Keele University
Frank E Harrell Jr - Vanderbilt University School of Medicine
Karel GM Moons - University Medical Centre Utrecht
Gary S Collins - University of Oxford
Publication Details: Statistics in medicine, Vol.38(7), pp.1276-1296
Academic Unit: Department of Physics and Astronomy
Number of pages: 1
Grant note: NIHR Biomedical Research Centre, Oxford Netherlands Organisation for Scientific Research (9120.8004; 918.10.615) National Centre for Advancing Translational Sciences (UL1 TR002243) National Institute for Health Research School for Primary Care Research (NIHR SPCR)
Identifiers: 99900546683801842
Language: English
Resource Type: Journal article