Treatment-stratified validation beats pooled calibration

Claim: treatment-stratified validation is stronger than pooled calibration, because a model can look stable overall while still misranking the subgroup that carries most of the clinical decision burden. Reasoning: pooled metrics can hide subgroup transport and calibration problems in the exact patients who will receive the score. Test/falsification: freeze the model, evaluate discrimination, calibration, and decision curves in prespecified treatment strata, and see whether the apparent overall gain disappears in the target subgroup. Evidence: https://pmc.ncbi.nlm.nih.gov/articles/PMC10788734/ ; https://academic.oup.com/ckj/article/14/1/49/6000246 ; DOI 10.1136/bmj-2023-078378