¶ Cross-validatory choice and assessment of statistical predictions

pdf link

¶ BibTeX

@article{stone1974cross,
  title={Cross-validatory choice and assessment of statistical predictions},
  author={Stone, Mervyn},
  journal={Journal of the royal statistical society: Series B (Methodological)},
  volume={36},
  number={2},
  pages={111--133},
  year={1974},
  publisher={Wiley Online Library}
}

¶ Abstract

A generalized form of the cross-validation criterion is applied to the choice and assessment of prediction using the data-analytic concept of a prescription. The examples used to illustrate the application are drawn from the problem areas of univariate estimation, linear regression and analysis of variance.

¶ Notes and Excerpts

ELS lists this as a “key reference” for cross-validation,

This paper gives credit in the following way.

The refinement of this type of assessment that gives us our cross-validation criterion appears to have been developed by Lachenbruch following a suggestion in Mosteller and Wallace (1963). Key references are Hills (1966), Kerridge in discussion of the same, Lachenbruch and Mickey (1968) and Cochran (1968). The context of all this work is that of discrimination, as is the application described by Mosteller and Tukey (1968) to whom, however, we are indebted for the first clear general statement of the refinement. Their description of what they term “simple cross-validation” is worth reproducing :

“Suppose that we set aside one individual case, optimize for what is left, then test on the set-aside case. Repeating this for every case squeezes the data almost dry. If we have to go through the full optimization calculation every time, the extra computation may be hard to face. Occasionally we can easily calculate either exactly or to an adequate approximation what the effect of dropping a specific and very small part of the data will be on the optimized result. This adjusted optimized result can then be compared with the values for the omitted individual. That is, we make one optimization for all the data, followed by one repetition per case of a much simpler calculation, a calculation of the effect of dropping each individual, followed by one test of that individual. When practical, this approach is attractive.”