Statistical significance tests.

Statistical methods for the analysis of data can be classed as descriptive or as probabilistic. The former methods include tabulation and graphical display, of great importance both in detailed analysis and also in presentation of conclusions. Probability enters statistical analysis in various ways. The most obvious is that one recognizes explicitly that conclusions drawn from limited data are uncertain and that therefore it is a good thing to measure and control probabilities of drawing the wrong conclusions, partly, but by no means solely, as a precaution against overinterpretation. The statistical significance test is the most widely known device concerned with such probabilities and is the subject of the present paper. The nature, use and misuse of significance tests have been much discussed in both statistical and nonstatistical circles. The view taken in this paper is that such tests play an important but nevertheless strictly limited role in the critical analysis of data. There is an extensive mathematical literature on the theory of such tests and on the distributions appropriate for special tests. The mathematics can largely be separated from critical discussion of the underlying concepts, that being the concern of the present paper.