For those who have not recognized the disparate natures of tests of statistical hypotheses and tests of scientific hypotheses, one-tailed statistical tests of null hypotheses such as ∂ ≤ 0 or ∂ ≥ 0 have often seemed a reasonable procedure. We earlier reviewed the many grounds for not regarding them as such. To have at least some power for detection of effects in the unpredicted direction, sever...

Statistical analysis of a data set typically involves testing not just a single hypothesis, but rather many (often very many!). For any particular test, we may assign a pre-set probability α of a type-1 error (i.e., a false positive, rejecting the null hypothesis when in fact it is true). The problem is that using a (say) value of α = 0.05 means that roughly one out of every twenty such tests w...

This research was sponsored jointly by the United States Air Force through the Air Force Office of Scientific Research of the Air Research and Development Command under' Contract No. AF 49(638)-e61, by the Office of Naval Research under Contract No. Nonr 855(06) for research in probability and statistics at Chapel Hill, and by the U.S. Public Health Service. Reproduction in whole or in part for...

The objective of many animal experiments is to detect meaningful relationships among treatments and associated responses. Types of comparisons of means include pairwise multiple comparisons, planned orthogonal or nonorthogonal contrasts, and orthogonal polynomials. Some procedures are appropriate only for specific types of treatment designs and specific types of objectives. Pairwise, multiple c...

While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines se...

We study the problem of multiple hypothesis testing for correlated clustered data. As the existing multiple comparison procedures based on maximum likelihood estimation could be computationally intensive, we propose to construct multiple comparison procedures based on composite likelihood method. The new test statistics account for the correlation structure within the clusters and are computati...

This paper provides an overview of nonparametric predictive inference for comparison of multiple groups of data including right-censored observations. Different right-censoring schemes discussed are early termination of an experiment, progressive censoring and competing risks. Theoretical results are briefly stated, detailed justifications are presented elsewhere. The methods are illustrated an...

172 This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/ by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Statistical notes for clinical researchers: post-hoc multiple comparisons Open lecture ...