Use of statistical tests of equivalence (bioequivalence tests) in plant pathology.

Hypothesis tests currently used in plant pathology are almost always based on a null hypothesis of equal means. In this framework, the experimenter determines whether or not there is evidence that the means are, in fact, different. This framework makes sense for many common questions such as whether a new management technique gives an increase in yield over existing management techniques. But suppose, for example, that a disease management technique is so effective that an experimenter is interested in whether its use in the presence of disease achieves the same yield as in the absence of disease. In this case, a more appropriate null hypothesis would be that mean yields are different. Examples of questions in plant pathology for which a null hypothesis of equal treatment means is not suitable include (corresponding phrasing for one-sided questions is in parentheses as appropriate): (i) Is an engineered organism equivalent to the original organism in all relevant characteristics except the intended change? (ii) Is disease severity the same for a cheaper or safer management strategy as for a standard strategy? (Is it at least as low?) (iii) Does a cultivar with potential for higher yield have the same level of disease resistance as a proven resistant cultivar? (Is it at least as high?) (iv) Is the level of pesticide on a plant surface the same for different application techniques? (v) Does the yield reach that of pathogen-free or disease-free plants when • a biocontrol agent is used? • a pesticide is applied? • a resistant cultivar is grown? (Is it at least as high?) (vi) In general, does a new approach with certain advantages perform as well as a standard “best” approach? (Does it perform at least as well?) For the last question, in the standard hypothesis testing framework, a test would be made to determine whether there is evidence that performance of the new approach is different from performance of the standard. Based on a null hypothesis of equal means, if there is not evidence for this difference, the experimenter may fail to reject the null hypothesis, but cannot, therefore, conclude that performance is equal. Failure to find evidence for a difference might result simply because the experiment has low statistical power (see below). The equivalence null hypothesis framework offers techniques for testing whether means are functionally equivalent using a null hypothesis of different means (4,5,7). Equivalence tests are perhaps the second most useful general class of hypothesis tests after the standard hypothesis testing framework. Following is a discussion of the standard hypothesis framework, the concept of statistical power, how the equivalence framework differs from the standard framework, an example of an equivalence test, and a summary of some of the available equivalence test techniques. Standard hypothesis framework. The standard hypothesis testing framework is based on the idea that what constitutes “important news” is evidence of a difference between treatment means. This is, of course, true for many questions. Traditionally, greater emphasis has been placed on protecting against type I errors (concluding there is evidence of a difference when a difference does not exist) than on protecting against type II errors (concluding there is not evidence of a difference when a difference does exist) (2). This has been part of a general notion that the burden of proof should be on researchers to demonstrate that they have important news if their work is to be published and considered. There has generally been little concern for whether negative results are also published and given consideration, which has the potential to result in the “file drawer” problem (13). In other words, positive results may be published, while negative results are filed and a literature-wide bias may result. When the standard hypothesis framework is used for questions in which interest lies in whether means are equivalent, the results are often inconclusive. If the difference between means is relatively large and there is adequate power, there may be evidence to reject a null hypothesis of equivalent means. If the means are similar or there is low power, the typical emphasis on protecting against type I errors may mean that there will not be evidence to reject the null hypothesis of equivalent means. This does not indicate that the means are equivalent, however. The experiment may simply have had low power because of a small sample size or high variation. Thus, an experiment that is too small may be more likely to result in a lack of evidence for a difference, regardless of what the means actually are. Statistical power. Power is the probability that the null hypothesis will be rejected if it is not true, or one minus the probability of a type II error (3). The power of a test increases as the sample size increases and as the level of unexplained variability decreases. For a null hypothesis of equal means, power increases as the actual difference between means increases. High power also results in narrower confidence intervals around parameter estimates. Low power may make it difficult to demonstrate a real difference between treatments, especially if the difference, or effect size, is small in magnitude. A power analysis can be an important part of planning an experiment, allowing an experimenter to pick an appropriate sample size for an estimate of the variance and the effect size (3). Equivalence test hypothesis framework. The equivalence test hypothesis framework employs a null hypothesis of unequal means. In this context, “important news” is evidence that means are equivalent. Because two population means will never be truly identical, the null hypothesis used in practice is that the difference Corresponding author: K. A. Garrett; E-mail address: [email protected]