Comparing χ tests for separability
نویسنده
چکیده
This paper describes a series of novel statistical meta-tests for comparing experimental runs for significant difference, for conditions where experiments are carried out using Binomial or multinomial contingency statistics (χ, z, log-likelihood tests, etc.). The new tests permit us to evaluate whether experiments have failed to replicate on new data; whether a particular data source or subcorpus obtains a significantly different result than another; or whether changing experimental parameters obtains a stronger effect. Recognising when an experiment obtains a significantly different result and when it does not is an issue frequently overlooked in research publication. Papers are frequently published citing ‘p values’ or test scores suggesting a ‘stronger effect’, substituting for sound statistical reasoning. This paper sets out a series of tests which together illustrate the correct approach to this question, namely, to compute confidence intervals for differences between effect sizes and distributions. The meta-tests presented are derived mathematically from the χ test and the Wilson score interval, and cover both ‘goodness of fit’ and pairwise ‘homogeneity’ tests. Meta-tests for comparing tests with one degree of freedom (‘2 × 1’ and ‘2 × 2’ tests) are generalised to those of arbitrary size (‘r × 1’ and ‘r × c’). Finally, we compare our approach with a competing approach offered by Zar (1999), which, while straightforward to calculate, turns out to be both less powerful and less robust.
منابع مشابه
Comparing χ tables for separability of distribution and effect Meta-tests for comparing homogeneity and goodness of fit contingency test outcomes
This paper 1 describes a series of statistical meta-tests for comparing independent contingency tables for different types of significant difference. Recognising when an experiment obtains a significantly different result and when it does not is frequently overlooked in research publication. Papers are frequently published citing ‘p values’ or test scores suggesting a ‘stronger effect’ substitu...
متن کاملEvaluation of Tests for Separability and Symmetry of Spatio-temporal Covariance Function
In recent years, some investigations have been carried out to examine the assumptions like stationarity, symmetry and separability of spatio-temporal covariance function which would considerably simplify fitting a valid covariance model to the data by parametric and nonparametric methods. In this article, assuming a Gaussian random field, we consider the likelihood ratio separability test, a va...
متن کاملz-squared: The Origin and Application of χ2
A set of statistical tests termed contingency tests, of which χ is the most well-known example, are commonly employed in linguistics research. Contingency tests compare discrete distributions, that is, data divided into two or more alternative categories, such as alternative linguistic choices of a speaker or different experimental conditions. These tests are highly ubiquitous, and are part of ...
متن کاملz-squared: The Origin and Application of v2*
A set of statistical tests termed contingency tests, of which χ is the most well-known example, are commonly employed in linguistics research. Contingency tests compare discrete distributions, that is, data divided into two or more alternative categories, such as alternative linguistic choices of a speaker or different experimental conditions. These tests are highly ubiquitous, and are part of ...
متن کاملSequential closures of σ-subalgebras for a vector measure
Let X be a locally convex space, m : Σ → X be a vector measure defined on a σ-algebra Σ, and L1(m) be the associated (locally convex) space of m-integrable functions. Let Σ(m) denote {χ E ;E ∈ Σ}, equipped with the relative topology from L1(m). For a subalgebra A ⊆ Σ, let Aσ denote the generated σ-algebra and As denote the sequential closure of χ(A) = {χ E ;E ∈ A} in L1(m). Sets of the form As ...
متن کامل