Decomposing Design Effects

1. Introduction Complex sample designs may include multistage sampling, stratification, and unequal selection probabilities. A consequence of the use of a complex sample design is that the sampling errors of the survey estimates cannot be computed assuming that the observed variables are independently and identically distributed (iid) random variables. Variances of survey estimates from complex sample designs may be estimated correctly by replication or by a Taylor Series linearization method (Wolter, 1985). When variances are computed properly accounting for the complex sample design, they typically are larger than those obtained from standard iid formulas. Kish coined the term "design effect" to denote the ratio of the variance of any estimate z obtained from a complex design to the variance of z that would apply with a SRS or unrestricted sample of the same size (Kish, 1965). The denominator of the Deff is the SRS variance. An alternative to the Deff is the Deft 2 , which has the same numerator but the denominator is the variance of an unrestricted sample (Kish, 1982). The only difference between Deff and Deft 2 is whether the fpc term is included. In our discussion, we assume that the fpc term can be ignored and thus use Deft 2. Since the design effect depends both on the sample design and on the survey estimate under consideration, we employ the notation D 2 (z) for the design effect (Deft 2) of the estimate z Variance of with complex design Variance of with unrestricted sample of the same size = D z z z. (1) The concept of a design effect has proven to be a valuable tool in the design of complex samples. It has also been used in the analysis of data from such samples, but with modern computing facilities the role of the design effect in analysis should diminish. Here we restrict our attention to uses in the design of surveys, particularly household surveys. This paper is based on a more extensive examination of these issues contained in a chapter prepared for a United Nations technical report (United Nations Statistical Division, forthcoming). The inclusion of a chapter in that report on using design effects in designing household surveys in developing countries attests to the practical importance of the topic. Complex designs involve a combination of a number of design components, such as stratification, multi-stage sampling, and selection with unequal probabilities. The analysis of the …