Testing the Reliability and Validity of the Wisconsin Long Term Care Functional Screen

Two of the leading measures of an instrument’s quality are reliability (consistency) and validity (the degree to which it measures the phenomenon it aims to measure). In an effort to document the quality of the Wisconsin Long Term Care Functional Screen (LTCFS), two studies to measure the reliability and validity of this centerpiece of Family Care were undertaken during 1999. The reliability study utilized an inter-rater reliability design and consisted of a pilot phase and a main study. During the pilot phase, dual screens were administered to a sample of 98 clients in 9 Resource Centers in Wisconsin. The two screen were administered by different workers, independently and within a short time frame, to each client in the sample. The analysis focused on the level of agreement between the two screeners on core items of the LTCFS that are used in the process of establishing Family Care eligibility. The results of the analysis and the following explanations from the Resource Centers’ workers, had led to the modification of the LTCFS and the development of its current Version 2. The main Inter Rater Reliability Test (IRRT) study, had used a similar design and dual screening were conducted on a different sample of 96 clients. The sample included cases from all 3 Family Care target groups. The statistical analysis focused on 21 core items in the areas of ADL, IADL, Cognition, and Communication, and utilized two primary Statistics: the Chi-Square test of association, and Cohen’s Kappa correlation coefficient (for agreement between two raters). The analysis established highly significant levels of association and correlation. The Chi-Square results were all significant at the p<0.0001 level and the magnitudes of the Cohen’s Kappa which ranged from 0.59 to 0.92, were also statistically significant given the sample size and Degrees of Freedom. The results of the reliability study suggest that the LTCFS is a consistent tool that is expected to yield similar results when administered to clients in similar circumstances, within a short time frame. The methodology of building the LTCFS, had contributed to its second measure of quality, validity. The involvement of in-house and external experts, the borrowing of tested components from established instruments, and the consulting of written materials from other states and independent research sources, have greatly enhanced the face, content, and construct validity of the tool. Nevertheless, the Center undertook a special