Reporting Response Rates when Survey and Administrative Data are Combined

At Statistics Canada, annual and monthly business surveys are using administrative data at an ever increasing rate. An important set of administrative data is received from the Canada Revenue Agency as a result of its collection of income tax reports and the Goods and Services Tax reports. These data are not only used to build and maintain a central frame, such as Statistics Canada’s Business Register, or to assist in the imputation of survey data, they are now used to completely or partially replace subpopulations that would have traditionally been surveyed. The primary aim of the increased use is to reduce response burden and survey costs. Relying on a strong correlation between the administrative data and the survey data, the survey data can be either replaced directly with administrative data or indirectly through the production of modelled values based on the relationship between the two sets of data. As such, more and more business survey estimates are being based on a combination of survey and administrative information. The traditional data quality indicators reported by surveys have been the sampling variance, coverage error, response rate and imputation rate. Are these indicators still relevant and sufficient in a context where administrative data are used? In fact, it is not unusual for some surveys to produce estimates that are based largely on the administrative source and thus reporting virtually no sampling error while other errors may be gaining in prominence: imputation error, model error, mode effects, etc. In 2004, a Task Force on Quality Indicators was set up at Statistics Canada to look into these issues and to recommend a strategy on how to report data quality in the context where survey and administrative data are combined. One of the main accomplishments achieved by the Task Force is a proposal to modify Statistics Canada’s Standards and Guidelines for Reporting of Nonresponse Rates. This proposal is the focus of this paper. The use of a pie chart is also proposed as a visual means of simultaneously showing the different data sources and the response/nonresponse rates for each source.