Rejoinder: Some methodological issues in biosurveillance

I would first like to thank Professor David Buckeridge, Dr Howard Burkom, Dr Krista Hanni, Mr Henry Rolka, and Professors Bill Woodall and Kwok-Leung Tsui for their thoughtful, interesting, and informative commentaries. Given our divergent backgrounds—from local to federal public health, from academia to the CDC, from experience in industrial quality control to applied biosurveillance—I was a bit surprised but very pleased to find that we are all largely in agreement. In terms of the commentaries, let me begin by agreeing with Professor Buckeridge and Dr Hanni that developing standards and guidelines for biosurveillance studies would be very helpful in advancing the state of the art. As Professor Buckeridge correctly notes, my background and research interests focus on detection algorithms, but he is absolutely correct in suggesting that on-going research and development is necessary across the entire spectrum of biosurveillance functions and activities. For example, in some recent research with a local biosurveillance system based on EARS, we found that changing the text matching algorithms and/or the syndrome logic can have a large and significant impact on the number of individuals coded with a given syndrome. Specifically, for one year of county-level clinic data, 9093 visits (out of 153 696) were coded for the ILI syndrome using the unmodified EARS program. Modifying the symptom coding logic by allowing expanded text matching, in order to increase sensitivity, resulted in a 53 per cent increase in the number of ILI syndromes coded. In contrast, requiring at least two symptom matches prior to coding the ILI syndrome, in order to increase specificity, decreased the number of ILI syndromes coded to 8 per cent of the original. Making changes to the text matching logic, to improve both sensitivity and specificity, resulted similarly in large swings in the number of coded ILI syndromes. At issue is that little is known about which of these choices, all of which seem reasonable, would provide the best set of syndromic data upon which to use a detection algorithm. The obvious point is that a detection algorithm is only as good as the data, and there are arguably greater gains to be made with improvements in things such as data collection and management, natural language processing, and syndrome definitions than in trying to develop the next generation of detection algorithms. Directly related to this issue, Dr Hanni further makes the case for a standardized set of validated syndrome definitions, as well as standards for assessing the sensitivity and specificity of the definitions. The relevant question, as posed by Mr Rolka, is ‘How much reality is reflected in biosurveillance data?’ Mr Rolka further notes that the recent government initiatives are likely to lead to an ‘influx’ of data making the development of methods and procedures for ensuring data quality that much more important. Both Dr Hanni and Mr Rolka suggest that the public health community should take the lead in addressing this issue. In a similar vein, Professors Woodall and Tsui make the point that biosurveillance applications have considerably more sources of variation compared to the typical statistical process control (SPC) application. However, it is not at all clear, at least to me, that a concerted effort has been made to identify and eliminate as much variation as possible from