A Randomized Trial of the Effectiveness of On-demand versus Computer-triggered Drug Decision Support in Primary Care

Design: This was a cluster trial with 28 primary care physicians randomized to either automated or on-demand CDDS in the MOXXI drug management system for 3,449 of their patients seen over the next 6 months. Measurements: The CDDS generated alerts for prescribing problems that could be customized by severity level. Prescribing problems included dosing errors, drug–drug, age, allergy, and disease interactions. Physicians randomized to on-demand activated the drug review when they considered it clinically relevant, whereas physicians randomized to computer-triggered decision support viewed all alerts for electronic prescriptions in accordance with the severity level they selected for both prevalent and incident problems. Data from administrative claims and MOXXI were used to measure the difference in the prevalence of prescribing problems at the end of follow-up. Results: During follow-up, 50% of the physicians receiving computer-triggered alerts modified the alert threshold (n 7), and 21% of the physicians in the alert-on-demand group modified the alert level (n 3). In the ondemand group 4,445 prescribing problems were identified, 41 (0.9%) were seen by requested drug review, and in 31 problems (75.6%) the prescription was revised. In comparison, 668 (10.3%) of the 6,505 prescribing problems in the computer-triggered group were seen, and 81 (12.1%) were revised. The majority of alerts were ignored because the benefit was judged greater than the risk, the interaction was known, or the interaction was considered clinically not important (computer-triggered: 75.8% of 585 ignored alerts; on-demand: 90% of 10 ignored alerts). At the end of follow-up, there was a significant reduction in therapeutic duplication problems in the computertriggered group (odds ratio 0.55; p 0.02) but no difference in the overall prevalence of prescribing problems. Conclusion: Customization of computer-triggered alert systems is more useful in detecting and resolving prescribing problems than on-demand review, but neither approach was effective in reducing prescribing problems. New strategies are needed to maximize the use of drug decision support systems to reduce drug-related morbidity. J Am Med Inform Assoc. 2008;15:430–438. DOI 10.1197/jamia.M2606. Affiliations of the authors: Department of Epidemiology and Biostatistics (RT, YK, MA), Department of Medicine (RT, AH, GB, NW, LP), Faculty of Management (LT, MD, AP), Department of Family Medicine (RG), McGill University, Montreal, Quebec, Canada; College of Physicians of Quebec (AJ), Montreal, Quebec, Canada; Department of Public Health, Régie Régionale de Montréal (RP), Montreal, Quebec, Canada. Supported by Health Canada, Canadian Institutes of Health Research. The authors thank Mr. Christian Savard and Ms. France Bourque of the Régie de l’assurance maladie du Québec, and Mr. Jimmy Fragos, Mr. Alphonse Estruch, Mr. Yannick Lazzari, Mr. David Marques, Mr. Jonathan Richard, and Ms. Irena Sesartic of the MOXXI Development Team, for their assistance. Correspondence: Dr. Robyn Tamblyn, McGill University, Morrice House, 1140 Pine Avenue West, Montreal Quebec, Canada, H3A 1A3; e-mail: [email protected] . Received for review: 08/23/07; accepted for publication: 04/14/08 Introduction At least 2% to 3% of ambulatory patients are treated each year for preventable adverse drug events, 58% of which are related to prescribing errors. Drug-related illness accounts for 5% to 23% of hospital admissions, and is now claimed to be the sixth leading cause of mortality. Computerized decision support (CDS) is considered to be a critical safety feature needed to reduce the risk of preventable adverse drug-related events. This is because dosing errors as well as drug–allergy, drug–drug, and drug–disease interactions are responsible for an important share of preventable adverse events, and integrated CDS can be designed to alert physicians at the point of prescribing about potential problems before a prescription is generated. However, experience has shown that physicians override 49% to 96% of alerts for drug, allergy, and disease contraindications, substantially reducing any potential value that Journal of the American Medical Informatics Association Volume 15 Number 4 July / August 2008 431 CDS may have in preventing prescribing errors and adverse events. The factors that lead physicians to override drug alerts are complex. Commercial vendors aim for comprehensiveness and leave the judgment of relevance to individual clinicians. As a result, physicians receive many alerts; a substantial proportion of which are considered clinically irrelevant. Physicians report that the sheer volume of alerts interferes with workflow, increases the likelihood that they will fail to respond to critical prescribing problems, and creates substantial barriers to the use of electronic prescribing systems altogether. Systems developed in-house that reduce the volume of alerts by targeting a limited set of drug problems have achieved the best success in altering prescribing practices. This suggests that the capacity to customize commercial drug alert systems to the local context may improve physician acceptance of electronic prescribing and drug management systems. Two approaches that can be used to customize drug decision support and alerts within integrated systems are to: (1) provide decision support only when a physician considers it relevant to request this information (on-demand decision support) or (2) provide automated computer-triggered decision support that can be modified by the physician to exclude alerts of a severity level that are considered not relevant (customizable computer-triggered decision support). The on-demand approach places a greater emphasis on workflow congruence by closely matching the way in which physicians use resources such as specialist consultants and drug interaction databases in usual practice. Physicians seek advice when they believe it is needed, and do so when it fits within their usual workflow. By optimizing workflow congruence, there should be substantially greater acceptance by physicians and a higher rate of response to alerts for prescribing problems. Yet there are important limitations of this approach. Even physicians who are confident in their ability to identify clinically relevant prescribing problems can only identify 51% of relevant drug problems correctly. Further, on-demand systems for expert assistance provide no safety net for forgetting, inadvertent oversights or data entry errors—problems that occur frequently in complex health care systems. The customizable computer-triggered decision support places a greater emphasis on patient safety, even at the risk of substantial workflow disruption. The customizable approach takes advantage of severity ratings for drug and disease interactions that are included in many commercial knowledge bases to allow individual physicians to filter the alerts they view. Within this context, physicians can customize an automated surveillance system by selecting severity ratings for the level of alerts to be viewed as well as actively filtering out clinically irrelevant alerts at the alert or patient level. In theory, the capacity to customize automated computertriggered alerts systems should provide the optimal approach to patient safety by providing physicians with the tools to create clinically relevant alerts, combined with the safety net to identify clinically relevant prescribing problems. We undertook this study to determine whether there would be a greater reduction in potential prescribing problems and a lower rate of alert overrides with a customizable computertriggered drug decision support system compared to a physician on-demand decision support system. We tested this hypothesis in a cluster-randomized controlled trial of primary care physicians and their patients.