Extracting Useful Advice From Conflicting Expertise

A method for automatically identifying areas of disagreement and their sources is presented for multiexpert knowledge-based systems in the context of the Prospector consultation system. It employs performance evaluation techniques in combination with the explanatory facilities present in many expert systems to assist the user of an expert system in deciding which among several possibly conflicting expert opinions he should choose. 1 . I N T R O D U C T I O N An important issue encountered in the construction of expert problem-solving programs is how to deal with conflicting expert advice. This question arises even more tangibly in real-life decision-making—whether in the courts, the financial world, or in the context of our political and social institutions. Typical ly, groups supporting different sides of an issue enlist the services of highly respected experts who, as often as not, arrive at diametrically opposing conclusions or suggest conflicting courses of action. Even when experts do arrive at the same conclusion, how do we know they reached it for the same reasons7 The process by which the decision-maker, who presumably is not an expert himself (otherwise we might, witness another conflicting opinion), finally decides whose advice to follow is at best highly subjective. A significant consequence of the development of knowledge-based expert systems is that they render the reasoning processes of the expert more explicit and therefore open to scrutiny, evaluation, and testing. Most expert systems provide explanatory facilities that can communicate these reasoning processes to the user, thus enabling him to accept the system's opinion wi th greater confidenee--or at least providing him with a tool to assess their applicability to his problem. The question then, is: how can we take advantage of this explicit representation of expertise to design more objective multiexpert decision support tools? The knowledge bases of many existing expert systems are each the product of contributions from several experts, and could therefore be termed multiexpert This research was supported in part by the United States Geological Survey under Contract No. 14-18-001-20717. knowledge bases. Typically, the section of the knowledge base contributed by one expert encodes his particular area of expertise. For the purpose of this paper, however, we shall refer to these knowledge bases as nonconf l ic t ing multiexpert knowledge bases, restricting the term multiexpert knowledge, base to refer to the encoding by two or more experts of the same area of expertise. Because a multiexpert knowledge base encodes expertise obtained from several expert sources, segments of that expertise wil l undoubtedly be duplicated, while other segments might vary in diverse ways. For instance, source A could believe more strongly than source B in the association between evidence E1 and hypothesis H; source C, on the other hand, might not consider E1 important at all, but might ascribe more relevance to some entirely different observation, E2Obviously, there wil l be many interactions and inconsistencies in the resulting knowledge base. The problems encountered in representing such interactions, however, are not peculiar to conflicting multiexpert knowledge. In fact, similar interactions are encountered in nonconflicting knowledge bases, where the problem of maintaining the consistency of the knowledge base as it grows must be dealt with in a similar fashion. In [7] and [8| we explain in detail how such conflicts are discovered and resolved in the Prospector environment. For simplicity, and without loss of generality, we shall assume in the ensuing discussions that we have encoded in some suitable expert system the opinions of just two experts, A and 13, about a particular subject. The conclusions arrived at in the course of consultation with such a system might correspond, essentially, to the following: I f you were to consu l t w i t h exper t A, be would suggest . . . I f j o u were to consu l t w i t h exper t B , he would suggest . . . For most user categories these conclusions are unsatisfactory because the user sti l l has to make the final decision. If an appropriate explanatory facility is provided in the expert system, however, he might be able to probe the rationale underlying each expert's conclusions, identify the sources of possible disagreement, and perhaps derive a consensus opinion.