Intelligent Assistance through Collaborative Manipulation

This paper introduces, motivates, and illustrates an approach to the construction of intelligent assistance systems that we call collaborative manipulation. We show how a system can offer effective assistance through collaborative manipulation of objects in a shared workspace. We have developed this approach through experience with an intelligent knowledge editing tool, the HITS Knowledge Editor. We illustrate its effectiveness using scenarios taken from a user study. 1 Intelligent Assistance Intelligent assistance is an active research field within AI [Chin, 1988; Lochbaum et al., 1990; Lemke and Fischer, 1990; Miller et al., 1990]. This research is motivated by several factors, including (1) the importance of collaboration in intelligent activity, (2) the scarcity of totally formalizable domains, and (3) people's need for help with increasingly complex computer applications. Two key issues in the design of an intelligent assistance system are what is the role of the system in the inter action? and how is the system-user interaction managed! Our work in building an intelligent assistant for the task of knowledge editing has led us to three design principles that address these issues. We call the approach characterized by these principles collaborative manipulation. 2 Collaborative Manipulation We carry out our research within the paradigm of cooperative problem solving systems [Lemke and Fischer, 1990]. This approach begins from the premise that people and computers have vastly different strengths and weaknesses and that effective cooperation needs a division of responsibility based on the strengths of each party. Our contribution is to base system assistance on collaborative manipulation of objects in a shared workspace. The approach has three key aspects. 1. Provide a workspace for joint user-system problem solving. People at everyday tasks construct personalized work contexts that include task-relevant materials and partial specifications of solutions think of a kitchen while someone is cooking or your workstation and desk while you This work was done while all three authors were at the MCC Human Interface Lab. debug a program or write a paper. In this paper, we focus on two properties of a workspace that are useful for an intelligent assistant: first, it provides access to users partial solutions, enabling the assistant to compute advice in a timely manner, and second, the assistant can deliver significant aspects of its advice by manipulating objects in the shared workspace. We call the latter process advertisement [Wroblewski et al., 1991]. 2. An effective role for an intelligent assistance system is that of a design critic [Fischer et al., 1990]. In design, a person constructs an artifact meeting certain constraints in knowledge editing, users construct knowledge structures that encode their understanding of a domain and fit in with the constraints of a knowledge representation system. A critic "looks over the shoulder" of users as they perform a task and offers advice occasionally. In our system, critics propose completions of unfinished objects, detect problems, and suggest additional issues. 3. The user-system interaction must be managed according to conventions that are appropriate to the abilities and roles of each party. For example, a system should (1) avoid taking the initiative from users and forcing them to deal with advice at the convenience of the system, and (2) avoid imposing a fixed order of work on users. In our system, interaction is organized in terms of propose-critique-refine interchanges, in which the user always has the final say. 3 The HITS Knowledge Editor Knowledge editing involves the entry, viewing, access, and maintenance of information in a knowledge base. Many systems make strong assumptions about the type and use of knowledge being entered [e.g. Kahn et al., 1987; Musen et al, 1987] in order to guide users. Our approach, however, along with Murray and Porter [1990] is to assist users in the knowledge editing task without making such assumptions. Representing knowledge in a knowledge base is a difficult task. People must articulate knowledge to a higher degree of precision than is required for everyday communication and must encode the new knowledge in harmony with existing knowledge and representational conventions. The HITS Knowledge Editor (HKE) assists users in this task. HKE is a browsing/entry interface to CYC [Lenat and Guha, 1990]. HKE embodies an analysis of knowledge editing into six sub-activities [Terveen, 1991]. Terveen, Wroblewski, and Tighe 9 Double lines around an icon indicate that an object by this name exists in the knowledge base, e.g., Mac exists but ACT does not. The same convention is used for slot buttons in the Choose Slot menu: e.g., performs exists in the KB but product does not. The Choose Slot menu also functions like the legend of a map. Each slot has an associated line pattern, e.g., subOrganizations is represented by a thick solid line. The icons for two objects related by one of these slots are linked by the appropriate line pattern, e.g., MCC and ACT are related by the subOrganizations slot, so their icons are linked by a thick solid line.