Intelligent Critiquing of Design Sketches

Design Evaluator is a pen-based system that provides designers with critical feedback on their sketches in various visual forms. The goal of these system-generated critiques is to help end users who draw and then reason about their drawings to solve design problems. This paper outlines the implementation strategies of the Design Evaluator and shows example applications in two visual design domains: architectural floor plans and Web page layout. Machine Annotation of Freehand Drawings Designers draw diagrams and sketches to record design idea and to reason about design constraints or alternatives. Designers in many domains (e.g. engineering, architecture) sketch graphical elements, accompanied by shorthand notes that record design rationale and other design information (Davis 2002). Design sketching is considered a component of a reasoning process that uses both visual and textual representations to solve problems. In this process, the ‘seeing’ that follows sketching involves both interpreting and evaluating designs that are represented as drawing marks on paper (Verstijinen, Heylighen et al. 1991). Seeing initiates subsequent design moves and problem reframing (Schön 1985; Goldschmidt 1991)). In this seeing process, textual and visual annotation or other forms of critique on the design drawing may help designers to explore alternatives and to reframe design problems. Critiquing is a familiar and important activity for designers. In a studio setting, a critic sees and reframes the design problem for another designer. Knowledge transfer occurs when the critic communicates what s/he sees and thinks to the designer through the combined media of sketching, talking, and showing examples. These acts help to restructure the designer’s knowledge, which in turn helps the designer to discover and explore new design alternatives. Critiquing systems were the research focus of Gerhard Fischer’s group at the University of Colorado during the late 1980s and 1990s. This effort resulted in a series of projects such as JANUS, which linked critiquing of kitchen layouts with design rationale (Fischer and Mørch 1988), KID, which linked text critiques with a construction kit (Fischer, Nakakoji et al. 1993), and the visual critiquing system Petri-NED (Stolze 1994). Each of these systems employed critiquing to support design. However they all employed structured editors which, we argue, tend to inhibit early design exploration. Sketch interaction has been explored in various domains. In UI design, for example, Landay and Myers’s (Landay and Myers 1995) SILK allows a designer to sketch elements of a prototype user interface; the system then transforms the sketch into a working interface. Sketch systems have also been built that support simulation in mechanical engineering (Davis 2002). Forbus, Usher and Chapman (2003) have explored pen-based interaction for military course-of-action planning. Our Electronic Cocktail Napkin project explored using sketch recognition as an interaction paradigm for a variety of tasks and domains, including accessing cases in a design database, simulation and knowledge based advising, and constraint based graphical editing (Gross 1996; Gross and Do 2000). In the Right Tool Right Time project (Do 1998) the system attempted to infer the user’s task; then based on this inference it suggested an appropriate knowledge-based tool. Although there is a great deal of interest in applying sketch interfaces to other kinds of intelligent systems, we are unaware of any other effort to provide a sketch interface to knowledge based critiquing. We believe this is a powerful combination, and that is the motivation behind our current work on the Design Evaluator. The Design Evaluator is an intelligent sketch system that attempts to reason about the design at hand and provide the designer with useful feedback in the form of criticism and advice. An intelligent sketch system requires two basic components: First, because a design problem-solving process inevitably involves specific domain knowledge, the system should have access to knowledge about the domain. Second, the system should present advice or critiques in appropriate form. For example, when a design process is heavily involved in sketching and diagramming, critiques should appear in similar and related form without distracting the designer from the tasks in which s/he is engaged. This paper describes the current Design Evaluator system, a domain-independent framework for sketch based critiquing. The system is composed of three layers: description, evaluation, and visualization. We have created specialized versions of the Design Evaluator for two design domains: architectural floor plans and Web page layout design. We describe each of these domain-specific versions, which have different rules and methods of displaying critiques. Design Evaluator— the three layers We have implemented the Design Evaluator, a pen-based critiquing system, in Macintosh Common Lisp. Design Evaluator consists of three layers: Description, Evaluation, and Visualization. Each layer supports a different sort of activities that designers carry out. After recording the designer’s sketch, the system tests it against previously stored predicates that express domain-specific design rules. Then, the system generates critiques and displays them in textual and visual forms.