Magic for Filter Optimization in Dynamic Bottom-up Processing

Off-line compilation of logic grammars using Magic allows an incorporation of filtering into the logic underlying the grammar. The explicit definite clause characterization of filtering resulting from Magic compilation allows processor independent and logically clean optimizations of dynamic bottom-up processing with respect to goal-directedness. Two filter optimizations based on the program transformation technique of Unfolding are discussed which are of practical and theoretical interest. 1 I n t r o d u c t i o n In natural language processing filtering is used to weed out those search paths that are redundant, i.e., are not going to be used in the proof tree corresponding to the natural language expression to be generated or parsed. Filter optimization often comprises an extension of a specific processing strategy such that it exploits specific knowledge about grammars and/or the computational task(s) that one is using them for. At the same time it often remains unclear how these optimizations relate to each other and what they actually mean. In this paper I show how starting from a definite clause characterization of filtering derived automatically from a logic grammar using Magic compilation, filter optimizations can be performed in a processor independent and logically clean fashion. Magic (templates) is a general compilation technique for efficient bottom-up evaluation of logic programs developed in the deductive database community (Ramakrishnan et al., 1992). Given a logic program, Magic produces a new program in which the filtering as normally resulting from top-down evaluation is explicitly characterized through, so-called, *url: http://www.sfs.nphil.uni-tuebingen/'minnen magic predicates, which produce variable bindings for filtering when evaluated bottom-up. The original rules of the program are extended such that these bindings can be made effective. As a result of the definite clause characterization of filtering, Magic brings filtering into the logic underlying the grammar. I discuss two filter optimizations. These optimizations are direction independent in the sense that they are useful for both generation and parsing. For expository reasons, though, they are presented merely on the basis of examples of generation. Magic compilation does not limit the information that can be used for filtering. This can lead to nontermination as the tree fragments enumerated in bottom-up evaluation of magic compiled grammars are connected (Johnson, forthcoming). More specifically, 'magic generation' falls prey to non-termination in the face of head recursion, i.e., the generation analog of left recursion in parsing. This necessitates a dynamic processing strategy, i.e., memoization, extended with an abstraction function like, e.g., restriction (Shieber, 1985), to weaken filtering and a subsumption check to discard redundant results. It is shown that for a large class of grammars the subsumption check which often influences processing efficiency rather dramatically can be eliminated through fine-tuning of the magic predicates derived for a particular grammar after applying an abstraction function in an off-line fashion. Unfolding can be used to eliminate superfluous filtering steps. Given an off-line optimization of the order in which the right-hand side categories in the rules of a logic grammar are processed (Minnen et al., 1996) the resulting processing behavior can be considered a generalization of the head corner generation approach (Shieber et al., 1990): Without the need to rely on notions such as semantic head and chain rule, a head corner behavior can be mimicked in a strict bottom-up fashion.