Parse disambiguation for a rich HPSG grammar

The fine-grained nature of the HPSG representations found in the Redwoods treebank raises novel issues in parse disambiguation relative to more traditional treebanks such as the Penn treebank, which have been the focus of most past work on probabilistic parsing (e.g., Charniak 1997; Collins 1997). The Redwoods treebank is much richer in the representations it makes available. Most similar to Penn treebank parse trees are the phrase structure trees (Figure 1(b)). In this work we have concentrated on the derivation trees (Figure 1(a)), which represent combining rule schemas of the HPSG grammar. The nodes represent, for example, head-complement, head-specifier, and head-adjunct schemas and the derivation trees are consequently significantly different from phrase structure trees. The preterminals of the derivation trees are from a set of about 8,000 lexical labels and are much finer grained than Penn treebank labels, which are about 45 part-of-speech tags, and 27 phrasal node labels.