Evaluating the Utility of Appraisal Hierarchies as a Method for Sentiment Classification

Recent studies of sentiment classification (determining whether a text is “positive” or “negative”) using Appraisal theory have provided mixed results. While some good results have been obtained, it is difficult to tell what aspects of Appraisal are particularly useful for this task. In this paper, we present a series of experiments to isolate features of Appraisal, in order to compare which parts aid the task of sentiment classification on movie reviews. We report results which on the surface challenge the utility of Appraisal Hierarchies for this task, when modelled using systemic features. However in the context of making a trade-off between coverage and scale of feature space, our results appear promising. We hence discuss the need for a balance between the size of a classifier’s structure and the overall accuracy.