New formats and interfaces for multi-document news summarization and its evaluation

News production, delivery, and consumption are increasing in ubiquity and speed, spreading over more software and hardware platforms, in particular mobile devices. This has led to an increasing interest in automated methods for multi-document summarization. We start this chapter with discussing several new alternatives for automated news summarization, with a particular focus on temporal text mining, graphbased methods, and graphical interfaces. Then we present automated and user-centric frameworks for cross-evaluating summarization methods that output different summary formats, and describe the challenges associated with each evaluation framework. Based on the results of our user studies, we argue that it is crucial for effective summarization to integrate the user into sense-making through usable, entertaining and ultimately useful interactive summarization-plus-document-search interfaces. In particular, graph-based methods and interfaces may be a better preparation for people to concentrate on what is essential in a collection of texts, and thus may be a key to enhancing the summary evaluation process by replacing the “one gold standard fits all” approach with carefully designed user studies built upon a variety of summary representation formats.