Automatic Text Summarization using Pathfinder Network Scaling

This thesis describes an automatic text summarization system based on graph-theory. Summaries are useful indicators of the document content. Traditionally summaries are created by humans by reading the text and identifying the important points in the text. So called information-overload makes such manual work very difficult (if not impossible). Many attempts have been made to automate the summarization task. This task is very difficult and involves natural language understanding. We propose an extractive text summarization system. Extractive summarization works by selecting a subset of sentences from the original text. Thus the system needs to identify most important sentences in the text. We use graph theory to identify relative importance of a sentence. The proposed method, SumGraph, uses a fully connected network to model a text with nodes as sentences and the lexical dissimilarity between pairs sentences as the link weights. This fully connected network is then scaled using Pathfinder Network Scaling technique. The scaled network reveals the conceptual organization of the sentences. Sentences are then selected for inclusion in the summary depending upon their relative importance in the conceptual network. Performance of the method is illustrated on a collection of newspaper articles. The data comes from Document Understanding Conferences. ROUGE measure is used for evaluating goodness of a summary. Comparison with other methods indicate that SumGraph produces better summaries.