Mining Disjunctive Sequential Patterns from News Stream

Frequent disjunctive pattern is known to be a sophisticated method of text mining in a single document that satisfies anti-monotonicity, by which we can discuss efficient algorithm based on APRIORI. In this work, we propose a new online and single-pass algorithm by which we can extract current frequent disjunctive patterns by a weighting method for past events from a news stream. And we discuss some experimental results. 1 Motivation Recently much attention has been focused on quantitative analysis of several features of text thus a new approach, called text mining, has been proposed to extend traditional methodologies[13]. Text mining approach comes from frequency and co-occurrence among text 1 , the former means important words arise many times while the latter says related words occur at the same time[10]. On the other hand, recently much attention has been focused on data mining on data stream which is huge amount dynamic data on network[6]. Generally we see that data stream has some characteristics;(1)huge amount, (2)high speed, (3)changing data distribution, (4)continuous. Therefor we like algorithms which should be able to output approximate solutions for any time and any stream during the whole process and rapid analysis with limited calculation resources. That's why we cannot apply traditional methods for this problem[6]. Especially data stream such as news articles and broadcast transcription is called news stream. Generally we know that issue interval of news articles are irregular. So we assume that time stamp of news articles correspond to content-time of news articles. Under the news stream environment, we see that news articles always occur, and collections of news articles grow incrementally with new ones. We usually have interests in recent ones, i.e., new ones are more important than old. There have been important research results obtained based on anti-monotonicity such as APRIORI[2, 4]. Generally we can avoid vast range of search but not enough to sequence data, since in text mining the property doesn't hold any more and we can't apply APRIORI technique any more. An interesting approach of disjunctive patterns has been proposed with a new counting method for frequency to get efficient pattern mining algorithm on single sequence data. The counting method satisfies anti-monotonicity thus we can obtain efficient mining algorithm based on APRIORI[11].