Efficient Big Text Data Clustering Algorithms using Hadoop and Spark

Document clustering is a traditional, efficient and yet quite effective, text mining technique when we need to get better insight of the documents collection that could be grouped together. The K-Means algorithm Hierarchical Agglomerative Clustering (HAC) are two most known commonly used algorithms; former due its low time cost latter accuracy. However, even use in over large-scale collections can lead unacceptable costs. In this paper first address some valuable approaches for document such 'big data' (large-scale) collections. We then present very promising alternatives: (a) variation an existing K-Means-based fast (known as BigKClustering - BKC) so it applied clustering, (b) hybrid approach based on customized version Buckshot algorithm, which applies hierarchical procedure sample input dataset uses results initial centers assignment rest documents, with few iterations. also give highly adaptations proposed techniques MapReduce model experimentally tested using Apache Hadoop Spark real cluster environment. As comes out experiments, they both acceptable quality well significant improvements (compared especially Buckshot-based algorithm), thus constituting alternatives big