The ensemble clustering with maximize diversity using evolutionary optimization algorithms

Data clustering is one of the main steps in data mining, which is responsible for exploring hidden patterns in non-tagged data. Due to the complexity of the problem and the weakness of the basic clustering methods, most studies today are guided by clustering ensemble methods. Diversity in primary results is one of the most important factors that can affect the quality of the final results. Also, the quality of the initial results is another factor that affects the quality of the results of the ensemble. Both factors have been considered in recent research on ensemble clustering. Here, a new framework for improving the efficiency of clustering has been proposed, which is based on the use of a subset of primary clusters, and the proposed method answers the above questions and ambiguities. The selection of this subset plays a vital role in the efficiency of the assembly. Since evolutionary intelligent algorithms have been able to solve the majority of complex engineering problems, this paper also uses these intelligent methods to select subsets of primary clusters. This selection is done using three intelligent methods (genetic algorithm, simulation annealing and particle swarm optimization). In this paper a clustering ensemble method is proposed which is based on a subset of primary clusters. The main idea behind this method is using more stable clusters in the ensemble. The stability is applied as a goodness measure of the clusters. The clusters which satisfy a threshold of this measure are selected to participate in the ensemble. For combining the chosen clusters, a co-association based consensus function is applied. A new EAC based method which is called Extended Evidence Accumulation Clustering, EEAC, is proposed for constructing the Co-association Matrix from the subset of clusters. Experimental results on several standard datasets with normalized mutual information evaluation, Fisher and accuracy criteria compared to Alizadeh, Azimi, Berikov, CLWGC, RCESCC, KME, CFSFDP, DBSCAB, NSC and Chen methods show the significant improvement of the proposed method in comparison with other ones.  Keywords: Clustering Ensemble, local optimization, evolutionary algorithm, correlation matrix, diversity.