Evaluating Ontology Matchers Using Arbitrary Ontologies and Human Generated Heterogeneities

Automatic ontology matching is a hard problem. To address this problem many ontology matchers have evolved in the past several years. Consequently the evaluation of ontology matchers has become crucial in order to help improve a matcher’s performance. The evaluation frameworks used so far are limited to available pairs of ontologies in certain domains and require the reference alignments (i.e., gold standards) to be specified manually. In this paper we present a novel ontology matcher evaluation approach which can accept any OWL ontology as the source ontology. With little human efforts to specify the changes to the source ontology, our system can automatically construct the target ontology and generate the gold standard of correspondences. Compared to well-known evaluators (e.g., OAEI), our approach can provide more meaningful feedback besides traditional accuracy and completeness measures by indicating the performance of ontology matchers according to various types of heterogeneities.