A Performance Evaluation of OWL 2 DL Reasoners using ORE 2015 and Very Large Bio Ontologies

Abstract

In this paper, we evaluate the reasoning performance of six prominent OWL 2 DL reasoners, using two collections of ontologies: ORE 2015 and the 21 largest ontologies from the NCBO BioPortal. We observed that the majority of the reasoners were unable to successfully perform over half of the reasoning tasks in the NCBO BioPortal dataset which includes some very large ontologies. Despite of being a representative selection of the state-of-the-art OWL 2 DL reasoners, it came to our attention that many of them are no longer being actively maintained. These findings serve as a cautionary message to the OWL 2 reasoning community, emphasizing the need to focus research efforts towards keeping up with the growing requirements posed by new and very large ontologies and knowledge graphs.