Ontology Usability Scale: Context-aware Metrics for the Effectiveness, Efficiency and Satisfaction of Ontology Uses

Xiaogang Ma; Linyun Fu; Patrick West; Peter Fox

doi:10.5334/dsj-2018-010

Ontology Usability Scale: Context-aware Metrics for the Effectiveness, Efficiency and Satisfaction of Ontology Uses

Data Science Journal

Volume 17 (2018): Issue 0

By: Xiaogang Ma , Linyun Fu, Patrick West and Peter Fox

Open Access

|May 2018

Abstract

Both ontology builders and users need a way to evaluate ontologies in terms of usability, but existing ontology evaluation approaches do not fit this purpose. We propose the Ontology Usability Scale (OUS), a ten-item Likert scale derived from statements prepared according to a semiotic framework and an online poll in the Semantic Web community to provide a practical way of ontology usability evaluation. Case studies were conducted to bookkeep current usability evaluation results for ontologies expecting revisions in the future, and discussions of the poll results are presented to help proper use and customization of the OUS.

References

Bikakis, N, Tsinaraki, C, Gioldasis, N, Stavrakantonakis, I and Christodoulakis, S. 2013. The XML and semantic web worlds: technologies, interoperability and integration: a survey of the state of the art. In: Semantic Hyper/Multimedia Adaptation. Springer, pp. 319–360. DOI: 10.1007/978-3-642-28977-4_12
Open DOI Search in Google Scholar Back to article
Brank, J, Grobelnik, M and Mladenic, D. 2005. A survey of ontology evaluation techniques.
Search in Google Scholar Back to article
Brickley, D and Guha, R. 2014. RDF Schema 1.1. W3C Recommendation. Available at: https://www.w3.org/TR/rdf-schema/ (Accessed on April 17, 2018).
Search in Google Scholar Back to article
Brooke, J. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189: 4–7.
Search in Google Scholar Back to article
Burton-Jones, A, Storey, V, Sugumaran, V and Ahluwalia, P. 2005. A semiotic metrics suite for assessing the quality of ontologies. Data & Knowledge Engineering, 55(1): 84–102. DOI: 10.1016/j.datak.2004.11.010
Open DOI Search in Google Scholar Back to article
Casellas, N. 2009. Ontology Evaluation through Usability Measures. In: Meersman, R, Herrero, P and Dillon, T. (Eds.) On the Move to Meaningful Internet Systems: OTM 2009 Workshops, LNCS, Vol. 5872. Springer, Berlin, pp. 594–603. DOI: 10.1007/978-3-642-05290-3_73
Open DOI Search in Google Scholar Back to article
Fox, M, Barbuceanu, M and Gruninger, M. 1995. An organisation ontology for enterprise modelling: preliminary concepts for linking structure and behaviour. In: Enabling Technologies: Proceedings of the Fourth Workshop on Infrastructure for Collaborative Enterprises. IEEE, pp. 71–81. DOI: 10.1109/ENABL.1995.484550
Open DOI Search in Google Scholar Back to article
Fox, P and Lynnes, C. 2015. Additional items for ontology evaluation. Available at: http://tw.rpi.edu/web/project/SeSF/workinggroups/OntologyEvaluation (Accessed on Sept. 9, 2015).
Search in Google Scholar Back to article
Gangemi, A, Catenacci, C, Ciaramita, M and Lehmann, J. 2006. Qood grid: A metaontology-based framework for ontology evaluation and selection. In: Proceedings of the EON2006 Workshop.
Search in Google Scholar Back to article
Gomez-Perez, A. 2001. Evaluation of ontologies. International Journal of intelligent systems 16(3): 391–409. DOI: 10.1002/1098-111X(200103)16:3<;391::AID-INT1014>3.0.CO;2-2
Open DOI Search in Google Scholar Back to article
Gruber, T. 1995. Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human-Computer Studies 43(5–6): 907–928. DOI: 10.1006/ijhc.1995.1081
Open DOI Search in Google Scholar Back to article
Gruninger, M and Fox, M. 1995. Methodology for the design and evaluation of ontologies. Workshop on Basic Ontological Issues in Knowledge Sharing, IJCAI-95, Montreal, Canada. 10pp.
Search in Google Scholar Back to article
Hartmann, J, Spyns, P, Giboin, A, Maynard, D, Cuel, R, Suarez-Figueroa, M and Sure, Y. 2005. D1. 2.3 methods for ontology evaluation. EU-IST Network of Excellence (NoE) IST-2004-507482 KWEB Deliverable D, 1.
Search in Google Scholar Back to article
ISO. 1998. ISO 9241-11 Ergonomic requirements for office work with visual display terminals (VDT)s – Part 11 Guidance on usability. International Organization for Standardization.
Search in Google Scholar Back to article
Jokela, T, Iivari, N, Matero, J and Karukka, M. 2003. The standard of user-centered design and the standard definition of usability. In: CLIHC ’03 Proceedings of the Latin American conference on Human-computer interaction, pp. 53–60. DOI: 10.1145/944519.944525
Open DOI Search in Google Scholar Back to article
Lozano-Tello, A and Gomez-Perez, A. 2004. Ontometric: A method to choose the appropriate ontology. Journal of Database Management 2(15): 1–18. DOI: 10.4018/jdm.2004040101
Open DOI Search in Google Scholar Back to article
Lozano-Tello, A, Gomez-Perez, A and Sosa, E. 2003. Selection of ontologies for the semantic web. In: Web Engineering. Springer, pp. 413–416. DOI: 10.1007/3-540-45068-8_77
Open DOI Search in Google Scholar Back to article
Ma, X and Fox, P. 2013. Recent progress on geologic time ontologies and considerations for future works. Earth Science Informatics 6(1): 31–46. DOI: 10.1007/s12145-013-0110-x
Open DOI Search in Google Scholar Back to article
McGuinness, D and Van Harmelen, F. 2004. Owl web ontology language overview. W3C recommendation Available at: https://www.w3.org/TR/owl-features/ (Accessed on April 17, 2018).
Search in Google Scholar Back to article
Miller, E. 1998. An introduction to the resource description framework. Bulletin of the American Society for Information Science and Technology 25(1): 15–19. DOI: 10.1002/bult.105
Open DOI Search in Google Scholar Back to article
Morris, C. 1938. Foundations of the Theory of Signs. University of Chicago Press. Chicago, IL.
Search in Google Scholar Back to article
Pratchett, T. 1990. Moving Pictures. Gollancz, London.
Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.5334/dsj-2018-010 | Journal eISSN: 1683-1470

Journal RSS Feed

Language: English

Page range: 10 - 10

Submitted on: Feb 16, 2018

Accepted on: Apr 9, 2018

Published on: May 10, 2018

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Publication frequency: 1 issue per year

Keywords:

Ontology,

Ontology Usability,

Semiotics,

Human-centered Evaluation,

Semantic Web,

Linked Open Data

© 2018 Xiaogang Ma, Linyun Fu, Patrick West, Peter Fox, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 17 (2018): Issue 0