OntologySummit2013: Panel Session-07 - Thu 2013-02-28    (3LH7)

Summit Theme: "Ontology Evaluation Across the Ontology Lifecycle"    (3NHU)

Summit Track Title: Track-B: Extrinsic Aspects of Ontology Evaluation    (3NHV)

Session Topic: "Extrinsic Aspects of Ontology Evaluation-II"    (3NHN)

Panelists / Briefings:    (3NHW)

Archives:    (3NIL)

Abstract:    (3NKE)

OntologySummit2013 Session-07: "Extrinsic Aspects of Ontology Evaluation-II"    (3NKF)

This is our 8th Ontology Summit, a joint initiative by NIST, Ontolog, NCOR, NCBO, IAOA & NCO_NITRD with the support of our co-sponsors. The theme adopted for this Ontology Summit is: "Ontology Evaluation Across the Ontology Lifecycle."    (3NKG)

Currently, there is no agreed methodology for development of ontologies, and there are no universally agreed metrics for ontology evaluation. At the same time, everybody agrees that there are a lot of badly engineered ontologies out there, thus people use -- at least implicitly -- some criteria for the evaluation of ontologies.    (3NKH)

During this OntologySummit, we seek to identify best practices for ontology development and evaluation. We will consider the entire lifecycle of an ontology -- from requirements gathering and analysis, through to design and implementation. In this endeavor, the Summit will seek collaboration with the software engineering and knowledge acquisition communities. Research in these fields has led to several mature models for the software lifecycle and the design of knowledge-based systems, and we expect that fruitful interaction among all participants will lead to a consensus for a methodology within ontological engineering. Following earlier Ontology Summit practice, the synthesized results of this season's discourse will be published as a Communique.    (3NKI)

At the Launch Event on 17 Jan 2013, the organizing team provided an overview of the program, and how we will be framing the discourse around the theme of of this OntologySummit. Today's session is one of the events planned.    (3NKJ)

In this 7th virtual panel session of the Summit, we will invite several experts, who have been giving a lot of thoughts on this subject matter over the years, to join us on the panel and share their insights.    (3NKK)

More details about this OntologySummit is available at: OntologySummit2013 (homepage for this summit)    (3NKL)

Briefings:    (3NKM)

Abstract: We address a summit omission to date with a challenge. ''    (3NKN)

Abstract: This talk will present our approach to facilitate the validation of OntoUML models by transforming these into formal specifications in the logic-based language Alloy and using its analyzer to generate and visualize instances of the model. By allowing the observation of sequences of snapshots of model instances, the dynamics of entity creation, classification, association and destruction are revealed. This confronts the modelers with the implications of modeling choices and allows them to uncover mistakes or gain confidence in the quality of conceptual models. We are specifically interested in assessing the correspondence of what is stated in an OntoUML model and what was the original intention of the modeler or domain expert.    (3NWE)

Abstract: ... This presentation will provide some insight into the role of performance engineering in the system test process. Performance testing of traditional computer systems will be reviewed. Thoughts about performance testing of ‘black boxes’ are provided as well as some concluding observations.    (3NKQ)

Abstract:    (3NWG)

This presentation applies some general software engineering discipline to ontology projects and to technology projects that incorporate ontologies. The result is primarily a process description, with an example process walk-through. The process description focuses on the ontology-specific aspects. However, since the intended audience (ontology & semantic technology developers, project planners and managers) may have minimal familiarity with some of the general concepts borrowed from software engineering and technical project management, some additional explanation is provided where useful for this audiences’ understanding.    (3NWI)

Ontology projects, and semantic technology projects in general, do not typically happen for their own sakes. They happen because someone has an operational problem, a business problem, and someone thinks that some semantic technology, incorporating but typically not limited to one or more ontologies, can contribute enough to a solution to make them worthwhile. Even ontological research, not tied to specific instances of business problems, is normally motivated by (and its funding justified with reference to) some operational solutions that could be enabled or improved by the research results. There is, however, currently a significant knowledge gap concerning the relationship between business needs and the technical characteristics of ontologies and ontology-based solutions. There is also a gap in the perception of ontologies as technical artifacts, of ontology design and development as technical processes. As a result, good engineering practice are often not applied to ontology projects, or semantic technology projects in which ontologies are incorporated.    (3NWH)

In practice, semantic technology projects too often lack sufficient technical specification and direction to have a good chance at providing the desired solutions. Many projects that include development and/or incorporation of ontologies go forward without developing, or even thinking about, technical requirements for those ontologies. As a consequence, development of ontologies for these projects, or selection of existing ontologies for use in these projects, often proceeds without any meaningful guidelines. Ontology developers and selectors, depending on their level of experience, may have some notions of what are minimal quality (suitability) requirements in general, but they lack a set of well-grounded requirements for ontologies suitable for their own project specifically. Correspondingly, Ontology Evaluation is often omitted entirely; when performed, its value is limited by the lack of ontology requirements with respect to which ontologies can be evaluated. This presentation outlines a process for filling these gaps, providing technical requirements and direction to a project, and keeping ontology work on track by evaluating ontologies against the requirements so identified.    (3NWF)

Agenda:    (3NKS)

OntologySummit2013 - Panel Session-07    (3NKT)

Proceedings:    (3NKZ)

Please refer to the above    (3NL0)

IM Chat Transcript captured during the session:    (3NL1)

 see raw transcript here.    (3NL2)
 (for better clarity, the version below is a re-organized and lightly edited chat-transcript.)
 Participants are welcome to make light edits to their own contributions as they see fit.    (3NL3)
 -- begin in-session chat-transcript --    (3NL4)
 -- end of in-session chat-transcript --    (3NL5)

Additional Resources:    (3NLC)

For the record ...    (3NLJ)

How To Join (while the session is in progress)    (3NLK)

Conference Call Details    (3NIU)

Attendees    (3NJR)