Re: [ontology-summit] {hackathons-clinics} proposal: ontology support fo

[Amanda Vizedom <amanda.vizedom@xxxxxxxxx>]

All,

I'm re-sending this to the summit group, in light of some comments made during last week's summit session. Several people expressed interest in creating an ontology of the ontology evaluation domain, especially capturing some of the ideas that have been presented, refined, found useful and gained some support. This is in fact the very activity  proposed in the "Ontology Support for Ontology Evaluation" activity.  So I suspect that the proposal was missed by at least some folks in the storm of those that went by a couple of weeks ago.

Bottom line: if you are interested in an ontology of/for ontology evaluation, check out proposal A2. / H8. Ontology Support for Ontology Evaluation at http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013_Hackathon_Clinics#nid3O90

Summary is there, full proposal is linked just below title. Call for collaborators is open and appears just below the abstract. Please add your name if you're interested and able to contibrute!

Best,

Amanda

On Fri, Mar 8, 2013 at 5:20 PM, Amanda Vizedom <amanda.vizedom@xxxxxxxxx> wrote:

The following has been submitted as an application clinic, in keeping with prior suggestion; it has application clinic elements as it involves development and evaluation of an ontology with particular uses and requirements in mind. However, it might also be regarded as a hackathon, since there will be primary development involved. In this regard, it is like what is sometimes called a vocamp or ontocamp. In any case, here it is, for your consideration: 

APPLICATION CLINIC -- ONTOLOGY SUPPORT FOR ONTOLOGY EVALUATION

AMANDA VIZEDOM (INDEPENDENT CONSULTANT)

ABSTRACT:

The 2013 Ontology Summit is bringing together a rich range of perspectives on ontology evaluation. Some of these perspectives are fairly abstract, some are encoded in methods and practices, and some are encoded in tools. Critical interaction via summit sessions and discussion has resulted in greater sharing of knowledge and in richer understandings of ontology evaluation at multiple elements. This enrichment is likely to be apparent in the activities and future products of summit participants. Another manifestation of this enrichment will be the summit Communique. The goal of this “hackathons & clinics” activity is to add another manifestation: a formal ontology representing ontology evaluation elements, factors, relationships, processes, etc., as they have emerged from summit discussion.

The scope of this project is staged to increase with available resources or as time permits, beginning with representation of the most important general concepts and their relationships to one another, through representation of specific metrics, tools, ontology characteristics, evaluation processes, and multi-faceted organizations thereof. Scope and prioritization will be requirements-driven, based on two or more rough use cases developed ahead of time. These use cases will include, at minimum, one direct human-use scenario and one system-incorporation scenario, such as (for example): (a) a human user, using the ontology in the process of determining how well-suited a particular ontology is for a particular application; and (b) a repository environment incorporating the ontology into a feature set meant to enable matching of use cases and well-suited ontologies, through stated or inferred requirements and performed or recorded evaluations.

COLLABORATORS SOUGHT:

ONTOLOGY DEVELOPERS: 

The feasibility of this project as a 1-2 day ontology sprint depends on having at least a handful of folks to do the representation. The plan, pending consultation with collaborators, is to make major design decisions in collaboration, and to modularize and distribute chunks of KR throughout the team, working in parallel with open communication channels during the sprint.

POTENTIAL USERS: 

If you can see how you might use the resulting ontology, and have some real (albeit potential and result-dependent) interest in doing so, your use case would be most welcome. Real use cases, and real potential deployment, will help focus and energize this ontology development sprint.

DELIVERABLES AND OBJECTIVES:

The primary deliverable of this project is a formal ontology of ontology evaluation concepts and their relationships to one another.

The core work of this clinic will be the rapid development of an ontology of concepts important to ontology evaluation. This ontology will include such concepts as: formal ontologies; collections of ontologies; characteristics of ontologies; specific ontology applications and use cases; ontology requirements; possible relationships between ontology requirements and ontology characteristics; ontology metrics; possible relationships between ontology metrics and ontology characteristics; measurement methods and possible relationships between methods and ontology metrics; evaluation events and their methods used, metrics used, results; possible relationships between ontology requirements and ontology evaluation results. To scope this work and ground it, we will also seek to use these general concepts to represent some of approaches presented during the ontology summit (see Objectives for further detail), essentially seeding a knowledge base of such approaches.

It is the objective of this clinic that this ontology be itself generally useful; therefore we will aim to satisfy a number of potential use cases:

-- The concepts above will be modeled so as to be suitable for application in representation of the characteristics of particular ontologies, requirements of particular use cases, and characteristics and results of particular ontology evaluations.

-- The model will be suitable for use in reasoning about an ontology’s status with relationship to some set of requirements.

-- The model will be usable to represent two or more of the evaluation approaches, and corresponding analysis of evaluated ontologies.

-- The model will be extension-friendly, supporting addition of ontology characteristics, requirements, metrics, methods, etc.; it will also support a variety of ways of grouping or faceting these (e.g., intrinsic/extrinsic/relational; syntactic/semantic/modeling;…)

-- The model will be application friendly, supporting use in repository, development, or other environments.

Over the course of a single weekend, the result of this effort will of course be partial. However, it is the objective of this clinic that the result will be a formal ontology that can stand on its own, be evaluated against its own requirements (as noted above), be easily extended, and be sufficient to serves as a formal capture of some of the knowledge gained during the summit.

REMARKS:

In accordance with the number and experience level of ontology development collaborators participating, we will seek to build out as much of a knowledge base of ontology evaluation methods, metrics, tools, etc., as can be reasonably fit into the clinic timeline, leaving time for quality assurance on the model itself.

Beyond expressiveness and availability of development and reasoning tools, representation language will likely be chosen based on the skills of available collaborators. If some developers/providers of particular repository or evaluation tools express significant interest in potential use of the resulting formal ontology, this also may influence choice of representation language.

Distributed, modular development of ontologies is still supported by relatively few tools; this will likely be an issue, since we are seeking to engage in distributed work over a short time period. Openly available tools for remote communication will be used generously.

RESOURCES:

In one sense, all presented materials and references from Ontology Summit 2013 will serves as references.

In another sense, we will use as more direct references a subset of those materials that include explicit lists and descriptions of ontology evaluation elements, methods, tools, and so forth. The subset will be gathered as a sub-collection in the Ontology Summit 2013 Group Library on Zotero: https://www.zotero.org/groups/ontologysummit2013/items

Best,

Amanda

_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/   
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013  
Community Portal: http://ontolog.cim3.net/wiki/     (01)