OntologySummit2014_Hackathon - Project:    (48H4)

Post-event updates    (4BM1)

Short report, also provided to Summit community on 2014-04-01 by discussion list    (4BM2)

Ontological Catalog became VOCREF: Vocabulary and Ontology Characteristics Related to Evaluation of Fitness.    (4BM3)

On Saturday, 29 March, Core team consisting of AmandaVizedom, AndreaWesterinen, SimonSpero (all U.S., ETD) worked a very full day, in excess of 12 hours. MiezaHamka (Johor, Malaysia) and LamarHenderson (Maryland, USA) each observed and chatted for periods of time. TrishWhetzel (CA, USA) had intended to participate, but was unable to make it due to family emergency. Both AnatolyLevunchuk and DanBrickley joined for periods of time, Dan weighed in substantially on some issues. All three core team members returned for substantial periods of work on Sunday.    (4BM4)

Highlights of activities and accomplishments:    (4BM5)

An enormous amount of work got done populating and refining vocref-top. Many representations and concepts related to ontologies and evaluation, from a variety of sources, were considered and weeded out (as, for example, out-of-scope or specific to the assumptions and focus or their original context, or of dubious quality in one way or another). As a result, we arrived at a satisfyingly solid vocref-top, where work will continue.    (4BMA)

A good initial dent was also made in the review of candidate ontologies, vocabularies, and concepts from evaluation tools and methods for whole or partial reuse. Some were filtered out, some were partially integrated. We gathered a good stack of such resources to evaluate; and they are stored in the repository accordingly.    (4BMB)

One issue that was tagged for completion during the hackathon remains incomplete. This consists of adding sufficient annotations to concepts that were reused from the 2013 "ontology of ontology evaluation" output. This task is expected to be completed within the next week or so. An additional batch of three issues were tagged for completion by the end of the Ontology Summit. Eight issues have not been tied to any schedule, though several of these are still undergoing work and may be closable soon. A number of additional points and to-do items were noted in discussion and will be added.    (4BMD)

Some issues could not be worked on simultaneously simply because of the lack of sufficient automated merge capabilities when working on the same module (vocref-top). We did need to take turns with substantial changes to this ontology.    (4BME)

Overall, the hangout was incredibly productive. The collaboration time enabled the team to sort through considerable material effectively, consulting one another and bouncing ideas regularly. Pending a bit more documentation for newcomers should be provided (one of the near-term issues), the base now established is good and can support ongoing, asynchronous development of vocref as an open source resource.    (4BMF)

Best Regards, AmandaVizedom on behalf of the "Ontological Catalog" / VOCREF team.    (4BMG)

Post-event commentary on relation of VOCREF hackathon to OntologySummit2014 themes and developing synthesis    (4BMH)

...coming    (4BMI)

An ontological catalogue of ontology and metadata vocabulary characteristics relevant to suitability for semantic web and big data applications    (48H5)

Team    (49QQ)

Additional team members are welcome and appreciated. Direct ontology development from gathered resources will be the a primary activity. We would also welcome team members willing and able to use those resources (and/or own experience) to identify and document competency questions for this ontology. What questions and answers should it support about particular ontologies and vocabularies? Another valuable contribution would be the creation of examples in which the ontology developed so far is applied to one or more specific ontologies or vocabularies, to illustrate the use of the ontology and find coverage caps, reporting these as issues. If you would like to contribute in another way not mentioned, please contact us!    (4AWX)

Goals    (49QV)

The following are the goals of this hackathon:    (49QW)

Activities    (49R0)

During this hackathon, we will:    (49R1)

Motivation    (49R6)

In selecting, creating, or effectively using an ontology or metadata vocabulary, it is important to understand not only the requirements of the intended use, but the characteristics of any candidate ontology or vocabulary.    (49R7)

However, understanding the characteristics of a given ontology or vocabulary is made more difficult by the following:    (49R8)

Resources    (49RE)

In the GitHub repository for this project, in the directory References and Resources > Papers and Specs, are a growing sample of documents providing or describing systems for ontology evaluation. Each of these provides a more or less explicit listing of ontology characteristics for evaluation.    (4AWY)

There is less overlap than one might expect; while some expressions recur ("usability" or "accuracy," for example), discussion generally reveals that the underlying concepts are different, at least at the measurable / evaluable level.    (4AWZ)

It is a goal of this project to develop a vocabulary sufficient for expressing the ontology characteristics listed and described in these documents.    (4AX0)

In particular, some of these documents describe evaluation methods that have some implemented tools and/or user base. The ontology characteristics described in such documents should be given higher priority than those which may be more speculative. Insofar as there is more likely to be evaluation data based on these in-use characteristics, their coverage in VOCREF is most likely to be useful, or at least to have its usefulness put to the test.    (4AX1)

We've also collected some ontologies and vocabularies that cover some portion of the relevant metadata. We'll evaluate them for reuse, likely incorporating some and aligning to others.    (4AX2)

We will be reusing a portion of the draft work done during last year's "Ontology of Ontology Evaluation" Hackathon    (49RO)