[Top] [All Lists]

Re: [ontology-summit] [ont-of-ont] evaluated 40 large ontologies using t

To: Ontology Summit 2008 <ontology-summit@xxxxxxxxxxxxxxxx>
From: "John F. Sowa" <sowa@xxxxxxxxxxx>
Date: Tue, 15 Apr 2008 13:22:57 -0400
Message-id: <4804E471.1010205@xxxxxxxxxxx>
Denise and Ken,    (01)

DB> How can the evaluation and registry parameters be more
 > closely aligned?    (02)

The only things that can be evaluated on a linear scale are ones
that are already expressed in numbers.  But even those can cause
serious misunderstandings -- for example, comparing miles per
gallon of a sports car, a lawn mower, and a snow plow.    (03)

I realize that managers constantly ask for some kind of magic
number, but giving them numbers is often irresponsible.  The
greatest danger is that they might actually make a decision
based on those numbers.    (04)

KB> With one of my students, I have evaluated 40 large ontologies
 > using the OntologySummit2007 Framework Dimensions.  I have
 > written up some conclusions about the results and could present
 > them at some point.
 > I have posted it to
 > http://www.ccs.neu.edu/home/kenb/OntologySummit2007Assessment    (05)

The "radar diagrams" demonstrate the problems with that kind of
factor analysis.  For example, the diagram for WordNet and the
diagram for Human Disease - OBO are almost identical.   Yet those
two systems are totally different in purpose, representation, and
methodology.  But the web page makes the following claim:    (06)

 > The factor analysis is statistically significant with p-value 0.00137.    (07)

That statement makes the approach sound extremely significant, but
I would consider any method that would produce almost identical
numbers for WordNet and the Human Disease OBO as hopelessly misleading.    (08)

The list of 40 different ontologies/terminologies/resources (see below)
is useful.  But instead of those radar diagrams (or the numbers from
which they were derived), the following info would be far more valuable:    (09)

  1. A URL that points to the web site for that resource.    (010)

  2. A one-paragraph description derived from the documentation on
     that web site.    (011)

  3. One sentence that describes the subject matter of the resource.    (012)

  4. A statement of the notation used for the resource (i.e., whether
     it uses a natural language, a formalized language, or some
     combination of one or more languages and/or diagrams).    (013)

  5. Some measure of size in terms of the number of items defined
     (i.e., categories, concepts, words, terms, predicates, relations,
     types, classes, or whatevers).    (014)

If I were looking for an ontology, I would find the above information
useful as a *starting point*.  Those radar diagrams are misguided
because they give the impression that there is some kind useful
evaluation embodied in them.    (015)

John Sowa
_____________________________________________________________________    (016)

    1. MeSH
    2. Gene Ontology
    3. Basic Formal Ontology
    4. SUMO
    5. Engineering Math
    6. COSMO
    7. PSL
    8. Environmental Data Coding Specification
    9. Geospatial ML
   10. International Classification of Diseases
   11. Unified Medical Language System Metathesaurus
   12. World Bank MetaThesaurus
   13. Spase Ontology
   14. FOAF
   15. AKTiveSAOntology
   16. CMMI Smallscale Ontology
   17. Beer Ontology
   18. BioMoby Services
   19. Bio-Zen
   20. UNSPCS
   21. IMDB Mapping Movie Ontology
   22. Federal Enterprise Architecture Reference Model Ontology (FEA-RMO)
   23. Cyc
   24. Encyclopedia of Life
   25. Ontology for Biomedical Investigation
   26. Functional Genomics Investigation Ontology
   27. ProMed Mail
   28. Agricola
   29. GenBank
   30. Medline
   31. GBIF
   32. WHO health reports
   33. Human Disease - OBO
   34. Core Software Ontology
   35. Cell System Ontology
   36. CIA Factbook
   37. Evidence Code
   38. WordNet
   39. Ontology for Geographic Information ISO 19115:2003
   40. General Ontology for Linguistic Description    (017)

Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/ 
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008 
Community Portal: http://ontolog.cim3.net/    (018)
<Prev in Thread] Current Thread [Next in Thread>