Denise and Ken, (01)
DB> How can the evaluation and registry parameters be more
> closely aligned? (02)
The only things that can be evaluated on a linear scale are ones
that are already expressed in numbers. But even those can cause
serious misunderstandings -- for example, comparing miles per
gallon of a sports car, a lawn mower, and a snow plow. (03)
I realize that managers constantly ask for some kind of magic
number, but giving them numbers is often irresponsible. The
greatest danger is that they might actually make a decision
based on those numbers. (04)
KB> With one of my students, I have evaluated 40 large ontologies
> using the OntologySummit2007 Framework Dimensions. I have
> written up some conclusions about the results and could present
> them at some point.
>
> I have posted it to
>
> http://www.ccs.neu.edu/home/kenb/OntologySummit2007Assessment (05)
The "radar diagrams" demonstrate the problems with that kind of
factor analysis. For example, the diagram for WordNet and the
diagram for Human Disease - OBO are almost identical. Yet those
two systems are totally different in purpose, representation, and
methodology. But the web page makes the following claim: (06)
> The factor analysis is statistically significant with p-value 0.00137. (07)
That statement makes the approach sound extremely significant, but
I would consider any method that would produce almost identical
numbers for WordNet and the Human Disease OBO as hopelessly misleading. (08)
The list of 40 different ontologies/terminologies/resources (see below)
is useful. But instead of those radar diagrams (or the numbers from
which they were derived), the following info would be far more valuable: (09)
1. A URL that points to the web site for that resource. (010)
2. A one-paragraph description derived from the documentation on
that web site. (011)
3. One sentence that describes the subject matter of the resource. (012)
4. A statement of the notation used for the resource (i.e., whether
it uses a natural language, a formalized language, or some
combination of one or more languages and/or diagrams). (013)
5. Some measure of size in terms of the number of items defined
(i.e., categories, concepts, words, terms, predicates, relations,
types, classes, or whatevers). (014)
If I were looking for an ontology, I would find the above information
useful as a *starting point*. Those radar diagrams are misguided
because they give the impression that there is some kind useful
evaluation embodied in them. (015)
John Sowa
_____________________________________________________________________ (016)
1. MeSH
2. Gene Ontology
3. Basic Formal Ontology
4. SUMO
5. Engineering Math
6. COSMO
7. PSL
8. Environmental Data Coding Specification
9. Geospatial ML
10. International Classification of Diseases
11. Unified Medical Language System Metathesaurus
12. World Bank MetaThesaurus
13. Spase Ontology
14. FOAF
15. AKTiveSAOntology
16. CMMI Smallscale Ontology
17. Beer Ontology
18. BioMoby Services
19. Bio-Zen
20. UNSPCS
21. IMDB Mapping Movie Ontology
22. Federal Enterprise Architecture Reference Model Ontology (FEA-RMO)
23. Cyc
24. Encyclopedia of Life
25. Ontology for Biomedical Investigation
26. Functional Genomics Investigation Ontology
27. ProMed Mail
28. Agricola
29. GenBank
30. Medline
31. GBIF
32. WHO health reports
33. Human Disease - OBO
34. Core Software Ontology
35. Cell System Ontology
36. CIA Factbook
37. Evidence Code
38. WordNet
39. Ontology for Geographic Information ISO 19115:2003
40. General Ontology for Linguistic Description (017)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2008/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2008
Community Portal: http://ontolog.cim3.net/ (018)
|