[Top] [All Lists]

[ontology-summit] {Extrinsics} - Chat Gleanings from 24 Jan

To: Ontology Summit 2013 discussion <ontology-summit@xxxxxxxxxxxxxxxx>
From: "terry.longstreth" <terry.longstreth@xxxxxxxxxxx>
Date: Mon, 28 Jan 2013 00:23:46 -0500
Message-id: <51060B62.40102@xxxxxxxxxxx>

The Properly list from the following has been posted to the TrackB Synthesis page.    I'm sending the table to this mailing list for two reasons:

  1. I don't know how to reproduce the HTML table in the wiki
  2. to give it the widest coverage

As Todd mentioned in the chat the 24January2013 Track B session, "...We will need definitions, context, and possibly intent. But first I'd like to conduct a simple gathering exercise."

I've gone through the chat and gathered the comments that I believe represent possible properties or characteristics for evaluating an ontology extrinsically.  I've tried to show provenance by association the items with quotations from the chats, but beyond that I've made to serious attempt to organize (Ontologize?) these items.
/s/Terry Longstreth

Property / Characteristic

Full Text

JackRing: Reusefulness of an ontology or subset(s) thereof?

JackRing: In systems think the three basic dimensions are Quality, Parsimony, Beauty


MariCarmenSuarezFigueroa: In the legal part, maybe we should consider also license (and not only copyright)
costs (development, maintenance), Benefits,

MatthewWest: Relevance, Clarity, Consistency, Accessibility, timeliness,completeness, accuracy, costs (development, maintenance), Benefits

MatthewWest: Provenance

MatthewWest: Modularity

JackRing: No one has mentioned the dimension of complexness. Because ontologies quickly become complex topologies then the response time becomes very important if implemented on a von Neumann architecture. Therefore the structure of the ontology for efficiency of response becomes an important dimension


BobbinTeegarden: At DEC, we used an overlay on all engineering for RAMPSS -- Reliability, Availability, Maintainability, Performance, Scalability, and Security. Maybe these all apply for black box here? Mary has cited some of them...
domain integrity,
referential integrity,
semantic integrity,
    With_Respect_To(domain D, requirement R)

LeoObrst6: @MaryBalboni: re: slide 14: back in the day, we would characterize 3 kinds of integrity: 1) domain integrity (think value domains in a column, i.e., char, int, etc.), 2) referential integrity (key relationships: primary/foreign), 3) semantic integrity (now called "business rules"). Ontologies do have these issues. On the ontology side, they can be handled slightly differently: e.g., referential integrity (really mostly structural integrity) will be handled differently based on Open World Assumption (e.g., in OWL) or Closed World Assumption (e.g., in Prolog), with the latter being enforced in general by integrity constraints.
@Todd: your second set of slides, re: slide 4: Precision, Recall, Coverage, Correctness and perhaps others will also be important for Track A Intrinsic Aspects of Ontology Evaluation. Perhaps your metrics will be: Precision With_Respect_To(domain D, requirement R), etc.? Just a thought.
LeoObrst6: Perhaps the main difference between Intrinsic -> Extrinsic is that at least some of the Intrinsic predicates are also Extrinsic predicates with additional arguments, e.g., Domain, Requirement, etc.? 

BobbinTeegarden: @JackRing Would 'effectiveness' fall under beauty? What criteria?
@Bobbin, Effect-iveness is a Quality factor. Beauty is in the eye of the beer-holder
Requirements Satisfaction

MariCarmenSuarezFigueroa: We could also consider the verification of requirements (competency questions) using e.g. SPARQL queries.

TillMossakowski: further dimensions: consistency; correctness w.r.t. intended models (as in Megan's talk), completeness in the sense of having intended logical consequences

BobbinTeegarden: It seems we have covered correctness, precision, meeting requirements, etc well, but have we really addressed 'goodness' of an ontology? And certainly haven't addressed an 'elegant' ontology, or do we care? Is this akin to Jack's 'beauty' assessment?

BobSchloss: Because of the analogy we heard with Database Security Blackbox Assessment, I wonder if there is an analogy to "normalization" (nth normal form) for database schemas. Is some evaluation criteria related to factoring, simplicity, minimalism, straightforwardness.....
Update Impedance/ complexity/ cost
Degree of stability
Error Discovery Profile

TorstenHahmann: another requirement that I think hasn't been mentioned yet: granularity (level of detail)
BobSchloss: I am also thinking about issues of granularity and regularity ... If a program wants to remove one instance "entity" from a knowledge base, does this ontology make it very simple to just do the remove/delete, or is it so interconnected that removal requires a much more complicated syntax....
LeoObrst6: @Torsten: yes, that was my question, i.e., granularity.
MariCarmenSuarezFigueroa: I'm also think granularity is a very important dimension....
BobSchloss: Although this is driven by the domain, some indication of an ontology's rate of evolution or degree of stability or expected rate of change may be important to those using organizations. If there are 2 ontologies, and one, by being very simple and universal, doesn't have as many specifics but will be stable for decades; whereas another, because it is very detailed using concepts that are related to current technologies, current business practices, and therefore may need to be updated every year or two... I'd like to know this.
BobSchloss: Another analogy to the world of blackbox testing... the software engineers have ideas of Orthogoal Defect Classification and more generally, ways of estimating how many remaining bugs there are in some software based on the rates and kinds of discovery of new bugs that have happened over time up until the present moment. I wonder if there is something for an ontology... one that has a constant level of utilization, but which is having a decrease in reporting of errors.... can we guess how many other errors remain in the ontology? Again... this is an analogy.... some way of estimating "quality"...
MatthewWest: Yes, stability is an important criteria. For me that is about how much the existing ontology needs to change when you need to make an addition.


Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/   
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013  
Community Portal: http://ontolog.cim3.net/wiki/     (01)
<Prev in Thread] Current Thread [Next in Thread>
  • [ontology-summit] {Extrinsics} - Chat Gleanings from 24 Jan, terry.longstreth <=