Re: [ontology-summit] An example of the worth of ontology development

[Amanda Vizedom <amanda.vizedom@xxxxxxxxx>]

This is a prime reason for keeping ontology development situated within an application context (or more than one).  In my dozen+ years of working with SMEs, I have seen SMEs approve, reject, spit on, throw things at, celebrate, praise, and sometimes even understand, at some level, ontologies built on the basis of their input. None of these reactions is predictive of whether the ontology actually reflects the knowledge they and their supporting ontologists are attempting to capture, or whether it will function as intended for some tasks.

If we want SME validation of an ontology, we need to produce a validation test that uses the formalized ontology to do something, and that something needs to have results within the SMEs area of understanding and practice. If we want the ontology to be multipurpose and reusable, we should have a variety of such tests. Rather than relying on yet another semi-intelligible visualization of the ontology, and SME feedback on whether this seems right, we should incorporate functional testing into the ontology development and validation workflow. For example, if one way an ontology will be used is to create metadata / support semantic indexing and retrieval, a test corpus of SME domain documents should be IDed at the outset of development, and candidate ontologies should be used to create metadata for that corpus. Part of the SME validation should be search & retrieval tests on the corpus, to see whether the ontology creates results that reflect the SME knowledge.  

Yes, that means that the development environment needs to incorporate testing applications, possibly versions of the intended end-use applications. And thought needs to be put into the appropriateness of different tests. And a bit a time and effort need to go into developing these up front. IME that up-front time and effort is not often done, and budgets don't include it. That contributes strongly to a QA weakness in many projects, and lets some go for years, on parallel ontology development and application development tracks, without adequate information to know whether their methods will lead them to their goals. 

Not that I feel strongly about this point, or anything. ;-)

Amanda

On Wed, Mar 2, 2011 at 12:10, John F. Sowa <sowa@xxxxxxxxxxx> wrote:

Jack and Mike,

I agree with that point, but I'd like to add some qualifications:

JR

> The primary purpose of a semantic model is to facilitate knowledge
> exchange and choice making in a gaggle of humans in hopes of
> morphing the gaggle into a system. A key usage is to inform the
> development of an executable ontology, e.g., application software,
> for automation of information flow and decision. Another key purpose
> is to provide a basis for objective assessment of enterprise
> situation (aka evidence-based management).

MU

> Yes, this is the kind of thing I'm after.

The primary qualification is that the "gaggle of humans" can only
agree on what they understand.  The people who work in a field
can all agree that a list of familiar words, as documented in
their familiar texts, cover their familiar subject matter.

But when ontologists start to axiomatize those terms in some
arcane notation based on some arcane distinctions about
endurants, perdurants, continuants, etc., all bets are off.

John

_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2011/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2011
Community Portal: http://ontolog.cim3.net/wiki/

_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/   
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2011/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2011  
Community Portal: http://ontolog.cim3.net/wiki/     (01)