Whether you want to do validation or not you still need a clear statement of
purpose, intended usage (context and effects on context), measures of
effect-iveness and standards of acceptance (of the ontology).
Measures of Effectiveness shall include as a minimum, Likelihood of False
Positives and False Negatives.
Jack
On Mar 3, 2011, at 12:32 PM, John F. Sowa wrote: (01)
> On 3/3/2011 12:09 PM, Amanda Vizedom wrote:
>> If we want SME validation of an ontology, we need to produce a
>> validation test that uses the formalized ontology to do something, and
>> that something needs to have results within the SMEs area of
>> understanding and practice. If we want the ontology to be multipurpose
>> and reusable, we should have a variety of such tests. Rather than
>> relying on yet another semi-intelligible visualization of the ontology,
>> and SME feedback on whether this seems right, we should incorporate
>> functional testing into the ontology development and validation workflow.
>
> I very strongly agree with that point.
>
> For an example of a tool that we are currently developing at VivoMind,
> see slides 26 to 30 of the following presentation:
>
> http://www.jfsowa.com/talks/futures.pdf
>
> Slides 26 to 28 describe the proto-ontology extractor, which works with
> a SME to develop an ontology from a set of documents about some subject.
>
> The SMEs get immediate feedback from it by seeing that their changes
> to the ontology produce improved results: more accurate answers to
> their questions about the documents, and improved precision and recall
> of the desired data.
>
>> Part of the SME validation should be search & retrieval tests on the corpus,
>> to see whether the ontology creates results that reflect the SME knowledge.
>
> We are still at an early stage of development. But we demonstrated
> better results in comparison with other tools that the SMEs had used.
>
>> And a bit a time and effort need to go into developing these up front.
>> IME that up-front time and effort is not often done, and budgets don't
>include it.
>
> Speaking as a partner in a small business, we can only do what
> our clients pay us to do. They usually pay for results, not for
> studies that show how or whether we could produce results.
>
> John
>
>
>
>
> _________________________________________________________________
> Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
> Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
> Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2011/
> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2011
> Community Portal: http://ontolog.cim3.net/wiki/ (02)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2011/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2011
Community Portal: http://ontolog.cim3.net/wiki/ (03)
|