On 3/3/2011 12:09 PM, Amanda Vizedom wrote:
> If we want SME validation of an ontology, we need to produce a
> validation test that uses the formalized ontology to do something, and
> that something needs to have results within the SMEs area of
> understanding and practice. If we want the ontology to be multipurpose
> and reusable, we should have a variety of such tests. Rather than
> relying on yet another semi-intelligible visualization of the ontology,
> and SME feedback on whether this seems right, we should incorporate
> functional testing into the ontology development and validation workflow. (01)
I very strongly agree with that point. (02)
For an example of a tool that we are currently developing at VivoMind,
see slides 26 to 30 of the following presentation: (03)
http://www.jfsowa.com/talks/futures.pdf (04)
Slides 26 to 28 describe the proto-ontology extractor, which works with
a SME to develop an ontology from a set of documents about some subject. (05)
The SMEs get immediate feedback from it by seeing that their changes
to the ontology produce improved results: more accurate answers to
their questions about the documents, and improved precision and recall
of the desired data. (06)
> Part of the SME validation should be search & retrieval tests on the corpus,
> to see whether the ontology creates results that reflect the SME knowledge. (07)
We are still at an early stage of development. But we demonstrated
better results in comparison with other tools that the SMEs had used. (08)
> And a bit a time and effort need to go into developing these up front.
> IME that up-front time and effort is not often done, and budgets don't
>include it. (09)
Speaking as a partner in a small business, we can only do what
our clients pay us to do. They usually pay for results, not for
studies that show how or whether we could produce results. (010)
John (011)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2011/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2011
Community Portal: http://ontolog.cim3.net/wiki/ (012)
|