Besides Communique's specific wording (I have not yet found the version you're
talking about), empirical evidence for the usefulness of some *methods* (maybe
not full-fledged methodologies like we find in management and heavyweight
software engineering) does exist. See e.g. [1] and [2].
In favor of a disclaimer to the existence of tantalizing methodologies for
ontology engineering, the methods I refer to are more alike guidelines to apply
design patterns and extreme programming, so quite far away from a managerial
fantasy.
Ciao
Aldo (01)
[1] http://dl.acm.org/citation.cfm?id=1597743
[2] http://link.springer.com/chapter/10.1007/978-3-642-16438-5_9#page-1 (02)
On Apr 25, 2013, at 6:58:34 AM , Pat Hayes <phayes@xxxxxxx> wrote: (03)
>
> On Apr 23, 2013, at 1:44 PM, Fabian Neuhaus wrote:
>
>> All,
>> I received an off-list reply from somebody who read my last email as a
>dismissal of ongoing work in ontology and existing practices. I exaggerated
>the current situation for humorous effects, that obviously backfired.
>>
>> So, just to make it clear: I don't believe that ontologies are pieces of
>art. Although many ontologies are developed in an ad hoc fashion, there are,
>of course, people who are applying sound methodologies in ontology development.
>
> Are there, indeed? Have these 'sound' methodologies been tested in practice,
>with comparatives between them? What empirical results are available to show
>that these methodologies have any value?
>
> Any methodology has its limitations and problems: what can be said about the
>limits of applicability of any of these methodologies, or the conditions under
>which they might be expected to fail?
>
>> The communique is based on their experiences
>
> Based on how many of them, used under what circumstances, on what projects?
>Were they all of a similar scale, or varied? Were any of them Web-based? What
>purposes did they serve? How large is the sample size represented here?
>
> It would be better to read about the actual experiences. Why not have the
>communique cite them in support of its claims, so that readers can make up
>their own minds about how "sound" all this really is?
>
> Pat
>
>
>> , we try to merge them into a big picture on how to develop ontologies.
>>
>> Best
>> Fabian
>>
>> On 4/23/13 11:42 AM, Fabian Neuhaus wrote:
>>> Pat,
>>> It is always a joy to read your emails. Even if one is the target of your
>scathing remarks. :-)
>>>
>>> Since I came up with the initial outline of the communique, I guess I am
>mainly to blame for the development of the "managerial fantasy". So, let me
>try to defend it. You call the document a phantasy, because it "is not based
>on observations about how the work is ACTUALLY done." And you are right.
>>>
>>> Here is my draft for a communique that is based on how the work is actually
>done:
>>> "In spite of a lot of literature on the subject, ontology evaluation plays
>no important role in the development of ontologies today (except for checking
>for logical consistency). The main reason for this situation is that the
>development of ontologies is currently not practiced as engineering activity,
>but as a form of art, where design decisions are more driven by schools of
>taste than by empirically validated methodologies. Since ontologies seem to be
>pieces of art, we suggest a metric that has been proven successful in the
>movie industry for a very long time: so give 2 thumps up for your favorite
>ontology!"
>>>
>>> I am not sure whether you would have endorsed this version of the
>communique, but at least it would be short. :- )
>>>
>>> Instead of describing the current situation of ontology evaluation, we are
>trying to do something more productive. Namely, to produce a consensus on how
>ontology development should be done, and which forms of evaluation should
>happen during the life cycle. So its a fantasy, all right. But some peoples
>fantasies are other peoples visions, and this vision is an attempt to merge
>the recommendations by the participants in the Ontology Summit on how to
>organize the development of an ontology. Hence, the document might not be an
>empirical description of what is happening currently, but it is grounded in
>the experience of many people who have developed ontologies.
>>>
>>> You argue that developing such a document can cause harm, because it might
>be used by managers to impose work habits on other people for no good reason.
>I agree that this is a problem. Any well-intentioned recommendation might be
>turned into a dogma that is imposed blindly. However, you ignore the
>potential benefits of the document. It is intended to provide help to people
>who are trying to build (or use) ontologies and are looking for
>recommendations on how to organize themselves by more experienced ontologists.
>>>
>>> So, I guess, it boils down to the question: Do we know enough about the
>ontology life cycle to make recommendations about development and evaluation
>that are of any value? You seem to believe that we don't, and, thus, that "to
>go on record with a detailed, confidently stated account which claims to be
>normative, is both inappropriate and harmful." I agree that there are many,
>many open questions
>>> However, this does not mean that we have no idea about how to develop
>ontologies. Thus, the open questions should not prevent us from synthesizing
>the collective experience that is available right now. We will not get
>everything right, but it will enable other people to avoid some mistakes.
>>>
>>> That's said, I see you point about the potential drawbacks. To minimize the
>risk of turning the Communique into management dogma, we should put some
>language into the executive summary and/or the introduction that state the
>limits of our existing experience explicitly.
>>>
>>> Best
>>> Fabian
>>>
>>>
>>> On 4/21/13 12:32 PM, Pat Hayes wrote:
>>>>
>>>> On Apr 21, 2013, at 5:11 AM, Amanda Vizedom wrote:
>>>>
>>>>> Pat,
>>>>>
>>>>> Will you share with us the reasons for your non-support of this
>communique?
>>>>
>>>> Hi Amanda
>>>>
>>>> Sure. (I didn't want to pester you unless you asked :-) Several reasons.
>First, it is way too long and detailed, more of an essay than a communique. It
>is hard, I would suggest, for anyone to agree with all of it unequivocally. I
>feel like I want to nit-pick with the text all over the place. For example,
>this claim that ontologies must be "transparent to all intended users" (who
>might not be ontologists) seems to me to be ridiculously optimistic (or
>perhaps ridiculously restrictive, if it is interpreted as a design constraint)
> and to be rooted in a naive idea that correctness of an ontology should be
>obvious to a domain expert. But this is simply false. Just as one would not
>expect users of a program to find the code source obvious, one cannot expect
>users of an ontology to find the details of the ontology obvious. There may
>well be decisions taken in the design, the reasons for which are only apparent
>to professional ontologists (such as whether to use continuants in t
> alking about time and change.)
>>>>
>>>> I could go through the document nitpicking like this, but it seems
>pointless, because the entire enterprise is flawed. The fact is, this whole
>document is a mangerial fantasy. We do not have enough experience with
>ontology design and deployment to know what the objective standards of
>"quality" are, still less how to manage teams to achieve this nonexistent
>standard. We don't know what are the "activities that need to occur during the
>phases of a life cycle of an ontology", so to go on record with a detailed,
>confidently stated account which claims to be normative, is both inappropriate
>and harmful. As I say, this is pure fantasy, but it will be read by some as
>having an authority and will be used by managers (most of whom know absolutely
>nothing about ontologies) to impose work habits on other people for no good
>reason. For example, "
>>>> Does the ontology follow best
>>>> practices; in particular does it implement the upper
>>>> ontology...." Whoa. Is it "best practice" to even HAVE
>>>> an upper ontology? That is not clear. Most Web
>>>> ontologies, for example, are not subsumed under any
>>>> particular upper ontology. If out communique starts
>>>> being used to justify managers asking ontologists to
>>>> conform to an upper ontology, we will have done far more
>>>> harm than good.
>>>>
>>>>
>>>> I found it very telling that after pages of vacuous managerial-theory
>babble about life cycles and "phases", most of it content-free (such as "The
>requirements development and analysis phase involves extending and clarifying
>initial information until the intended usage is sufficiently captured and
>understood to effectively guide technical decisions. This process involves an
>interplay of technical, business, and project-sponsor understanding. Adequate
>requirements development and analysis is critical to the success of any
>ontology development or usage."), and an absurd schematic diagram showing
>tangles of arrows connecting meaningless boxes, we read the almost plaintive
>remark "
>>>> Generally, appreciation of the
>>>> full life cycle of an ontology is not well established
>>>> within the ontology community." Damn right. In other
>>>> words, none of this is based in actual reality. It is
>>>> written as though it comprised observations about the
>>>> right way to work, but in fact, it is not based on
>>>> observations about how the work is ACTUALLY done.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Another nit-pick, to end. Your
>>>> second final observation begins: "
>>>> Ontology development shares
>>>> strong similarities with information systems
>>>> development..." Does it, in fact? Is this based on
>>>> actual observations? (Of which projects?) Or is this
>>>> just an idea which the authors of this document feel
>>>> *should* be true?
>>>>
>>>> Again: "
>>>> Although there is much research
>>>> on ontology evaluation and many organizations use
>>>> sophisticated ontology evaluation and quality management
>>>> practices, awareness of this research, these practices,
>>>> and their importance to successful use of ontologies is
>>>> neither widespread nor sufficiently pooled to
>>>> constituted an accessible body of knowledge." The claim
>>>> that there is "much research" seems to me to be overly
>>>> optimistic, to put it mildly; but the main point is,
>>>> there is NOT widespread adoption of these practices.
>>>> There may be very good reasons for this lack of uptake:
>>>> the practices may be of limited utility, or of no real
>>>> utility at all. In my experience, that is the most
>>>> likely explanation.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> My advice would be to completely toss
>>>> this document aside, and start over not with some
>>>> ready-made management-science theory about phases
>>>> and work cycles, but try observing, if possible
>>>> with a somewhat more humble attitude, how some
>>>> large-scale ontologies were actually built. You
>>>> might start with CYC, the granddaddy of all
>>>> large-scale ontologies. You will find that the
>>>> process bears almost no relationship to the
>>>> fantasy you describe here.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Sorry, but you did ask.
>>>>
>>>>
>>>>
>>>>
>>>> Pat
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>> I'm asking not to try to argue, but because we haven't had any input or
>feedback from you, and I value your insights generally. We are still working
>to follow through on many of the suggestion and critiques offered so far. This
>follow-through may happen by substantal change to the communique or by
>clarification of its scope and, if possible and with summit community support,
>links to better and detailed references on issues that are out of scope,
>including other summit products. So, the request is not empty; if you will let
>us know the reason(s) for your discontent, we may be able to improve the
>communique by understanding them.
>>>>>
>>>>> Best,
>>>>> Amanda
>>>>> On Apr 20, 2013 11:25 PM, "Pat Hayes" <phayes@xxxxxxx> wrote:
>>>>> Peter and Mike, greetings.
>>>>>
>>>>> On Apr 19, 2013, at 7:31 PM, Peter Yim wrote:
>>>>>
>>>>>> Dear Ontology Summit Advisors,
>>>>>>
>>>>>> ...
>>>>>> p.s. additionally, two very important reminders: *** Please Note ***
>>>>>> ...
>>>>>> 2. We are expecting to have the Communique ready by the time of the
>>>>>> Symposium (no more wordsmithing of that document at the face-to-face
>>>>>> this time) and, as advised earlier, we are expecting all Advisors to
>>>>>> endorse the Communique (on an opt-out basis.)
>>>>>
>>>>> Please opt me out of endorsing this Communique. If you would prefer, you
>may remove me from the Advisory Committee, in order to maintain an appearance
>of solidarity.
>>>>>
>>>>> Best wishes
>>>>>
>>>>> Pat Hayes
>>>>>
>>>>>
>>>>> ------------------------------------------------------------
>>>>> IHMC (850)434 8903 or (650)494 3973
>>>>> 40 South Alcaniz St. (850)202 4416 office
>>>>> Pensacola (850)202 4440 fax
>>>>> FL 32502 (850)291 0667 mobile
>>>>> phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________
>>>>> Committee Msg Archives:
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
>>>>> Subscriber Config:
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
>>>>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
>>>>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
>>>>> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
>>>>> Community Portal: http://ontolog.cim3.net/wiki/
>>>>>
>>>>> _________________________________________________________________
>>>>> Committee Msg Archives:
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
>>>>> Subscriber Config:
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
>>>>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
>>>>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
>>>>> Community Wiki:
>http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
>>>>> Community Portal: http://ontolog.cim3.net/wiki/
>>>>
>>>> ------------------------------------------------------------
>>>> IHMC (850)434 8903 or (650)494 3973
>>>> 40 South Alcaniz St. (850)202 4416 office
>>>> Pensacola (850)202 4440 fax
>>>> FL 32502 (850)291 0667 mobile
>>>> phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>> _________________________________________________________________
>> Committee Msg Archives:
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
>> Subscriber Config:
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
>> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
>> Community Portal: http://ontolog.cim3.net/wiki/
>
> ------------------------------------------------------------
> IHMC (850)434 8903 or (650)494 3973
> 40 South Alcaniz St. (850)202 4416 office
> Pensacola (850)202 4440 fax
> FL 32502 (850)291 0667 mobile
> phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes
>
>
>
>
>
>
> _________________________________________________________________
> Committee Msg Archives:
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
> Subscriber Config:
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
> Community Portal: http://ontolog.cim3.net/wiki/ (04)
_________________________________________________________________
Committee Msg Archives: http://ontolog.cim3.net/forum/ontology-summit-advisors/
Subscriber Config:
http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
Community Portal: http://ontolog.cim3.net/wiki/ (05)
|