On Apr 23, 2013, at 1:44 PM, Fabian Neuhaus wrote: (01)
> All,
> I received an off-list reply from somebody who read my last email as a
>dismissal of ongoing work in ontology and existing practices. I exaggerated
>the current situation for humorous effects, that obviously backfired.
>
> So, just to make it clear: I don't believe that ontologies are pieces of art.
>Although many ontologies are developed in an ad hoc fashion, there are, of
>course, people who are applying sound methodologies in ontology development. (02)
Are there, indeed? Have these 'sound' methodologies been tested in practice,
with comparatives between them? What empirical results are available to show
that these methodologies have any value? (03)
Any methodology has its limitations and problems: what can be said about the
limits of applicability of any of these methodologies, or the conditions under
which they might be expected to fail? (04)
> The communique is based on their experiences (05)
Based on how many of them, used under what circumstances, on what projects?
Were they all of a similar scale, or varied? Were any of them Web-based? What
purposes did they serve? How large is the sample size represented here? (06)
It would be better to read about the actual experiences. Why not have the
communique cite them in support of its claims, so that readers can make up
their own minds about how "sound" all this really is? (07)
Pat (08)
> , we try to merge them into a big picture on how to develop ontologies.
>
> Best
> Fabian
>
> On 4/23/13 11:42 AM, Fabian Neuhaus wrote:
>> Pat,
>> It is always a joy to read your emails. Even if one is the target of your
>scathing remarks. :-)
>>
>> Since I came up with the initial outline of the communique, I guess I am
>mainly to blame for the development of the "managerial fantasy". So, let me
>try to defend it. You call the document a phantasy, because it "is not based
>on observations about how the work is ACTUALLY done." And you are right.
>>
>> Here is my draft for a communique that is based on how the work is actually
>done:
>> "In spite of a lot of literature on the subject, ontology evaluation plays
>no important role in the development of ontologies today (except for checking
>for logical consistency). The main reason for this situation is that the
>development of ontologies is currently not practiced as engineering activity,
>but as a form of art, where design decisions are more driven by schools of
>taste than by empirically validated methodologies. Since ontologies seem to be
>pieces of art, we suggest a metric that has been proven successful in the
>movie industry for a very long time: so give 2 thumps up for your favorite
>ontology!"
>>
>> I am not sure whether you would have endorsed this version of the
>communique, but at least it would be short. :- )
>>
>> Instead of describing the current situation of ontology evaluation, we are
>trying to do something more productive. Namely, to produce a consensus on how
>ontology development should be done, and which forms of evaluation should
>happen during the life cycle. So its a fantasy, all right. But some peoples
>fantasies are other peoples visions, and this vision is an attempt to merge
>the recommendations by the participants in the Ontology Summit on how to
>organize the development of an ontology. Hence, the document might not be an
>empirical description of what is happening currently, but it is grounded in
>the experience of many people who have developed ontologies.
>>
>> You argue that developing such a document can cause harm, because it might
>be used by managers to impose work habits on other people for no good reason.
>I agree that this is a problem. Any well-intentioned recommendation might be
>turned into a dogma that is imposed blindly. However, you ignore the
>potential benefits of the document. It is intended to provide help to people
>who are trying to build (or use) ontologies and are looking for
>recommendations on how to organize themselves by more experienced ontologists.
>>
>> So, I guess, it boils down to the question: Do we know enough about the
>ontology life cycle to make recommendations about development and evaluation
>that are of any value? You seem to believe that we don't, and, thus, that "to
>go on record with a detailed, confidently stated account which claims to be
>normative, is both inappropriate and harmful." I agree that there are many,
>many open questions
>> However, this does not mean that we have no idea about how to develop
>ontologies. Thus, the open questions should not prevent us from synthesizing
>the collective experience that is available right now. We will not get
>everything right, but it will enable other people to avoid some mistakes.
>>
>> That's said, I see you point about the potential drawbacks. To minimize the
>risk of turning the Communique into management dogma, we should put some
>language into the executive summary and/or the introduction that state the
>limits of our existing experience explicitly.
>>
>> Best
>> Fabian
>>
>>
>> On 4/21/13 12:32 PM, Pat Hayes wrote:
>>>
>>> On Apr 21, 2013, at 5:11 AM, Amanda Vizedom wrote:
>>>
>>>> Pat,
>>>>
>>>> Will you share with us the reasons for your non-support of this communique?
>>>
>>> Hi Amanda
>>>
>>> Sure. (I didn't want to pester you unless you asked :-) Several reasons.
>First, it is way too long and detailed, more of an essay than a communique. It
>is hard, I would suggest, for anyone to agree with all of it unequivocally. I
>feel like I want to nit-pick with the text all over the place. For example,
>this claim that ontologies must be "transparent to all intended users" (who
>might not be ontologists) seems to me to be ridiculously optimistic (or
>perhaps ridiculously restrictive, if it is interpreted as a design constraint)
> and to be rooted in a naive idea that correctness of an ontology should be
>obvious to a domain expert. But this is simply false. Just as one would not
>expect users of a program to find the code source obvious, one cannot expect
>users of an ontology to find the details of the ontology obvious. There may
>well be decisions taken in the design, the reasons for which are only apparent
>to professional ontologists (such as whether to use continuants in talking
>about time and change.)
>>>
>>> I could go through the document nitpicking like this, but it seems
>pointless, because the entire enterprise is flawed. The fact is, this whole
>document is a mangerial fantasy. We do not have enough experience with
>ontology design and deployment to know what the objective standards of
>"quality" are, still less how to manage teams to achieve this nonexistent
>standard. We don't know what are the "activities that need to occur during the
>phases of a life cycle of an ontology", so to go on record with a detailed,
>confidently stated account which claims to be normative, is both inappropriate
>and harmful. As I say, this is pure fantasy, but it will be read by some as
>having an authority and will be used by managers (most of whom know absolutely
>nothing about ontologies) to impose work habits on other people for no good
>reason. For example, "
>>> Does the ontology follow best
>>> practices; in particular does it implement the upper
>>> ontology...." Whoa. Is it "best practice" to even HAVE
>>> an upper ontology? That is not clear. Most Web
>>> ontologies, for example, are not subsumed under any
>>> particular upper ontology. If out communique starts
>>> being used to justify managers asking ontologists to
>>> conform to an upper ontology, we will have done far more
>>> harm than good.
>>>
>>>
>>> I found it very telling that after pages of vacuous managerial-theory
>babble about life cycles and "phases", most of it content-free (such as "The
>requirements development and analysis phase involves extending and clarifying
>initial information until the intended usage is sufficiently captured and
>understood to effectively guide technical decisions. This process involves an
>interplay of technical, business, and project-sponsor understanding. Adequate
>requirements development and analysis is critical to the success of any
>ontology development or usage."), and an absurd schematic diagram showing
>tangles of arrows connecting meaningless boxes, we read the almost plaintive
>remark "
>>> Generally, appreciation of the
>>> full life cycle of an ontology is not well established
>>> within the ontology community." Damn right. In other
>>> words, none of this is based in actual reality. It is
>>> written as though it comprised observations about the
>>> right way to work, but in fact, it is not based on
>>> observations about how the work is ACTUALLY done.
>>>
>>>
>>>
>>>
>>>
>>> Another nit-pick, to end. Your
>>> second final observation begins: "
>>> Ontology development shares
>>> strong similarities with information systems
>>> development..." Does it, in fact? Is this based on
>>> actual observations? (Of which projects?) Or is this
>>> just an idea which the authors of this document feel
>>> *should* be true?
>>>
>>> Again: "
>>> Although there is much research
>>> on ontology evaluation and many organizations use
>>> sophisticated ontology evaluation and quality management
>>> practices, awareness of this research, these practices,
>>> and their importance to successful use of ontologies is
>>> neither widespread nor sufficiently pooled to
>>> constituted an accessible body of knowledge." The claim
>>> that there is "much research" seems to me to be overly
>>> optimistic, to put it mildly; but the main point is,
>>> there is NOT widespread adoption of these practices.
>>> There may be very good reasons for this lack of uptake:
>>> the practices may be of limited utility, or of no real
>>> utility at all. In my experience, that is the most
>>> likely explanation.
>>>
>>>
>>>
>>>
>>>
>>>
>>> My advice would be to completely toss
>>> this document aside, and start over not with some
>>> ready-made management-science theory about phases
>>> and work cycles, but try observing, if possible
>>> with a somewhat more humble attitude, how some
>>> large-scale ontologies were actually built. You
>>> might start with CYC, the granddaddy of all
>>> large-scale ontologies. You will find that the
>>> process bears almost no relationship to the
>>> fantasy you describe here.
>>>
>>>
>>>
>>>
>>>
>>> Sorry, but you did ask.
>>>
>>>
>>>
>>>
>>> Pat
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>
>>>> I'm asking not to try to argue, but because we haven't had any input or
>feedback from you, and I value your insights generally. We are still working
>to follow through on many of the suggestion and critiques offered so far. This
>follow-through may happen by substantal change to the communique or by
>clarification of its scope and, if possible and with summit community support,
>links to better and detailed references on issues that are out of scope,
>including other summit products. So, the request is not empty; if you will let
>us know the reason(s) for your discontent, we may be able to improve the
>communique by understanding them.
>>>>
>>>> Best,
>>>> Amanda
>>>> On Apr 20, 2013 11:25 PM, "Pat Hayes" <phayes@xxxxxxx> wrote:
>>>> Peter and Mike, greetings.
>>>>
>>>> On Apr 19, 2013, at 7:31 PM, Peter Yim wrote:
>>>>
>>>> > Dear Ontology Summit Advisors,
>>>> >
>>>> > ...
>>>> > p.s. additionally, two very important reminders: *** Please Note ***
>>>> > ...
>>>> > 2. We are expecting to have the Communique ready by the time of the
>>>> > Symposium (no more wordsmithing of that document at the face-to-face
>>>> > this time) and, as advised earlier, we are expecting all Advisors to
>>>> > endorse the Communique (on an opt-out basis.)
>>>>
>>>> Please opt me out of endorsing this Communique. If you would prefer, you
>may remove me from the Advisory Committee, in order to maintain an appearance
>of solidarity.
>>>>
>>>> Best wishes
>>>>
>>>> Pat Hayes
>>>>
>>>>
>>>> ------------------------------------------------------------
>>>> IHMC (850)434 8903 or (650)494 3973
>>>> 40 South Alcaniz St. (850)202 4416 office
>>>> Pensacola (850)202 4440 fax
>>>> FL 32502 (850)291 0667 mobile
>>>> phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________________________
>>>> Committee Msg Archives:
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
>>>> Subscriber Config:
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
>>>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
>>>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
>>>> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
>>>> Community Portal: http://ontolog.cim3.net/wiki/
>>>>
>>>> _________________________________________________________________
>>>> Committee Msg Archives:
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
>>>> Subscriber Config:
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
>>>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
>>>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
>>>> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
>>>> Community Portal: http://ontolog.cim3.net/wiki/
>>>
>>> ------------------------------------------------------------
>>> IHMC (850)434 8903 or (650)494 3973
>>> 40 South Alcaniz St. (850)202 4416 office
>>> Pensacola (850)202 4440 fax
>>> FL 32502 (850)291 0667 mobile
>>> phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes
>>>
>>>
>>>
>>>
>>
>
>
> _________________________________________________________________
> Committee Msg Archives:
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
> Subscriber Config:
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
> Community Portal: http://ontolog.cim3.net/wiki/ (09)
------------------------------------------------------------
IHMC (850)434 8903 or (650)494 3973
40 South Alcaniz St. (850)202 4416 office
Pensacola (850)202 4440 fax
FL 32502 (850)291 0667 mobile
phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes (010)
_________________________________________________________________
Committee Msg Archives: http://ontolog.cim3.net/forum/ontology-summit-advisors/
Subscriber Config:
http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
Community Portal: http://ontolog.cim3.net/wiki/ (011)
|