ontology-summit-advisors
[Top] [All Lists]

Re: [ontology-summit-advisors] Developing a managerial fantasy?

To: OntologySummit2013 Advisory Committee <ontology-summit-advisors@xxxxxxxxxxxxxxxx>
From: Aldo Gangemi <gangemi@xxxxxxx>
Date: Thu, 25 Apr 2013 10:27:59 +0200
Message-id: <F77EC0FE-DFFD-4951-AEBB-0671B4E923C7@xxxxxxx>
Besides Communique's specific wording (I have not yet found the version you're 
talking about), empirical evidence for the usefulness of some *methods* (maybe 
not full-fledged methodologies like we find in management and heavyweight 
software engineering) does exist. See e.g. [1] and [2].
In favor of a disclaimer to the existence of tantalizing methodologies for 
ontology engineering, the methods I refer to are more alike guidelines to apply 
design patterns and extreme programming, so quite far away from a managerial 
fantasy.
Ciao
Aldo    (01)

[1] http://dl.acm.org/citation.cfm?id=1597743
[2] http://link.springer.com/chapter/10.1007/978-3-642-16438-5_9#page-1    (02)

On Apr 25, 2013, at 6:58:34 AM , Pat Hayes <phayes@xxxxxxx> wrote:    (03)

> 
> On Apr 23, 2013, at 1:44 PM, Fabian Neuhaus wrote:
> 
>> All, 
>> I received an off-list reply from somebody who read my last email as a 
>dismissal of ongoing work in ontology and existing practices. I exaggerated 
>the current situation for humorous effects, that obviously backfired. 
>> 
>> So, just to make it clear: I don't believe that ontologies are pieces of 
>art. Although many ontologies are developed in an ad hoc fashion, there are, 
>of course, people who are applying sound methodologies in ontology development.
> 
> Are there, indeed? Have these 'sound' methodologies been tested in practice, 
>with comparatives between them? What empirical results are available to show 
>that these methodologies have any value? 
> 
> Any methodology has its limitations and problems: what can be said about the 
>limits of applicability of any of these methodologies, or the conditions under 
>which they might be expected to fail? 
> 
>> The communique is based on       their experiences
> 
> Based on how many of them, used under what circumstances, on what projects? 
>Were they all of a similar scale, or varied? Were any of them Web-based? What 
>purposes did they serve? How large is the sample size represented here?  
> 
> It would be better to read about the actual experiences. Why not have the 
>communique cite them in support of its claims, so that readers can make up 
>their own minds about how "sound" all this really is?
> 
> Pat
> 
> 
>> , we try to merge them into a big picture on how to develop ontologies. 
>> 
>> Best
>> Fabian 
>> 
>> On 4/23/13 11:42 AM, Fabian Neuhaus wrote:
>>> Pat, 
>>> It is always a joy to read your emails. Even if one is the target of your 
>scathing remarks. :-) 
>>> 
>>> Since I came up with the initial outline of the communique, I guess I am 
>mainly to blame for the development of the "managerial fantasy". So, let me 
>try to defend it. You call the document  a phantasy, because it "is not based 
>on observations about how the work is ACTUALLY done." And you are right. 
>>> 
>>> Here is my draft for a communique that is based on how the work is actually 
>done: 
>>> "In spite of a lot of literature on the subject, ontology evaluation plays 
>no important role in the development of ontologies today (except for checking 
>for logical consistency).  The main reason for this situation is that the 
>development of ontologies is currently not practiced as engineering activity, 
>but as a form of art, where design decisions are more driven by schools of 
>taste than by empirically validated methodologies. Since ontologies seem to be 
>pieces of art, we suggest a metric that has been proven successful in the 
>movie industry for a very long time: so give 2 thumps up for your favorite 
>ontology!"   
>>> 
>>> I am not sure whether you would have endorsed this version of the 
>communique, but at least it would be short. :- )
>>> 
>>> Instead of describing the current situation of ontology evaluation, we are 
>trying to do something more productive. Namely, to produce a consensus on how 
>ontology development should be done, and which forms of evaluation should 
>happen during the life cycle. So its a fantasy, all right. But some peoples 
>fantasies are other peoples visions, and this vision is an attempt to merge 
>the recommendations by the participants in the Ontology Summit on how to 
>organize the development of an ontology. Hence, the document might not be an 
>empirical description of what is happening currently, but it is grounded in 
>the experience of many people who have developed ontologies.  
>>> 
>>> You argue that developing such a document can cause harm, because it might 
>be used by managers to impose work habits on other people for no good reason. 
>I agree that this is a problem. Any well-intentioned recommendation might be 
>turned into a dogma that is imposed blindly.  However, you ignore the 
>potential benefits of the document. It is intended to provide help to people 
>who are trying to build (or use) ontologies and are looking for 
>recommendations on how to organize themselves by more experienced ontologists. 
>>> 
>>> So, I guess, it boils down to the question: Do we know enough about the 
>ontology life cycle to make recommendations about development and evaluation 
>that are of any value?  You seem to believe that we don't, and, thus, that "to 
>go on record with a detailed, confidently stated account which claims to be 
>normative, is both inappropriate and harmful." I agree that there are many, 
>many open questions 
>>> However, this does not mean that we have no idea about how to develop 
>ontologies. Thus, the open questions should not prevent us from synthesizing 
>the collective experience  that is available right now. We will not get 
>everything right, but it will enable other people to avoid some mistakes. 
>>> 
>>> That's said, I see you point about the potential drawbacks. To minimize the 
>risk of turning the Communique into management dogma, we  should put  some 
>language into the executive summary and/or the introduction that state the 
>limits of our existing         experience explicitly. 
>>> 
>>> Best
>>> Fabian 
>>> 
>>> 
>>> On 4/21/13 12:32 PM, Pat Hayes wrote:
>>>> 
>>>> On Apr 21, 2013, at 5:11 AM, Amanda Vizedom wrote:
>>>> 
>>>>> Pat,
>>>>> 
>>>>> Will you share with us the reasons for your non-support of this 
>communique?
>>>> 
>>>> Hi Amanda
>>>> 
>>>> Sure. (I didn't want to pester you unless you asked :-) Several reasons. 
>First, it is way too long and detailed, more of an essay than a communique. It 
>is hard, I would suggest, for anyone to agree with all of it unequivocally. I 
>feel like I want to nit-pick with the text all over the place. For example, 
>this claim that ontologies must be "transparent to all intended users" (who 
>might not be ontologists) seems to me to be ridiculously optimistic (or 
>perhaps ridiculously restrictive, if it is interpreted as a design constraint) 
> and to be rooted in a naive idea that correctness of an ontology should be 
>obvious to a domain expert.  But this is simply false. Just as one would not 
>expect users of a program to find the code source obvious, one cannot expect 
>users of an ontology to find the details of the ontology obvious. There may 
>well be decisions taken in the design, the reasons for which are only apparent 
>to professional ontologists (such as whether to use continuants in t
> alking about time and change.)
>>>> 
>>>> I could go through the document nitpicking like this, but it seems 
>pointless, because the entire enterprise is flawed. The fact is, this whole 
>document is a mangerial fantasy. We do not have enough experience with 
>ontology design and deployment to know what the objective standards of 
>"quality" are, still less how to manage teams to achieve this nonexistent 
>standard. We don't know what are the "activities that need to occur during the 
>phases of a life cycle of an ontology", so to go on record with a detailed, 
>confidently stated account which claims to be normative, is both inappropriate 
>and harmful. As I say, this is pure fantasy, but it will be read by some as 
>having an authority and will be used by managers (most of whom know absolutely 
>nothing about ontologies) to impose work habits on other people for no good 
>reason. For example, "
>>>> Does the ontology follow best
>>>>               practices; in particular does it implement the upper
>>>>               ontology...." Whoa. Is it "best practice" to even HAVE
>>>>               an upper ontology? That is not clear. Most Web
>>>>               ontologies, for example, are not subsumed under any
>>>>               particular upper ontology. If out communique starts
>>>>               being used to justify managers asking ontologists to
>>>>               conform to an upper ontology, we will have done far more
>>>>               harm than good.
>>>> 
>>>> 
>>>> I found it very telling that after pages of vacuous managerial-theory 
>babble about life cycles and "phases", most of it content-free (such as "The 
>requirements development and analysis phase involves extending and clarifying 
>initial information until the intended usage is sufficiently captured and 
>understood to effectively guide technical decisions. This process involves an 
>interplay of technical, business, and project-sponsor understanding. Adequate 
>requirements development and analysis is critical to the success of any 
>ontology development or usage."), and an absurd schematic diagram showing 
>tangles of arrows connecting meaningless boxes, we read the almost plaintive 
>remark "
>>>> Generally, appreciation of the
>>>>               full life cycle of an ontology is not well established
>>>>               within the ontology community." Damn right. In other
>>>>               words, none of this is based in actual reality. It is
>>>>               written as though it comprised observations about the
>>>>               right way to work, but in fact, it is not based on
>>>>               observations about how the work is ACTUALLY done. 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Another nit-pick, to end. Your
>>>>               second final observation begins: "
>>>> Ontology development shares
>>>>               strong similarities with information systems
>>>>               development..." Does it, in fact? Is this based on
>>>>               actual observations? (Of which projects?) Or is this
>>>>               just an idea which the authors of this document feel
>>>>               *should* be true? 
>>>> 
>>>> Again: "
>>>> Although there is much research
>>>>               on ontology evaluation and many organizations use
>>>>               sophisticated ontology evaluation and quality management
>>>>               practices, awareness of this research, these practices,
>>>>               and their importance to successful use of ontologies is
>>>>               neither widespread nor sufficiently pooled to
>>>>               constituted an accessible body of knowledge." The claim
>>>>               that there is "much research" seems to me to be overly
>>>>               optimistic, to put it mildly; but the main point is,
>>>>               there is NOT widespread adoption of these practices.
>>>>               There may be very good reasons for this lack of uptake:
>>>>               the practices may be of limited utility, or of no real
>>>>               utility at all. In my experience, that is the most
>>>>               likely explanation. 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> My advice would be to completely toss
>>>>                     this document aside, and start over not with some
>>>>                     ready-made management-science theory about phases
>>>>                     and work cycles, but try observing, if possible
>>>>                     with a somewhat more humble attitude, how some
>>>>                     large-scale ontologies were actually built. You
>>>>                     might start with CYC, the granddaddy of all
>>>>                     large-scale ontologies. You will find that the
>>>>                     process bears almost no relationship to the
>>>>                     fantasy you describe here. 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Sorry, but you did ask.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Pat
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> I'm asking not to try to argue, but because we haven't had any input or 
>feedback from you, and I value your insights generally.  We are still working 
>to follow through on many of the suggestion and critiques offered so far. This 
>follow-through may happen by substantal change to the communique or by 
>clarification of its scope and, if possible and with summit community support, 
>links to better and detailed references on issues that are out of scope, 
>including other summit products. So, the request is not empty; if you will let 
>us know the reason(s) for your discontent, we may be able to improve the 
>communique by understanding them.
>>>>> 
>>>>> Best,
>>>>> Amanda
>>>>> On Apr 20, 2013 11:25 PM, "Pat Hayes" <phayes@xxxxxxx> wrote:
>>>>> Peter and Mike, greetings.
>>>>> 
>>>>> On Apr 19, 2013, at 7:31 PM, Peter Yim wrote:
>>>>> 
>>>>>> Dear Ontology Summit Advisors,
>>>>>> 
>>>>>> ...
>>>>>> p.s. additionally, two very important reminders: *** Please Note ***
>>>>>> ...
>>>>>> 2. We are expecting to have the Communique ready by the time of the
>>>>>> Symposium (no more wordsmithing of that document at the face-to-face
>>>>>> this time) and, as advised earlier, we are expecting all Advisors to
>>>>>> endorse the Communique (on an opt-out basis.)
>>>>> 
>>>>> Please opt me out of endorsing this Communique. If you would prefer, you 
>may remove me from the Advisory Committee, in order to maintain an appearance 
>of solidarity.
>>>>> 
>>>>> Best wishes
>>>>> 
>>>>> Pat Hayes
>>>>> 
>>>>> 
>>>>> ------------------------------------------------------------
>>>>> IHMC                                     (850)434 8903 or (650)494 3973
>>>>> 40 South Alcaniz St.           (850)202 4416   office
>>>>> Pensacola                            (850)202 4440   fax
>>>>> FL 32502                              (850)291 0667   mobile
>>>>> phayesAT-SIGNihmc.us       http://www.ihmc.us/users/phayes
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _________________________________________________________________
>>>>> Committee Msg Archives: 
>http://ontolog.cim3.net/forum/ontology-summit-advisors/
>>>>> Subscriber Config: 
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
>>>>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
>>>>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
>>>>> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
>>>>> Community Portal: http://ontolog.cim3.net/wiki/
>>>>> 
>>>>> _________________________________________________________________
>>>>> Committee Msg Archives: 
>http://ontolog.cim3.net/forum/ontology-summit-advisors/ 
>>>>> Subscriber Config: 
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/  
>>>>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/ 
>>>>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/ 
>>>>> Community Wiki: 
>http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013 
>>>>> Community Portal: http://ontolog.cim3.net/wiki/ 
>>>> 
>>>> ------------------------------------------------------------
>>>> IHMC                                     (850)434 8903 or (650)494 3973   
>>>> 40 South Alcaniz St.           (850)202 4416   office
>>>> Pensacola                            (850)202 4440   fax
>>>> FL 32502                              (850)291 0667   mobile
>>>> phayesAT-SIGNihmc.us       http://www.ihmc.us/users/phayes
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
>> _________________________________________________________________
>> Committee Msg Archives: 
>http://ontolog.cim3.net/forum/ontology-summit-advisors/ 
>> Subscriber Config: 
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/  
>> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/ 
>> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/ 
>> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013 
>> Community Portal: http://ontolog.cim3.net/wiki/ 
> 
> ------------------------------------------------------------
> IHMC                                     (850)434 8903 or (650)494 3973   
> 40 South Alcaniz St.           (850)202 4416   office
> Pensacola                            (850)202 4440   fax
> FL 32502                              (850)291 0667   mobile
> phayesAT-SIGNihmc.us       http://www.ihmc.us/users/phayes
> 
> 
> 
> 
> 
> 
> _________________________________________________________________
> Committee Msg Archives: 
>http://ontolog.cim3.net/forum/ontology-summit-advisors/ 
> Subscriber Config: 
>http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/  
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/ 
> Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/ 
> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013 
> Community Portal: http://ontolog.cim3.net/wiki/     (04)


_________________________________________________________________
Committee Msg Archives: http://ontolog.cim3.net/forum/ontology-summit-advisors/ 
Subscriber Config: 
http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/  
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/ 
Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/ 
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013 
Community Portal: http://ontolog.cim3.net/wiki/     (05)
<Prev in Thread] Current Thread [Next in Thread>