|
All,
I received an off-list reply from somebody who read my last email
as a dismissal of ongoing work in ontology and existing practices.
I exaggerated the current situation for humorous effects, that
obviously backfired.
So, just to make it clear: I don't believe that ontologies are
pieces of art. Although many ontologies are developed in an ad hoc
fashion, there are, of course, people who are applying sound
methodologies in ontology development. The communique is based on
their experiences, we try to merge them into a big picture on how
to develop ontologies.
Best
Fabian
On 4/23/13 11:42 AM, Fabian Neuhaus wrote:
Pat,
It is always a joy to read your emails. Even if one is the
target of your scathing remarks. :-)
Since I came up with the initial outline of the communique, I
guess I am mainly to blame for the development of the
"managerial fantasy". So, let me try to defend it. You call the
document a phantasy, because it "is not based on observations
about how the work is ACTUALLY done." And you are right.
Here is my draft for a communique that is based on how the work
is actually done:
"In spite of a lot of literature on the subject, ontology
evaluation plays no important role in the development of
ontologies today (except for checking for logical consistency).
The main reason for this situation is that the development of
ontologies is currently not practiced as engineering activity,
but as a form of art, where design decisions are more driven by
schools of taste than by empirically validated methodologies.
Since ontologies seem to be pieces of art, we suggest a metric
that has been proven successful in the movie industry for a very
long time: so give 2 thumps up for your favorite ontology!"
I am not sure whether you would have endorsed this version of
the communique, but at least it would be short. :- )
Instead of describing the current situation of ontology
evaluation, we are trying to do something more productive.
Namely, to produce a consensus on how ontology development
should be done, and which forms of evaluation should happen
during the life cycle. So its a fantasy, all right. But some
peoples fantasies are other peoples visions, and this vision is
an attempt to merge the recommendations by the participants in
the Ontology Summit on how to organize the development of an
ontology. Hence, the document might not be an empirical
description of what is happening currently, but it is grounded
in the experience of many people who have developed ontologies.
You argue that developing such a document can cause harm,
because it might be used by managers to impose work habits on
other people for no good reason. I agree that this is a problem.
Any well-intentioned recommendation might be turned into a dogma
that is imposed blindly. However, you ignore the potential
benefits of the document. It is intended to provide help to
people who are trying to build (or use) ontologies and are
looking for recommendations on how to organize themselves by
more experienced ontologists.
So, I guess, it boils down to the question: Do we know enough
about the ontology life cycle to make recommendations about
development and evaluation that are of any value? You seem to
believe that we don't, and, thus, that "to go on record with a
detailed, confidently stated account which claims to be
normative, is both inappropriate and harmful." I agree that
there are many, many open questions
However, this does not mean that we have no idea about how to
develop ontologies. Thus, the open questions should not prevent
us from synthesizing the collective experience that is
available right now. We will not get everything right, but it
will enable other people to avoid some mistakes.
That's said, I see you point about the potential drawbacks. To
minimize the risk of turning the Communique into management
dogma, we should put some language into the executive summary
and/or the introduction that state the limits of our existing
experience explicitly.
Best
Fabian
On 4/21/13 12:32 PM, Pat Hayes wrote:
On Apr 21, 2013, at 5:11 AM, Amanda Vizedom wrote:
Pat,
Will you share with us the reasons for your non-support of
this communique?
Hi Amanda
Sure. (I didn't want to pester you unless you asked :-)
Several reasons. First, it is way too long and detailed, more
of an essay than a communique. It is hard, I would suggest,
for anyone to agree with all of it unequivocally. I feel like
I want to nit-pick with the text all over the place. For
example, this claim that ontologies must be "transparent to
all intended users" (who might not be ontologists) seems to me
to be ridiculously optimistic (or perhaps ridiculously
restrictive, if it is interpreted as a design constraint) and
to be rooted in a naive idea that correctness of an ontology
should be obvious to a domain expert. But this is simply
false. Just as one would not expect users of a program to find
the code source obvious, one cannot expect users of an
ontology to find the details of the ontology obvious. There
may well be decisions taken in the design, the reasons for
which are only apparent to professional ontologists (such as
whether to use continuants in talking about time and change.)
I could go through the document nitpicking like this, but
it seems pointless, because the entire enterprise is flawed.
The fact is, this whole document is a mangerial fantasy. We
do not have enough experience with ontology design and
deployment to know what the objective standards of "quality"
are, still less how to manage teams to achieve this
nonexistent standard. We don't know what are the "activities
that need to occur during the phases of a life cycle of an
ontology", so to go on record with a detailed, confidently
stated account which claims to be normative, is both
inappropriate and harmful. As I say, this is pure fantasy,
but it will be read by some as having an authority and will
be used by managers (most of whom know absolutely nothing
about ontologies) to impose work habits on other people for
no good reason. For example, "Does the ontology follow best
practices; in particular does it implement the upper
ontology...." Whoa. Is it "best practice" to even HAVE
an upper ontology? That is not clear. Most Web
ontologies, for example, are not subsumed under any
particular upper ontology. If out communique starts
being used to justify managers asking ontologists to
conform to an upper ontology, we will have done far more
harm than good.
I found it very telling that after pages of vacuous
managerial-theory babble about life cycles and "phases",
most of it content-free (such as "The requirements
development and analysis phase involves extending and
clarifying initial information until the intended usage is
sufficiently captured and understood to effectively guide
technical decisions. This process involves an interplay of
technical, business, and project-sponsor understanding.
Adequate requirements development and analysis is critical
to the success of any ontology development or usage."),
and an absurd schematic diagram showing tangles of arrows
connecting meaningless boxes, we read the almost plaintive
remark "Generally, appreciation of the
full life cycle of an ontology is not well established
within the ontology community." Damn right. In other
words, none of this is based in actual reality. It is
written as though it comprised observations about the
right way to work, but in fact, it is not based on
observations about how the work is ACTUALLY done.
Another nit-pick, to end. Your
second final observation begins: "Ontology development shares
strong similarities with information systems
development..." Does it, in fact? Is this based on
actual observations? (Of which projects?) Or is this
just an idea which the authors of this document feel
*should* be true?
Again: "Although there is much research
on ontology evaluation and many organizations use
sophisticated ontology evaluation and quality management
practices, awareness of this research, these practices,
and their importance to successful use of ontologies is
neither widespread nor sufficiently pooled to
constituted an accessible body of knowledge." The claim
that there is "much research" seems to me to be overly
optimistic, to put it mildly; but the main point is,
there is NOT widespread adoption of these practices.
There may be very good reasons for this lack of uptake:
the practices may be of limited utility, or of no real
utility at all. In my experience, that is the most
likely explanation.
My advice would be to completely toss
this document aside, and start over not with some
ready-made management-science theory about phases
and work cycles, but try observing, if possible
with a somewhat more humble attitude, how some
large-scale ontologies were actually built. You
might start with CYC, the granddaddy of all
large-scale ontologies. You will find that the
process bears almost no relationship to the
fantasy you describe here.
Sorry, but you did ask.
Pat
I'm asking not to try to argue, but because we haven't had
any input or feedback from you, and I value your insights
generally. We are still working to follow through on many
of the suggestion and critiques offered so far. This
follow-through may happen by substantal change to the
communique or by clarification of its scope and, if
possible and with summit community support, links to
better and detailed references on issues that are out of
scope, including other summit products. So, the request is
not empty; if you will let us know the reason(s) for your
discontent, we may be able to improve the communique by
understanding them.
Best,
Amanda
On Apr 20, 2013 11:25 PM, "Pat Hayes" <phayes@xxxxxxx>
wrote:
Peter and Mike, greetings.
On Apr 19, 2013, at 7:31 PM, Peter Yim wrote:
> Dear Ontology Summit Advisors,
>
> ...
> p.s. additionally, two very important reminders: ***
Please Note ***
> ...
> 2. We are expecting to have the Communique ready by
the time of the
> Symposium (no more wordsmithing of that document at
the face-to-face
> this time) and, as advised earlier, we are expecting
all Advisors to
> endorse the Communique (on an opt-out basis.)
Please opt me out of endorsing this Communique. If you
would prefer, you may remove me from the Advisory
Committee, in order to maintain an appearance of
solidarity.
Best wishes
Pat Hayes
------------------------------------------------------------
IHMC (850)434
8903 or (650)494 3973
40 South Alcaniz St. (850)202 4416 office
Pensacola (850)202 4440 fax
FL 32502 (850)291 0667
mobile
phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes
_________________________________________________________________
Committee Msg Archives: http://ontolog.cim3.net/forum/ontology-summit-advisors/
Subscriber
Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
Community Portal: http://ontolog.cim3.net/wiki/
_________________________________________________________________
Committee Msg Archives: http://ontolog.cim3.net/forum/ontology-summit-advisors/
Subscriber Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Discussion: : http://ontolog.cim3.net/forum/ontology-summit/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
Community Portal: http://ontolog.cim3.net/wiki/
|