|
Pat,
It is always a joy to read your emails. Even if one is the target
of your scathing remarks. :-)
Since I came up with the initial outline of the communique, I
guess I am mainly to blame for the development of the "managerial
fantasy". So, let me try to defend it. You call the document a
phantasy, because it "is not based on observations about how the
work is ACTUALLY done." And you are right.
Here is my draft for a communique that is based on how the work is
actually done:
"In spite of a lot of literature on the subject, ontology
evaluation plays no important role in the development of
ontologies today (except for checking for logical consistency).
The main reason for this situation is that the development of
ontologies is currently not practiced as engineering activity, but
as a form of art, where design decisions are more driven by
schools of taste than by empirically validated methodologies.
Since ontologies seem to be pieces of art, we suggest a metric
that has been proven successful in the movie industry for a very
long time: so give 2 thumps up for your favorite ontology!"
I am not sure whether you would have endorsed this version of the
communique, but at least it would be short. :- )
Instead of describing the current situation of ontology
evaluation, we are trying to do something more productive. Namely,
to produce a consensus on how ontology development should be done,
and which forms of evaluation should happen during the life cycle.
So its a fantasy, all right. But some peoples fantasies are other
peoples visions, and this vision is an attempt to merge the
recommendations by the participants in the Ontology Summit on how
to organize the development of an ontology. Hence, the document
might not be an empirical description of what is happening
currently, but it is grounded in the experience of many people who
have developed ontologies.
You argue that developing such a document can cause harm, because
it might be used by managers to impose work habits on other people
for no good reason. I agree that this is a problem. Any
well-intentioned recommendation might be turned into a dogma that
is imposed blindly. However, you ignore the potential benefits of
the document. It is intended to provide help to people who are
trying to build (or use) ontologies and are looking for
recommendations on how to organize themselves by more experienced
ontologists.
So, I guess, it boils down to the question: Do we know enough
about the ontology life cycle to make recommendations about
development and evaluation that are of any value? You seem to
believe that we don't, and, thus, that "to go on record with a
detailed, confidently stated account which claims to be normative,
is both inappropriate and harmful." I agree that there are many,
many open questions
However, this does not mean that we have no idea about how to
develop ontologies. Thus, the open questions should not prevent us
from synthesizing the collective experience that is available
right now. We will not get everything right, but it will enable
other people to avoid some mistakes.
That's said, I see you point about the potential drawbacks. To
minimize the risk of turning the Communique into management dogma,
we should put some language into the executive summary and/or
the introduction that state the limits of our existing experience
explicitly.
Best
Fabian
On 4/21/13 12:32 PM, Pat Hayes wrote:
On Apr 21, 2013, at 5:11 AM, Amanda Vizedom wrote:
Pat,
Will you share with us the reasons for your non-support of this
communique?
Hi Amanda
Sure. (I didn't want to pester you unless you asked :-)
Several reasons. First, it is way too long and detailed, more of
an essay than a communique. It is hard, I would suggest, for
anyone to agree with all of it unequivocally. I feel like I want
to nit-pick with the text all over the place. For example, this
claim that ontologies must be "transparent to all intended
users" (who might not be ontologists) seems to me to be
ridiculously optimistic (or perhaps ridiculously restrictive, if
it is interpreted as a design constraint) and to be rooted in a
naive idea that correctness of an ontology should be obvious to
a domain expert. But this is simply false. Just as one would
not expect users of a program to find the code source obvious,
one cannot expect users of an ontology to find the details of
the ontology obvious. There may well be decisions taken in the
design, the reasons for which are only apparent to professional
ontologists (such as whether to use continuants in talking about
time and change.)
I could go through the document nitpicking like this, but
it seems pointless, because the entire enterprise is flawed.
The fact is, this whole document is a mangerial fantasy. We do
not have enough experience with ontology design and deployment
to know what the objective standards of "quality" are, still
less how to manage teams to achieve this nonexistent standard.
We don't know what are the "activities that need to occur
during the phases of a life cycle of an ontology", so to go on
record with a detailed, confidently stated account which
claims to be normative, is both inappropriate and harmful. As
I say, this is pure fantasy, but it will be read by some as
having an authority and will be used by managers (most of whom
know absolutely nothing about ontologies) to impose work
habits on other people for no good reason. For example, "Does the
ontology follow best practices; in particular does it
implement the upper ontology...." Whoa. Is it "best
practice" to even HAVE an upper ontology? That is not
clear. Most Web ontologies, for example, are not subsumed
under any particular upper ontology. If out communique
starts being used to justify managers asking ontologists
to conform to an upper ontology, we will have done far
more harm than good.
I found it very telling that after pages of vacuous
managerial-theory babble about life cycles and "phases", most
of it content-free (such as "The requirements development
and analysis phase involves extending and clarifying initial
information until the intended usage is sufficiently
captured and understood to effectively guide technical
decisions. This process involves an interplay of technical,
business, and project-sponsor understanding. Adequate
requirements development and analysis is critical to the
success of any ontology development or usage."), and an
absurd schematic diagram showing tangles of arrows connecting
meaningless boxes, we read the almost plaintive remark "Generally,
appreciation of the full life cycle of an ontology is not
well established within the ontology community." Damn
right. In other words, none of this is based in actual
reality. It is written as though it comprised observations
about the right way to work, but in fact, it is not based
on observations about how the work is ACTUALLY done.
Another
nit-pick, to end. Your second final observation begins: "Ontology
development shares strong similarities with information
systems development..." Does it, in fact? Is this based on
actual observations? (Of which projects?) Or is this just
an idea which the authors of this document feel *should*
be true?
Again: "Although
there is much research on ontology evaluation and many
organizations use sophisticated ontology evaluation and
quality management practices, awareness of this research,
these practices, and their importance to successful use of
ontologies is neither widespread nor sufficiently pooled
to constituted an accessible body of knowledge." The claim
that there is "much research" seems to me to be overly
optimistic, to put it mildly; but the main point is, there
is NOT widespread adoption of these practices. There may
be very good reasons for this lack of uptake: the
practices may be of limited utility, or of no real utility
at all. In my experience, that is the most likely
explanation.
My
advice would be to completely toss this document
aside, and start over not with some ready-made
management-science theory about phases and work
cycles, but try observing, if possible with a
somewhat more humble attitude, how some large-scale
ontologies were actually built. You might start with
CYC, the granddaddy of all large-scale ontologies.
You will find that the process bears almost no
relationship to the fantasy you describe here.
Sorry, but you did ask.
Pat
I'm asking not to try to argue, but because we haven't had
any input or feedback from you, and I value your insights
generally. We are still working to follow through on many
of the suggestion and critiques offered so far. This
follow-through may happen by substantal change to the
communique or by clarification of its scope and, if possible
and with summit community support, links to better
and detailed references on issues that are out of scope,
including other summit products. So, the request is not
empty; if you will let us know the reason(s) for your
discontent, we may be able to improve the communique by
understanding them.
Best,
Amanda
On Apr 20, 2013 11:25 PM, "Pat Hayes" <phayes@xxxxxxx>
wrote:
Peter and Mike, greetings.
On Apr 19, 2013, at 7:31 PM, Peter Yim wrote:
> Dear Ontology Summit Advisors,
>
> ...
> p.s. additionally, two very important reminders: ***
Please Note ***
> ...
> 2. We are expecting to have the Communique ready by the
time of the
> Symposium (no more wordsmithing of that document at the
face-to-face
> this time) and, as advised earlier, we are expecting
all Advisors to
> endorse the Communique (on an opt-out basis.)
Please opt me out of endorsing this Communique. If you would
prefer, you may remove me from the Advisory Committee, in
order to maintain an appearance of solidarity.
Best wishes
Pat Hayes
------------------------------------------------------------
IHMC (850)434
8903 or (650)494 3973
40 South Alcaniz St. (850)202 4416 office
Pensacola (850)202 4440 fax
FL 32502 (850)291 0667 mobile
phayesAT-SIGNihmc.us http://www.ihmc.us/users/phayes
_________________________________________________________________
Committee Msg
Archives: http://ontolog.cim3.net/forum/ontology-summit-advisors/
Subscriber
Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
Community
Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Discussion:
: http://ontolog.cim3.net/forum/ontology-summit/
Community
Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
Community Portal: http://ontolog.cim3.net/wiki/
_________________________________________________________________
Committee Msg Archives:
http://ontolog.cim3.net/forum/ontology-summit-advisors/
Subscriber Config:
http://ontolog.cim3.net/mailman/listinfo/ontology-summit-advisors/
Community Files:
http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Discussion: :
http://ontolog.cim3.net/forum/ontology-summit/
Community Wiki:
http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
Community Portal: http://ontolog.cim3.net/wiki/
|