|
Hi Thanks for attending my presentation on Ontology Regression Testing last
Thursday. Here are the answer to the questions asked during the presentation: Amanda Vizedom's question was regarding
effectual and ineffectual additions and removals and how these indicate fault.
An effectual addition is an addition of axiom α from Oi to Oi
+1 if α is not asserted and not entailed in Oi and it is
asserted and entailed in Oi +1. Effectual removals of β from Oi
to Oi +1 if β is asserted and entailed in Oi and it is
not asserted and not entailed in Oi +1(it is removed from the asserted
and from the entailment sets for Oi +1). Ineffectual removals and
additions are those additions and removals that only take place on the asserted
set and not in the entailment set. In my presentation, I am not stating that
ineffectual removals are faults. As you pointed out in the discussion, there
could be many practical reasons for axioms to appear only in the entailment set
and not in the asserted ontology. What I am highlighting in my presentation is
that we find axioms through out the life of the ontology that enter, exit, and
re-enter the ontology in different versions unchanged. It is in this entry/exit/re-entry cycles that we find
patterns in the type of additions and removals, which help us to further
classify the cycles into indicating and suggesting faults in the sequence of
changes due to their effectual and ineffectual editing events. In the
presentation slides 29, 30, 31 you can see the different patterns of
entry/exit/re-entry and their effectual and ineffectual patterns. As an
example, we say that the presence of an axiom has an indication of faults in
the sequence of change if it is effectually added to the ontology in version 1,
then it is ineffectually removed in versions 4, then in version 5 it’s effectually
removed, yet it appears again in version 7 unchanged through and effectual
addition only to remain until version 10 where it is effectually removed. The
fact that this axiom enters the ontology in version 1 and in version 7
unchanged shows that there is a regression in the content of the ontology for
this axiom from version 7 to version 1. It also indicates that the first
removal was done in two separate stages, an ineffectual removal in version 4,
and an effectual removal in version 5, which suggests that the intent is for
this axiom to be completely removed from the ontology. Its reappearance in
version 7 suggest that the first removal could be a fault (an unintended
action) and it was ‘fixed’ in version 7. Or that the addition in version 1 and
removal in version 5 was intended but the re-introduction in version 7 is a
fault. From the ontology data itself we cannot say which one is the fault, but
we can point out to developers and domain experts these axiomatic content ‘regressions’ through out the life of the ontology. The study of axioms life span (how long does an axiom exist in an ontology) helps us identify these
content regression sequences and their patterns. Doug’s question is: “could these changes be
a result of different users having different ideas of what the terms (should)
mean?” I don’t think these re-entries of axioms are correlated to different
ideas of what the term means. A key characteristic of the axioms we studied is
that the axioms appear and disappear from the ontology unchanged. If there was
a different idea (representation) for a term, then we would see axioms of
different forms such as A Is_A B in
one version, and A Is_A B and C in
another version. In our study we find that A
Is_A B appears in version x, it is removed in version y, and re-enters
unchanged in version z. Ali Hashemi commented in about some confusion
regarding the graphs in the presentation. There was an error in one
of the graphs as Ali commented. I’ve corrected the slide and included a graph key section at the
end of the presentation to indicate the meaning of the lines used to show
sequences. The updated presentation will be part of the proceedings for this session. Thanks for your questions. Please let me
know if you have any follow up questions. The paper that accompanies this presentation has been submitted to WoDOOM
2013. Regards, Maria Copeland
|