Dear Alan, (01)
> My main point is that if there is any compilation step - and I fully
> support John's comment that this is not confined to DLs/OWL - then
> there is an extra step between what users assert and the final
> ontology, and that correspondingly, correcting errors may involves
> finding their origin in the source/asserted form.
>
> We looked at SNOMED CT, a large (>300Kclasses) medical terminology
> implemented in a subset of EL++ and now mandated for many uses in the
> US and numerous other countries. There are many occasions in which it
> is clear that editors have noted a missing subsumption and simply
> asserted or deleted it, rather than tracing down the cause in the
> original asserted form and correcting it. This led to unsystematic
> results. To give just one brief example, omission of a single axiom -
> "Skin SubClassOf SoftTissue" - led to many hundreds of errors in the
> compiled form. Editors asserted fixes for some but not others, so, for
> example, "injury to the skin of the ankle" was asserted to be a
> subclasses of "soft tissue injury" but the slightly more generic class
> "injury to skin of the lower leg" was not. Discussions with the
> organisation responsible (IHTSDO) made it clear that the editing life
> cycle did not adequately take account of the effects of the compilation
> when correcting reported errors. (02)
MW: This is really a quality management issue. Here the problem is that the
processes for updating the ontology are inadequate, or not followed, and
either the process, or compliance with the process need to be improved so
the same defects to not recur (as well as fixing the problem). My intension
is that these kinds of issues will be dealt with in Track C. Where what you
have described would make a good example. My experience in other domains is
very similar. (03)
Regards (04)
Matthew West
Information Junction
Tel: +44 1489 880185
Mobile: +44 750 3385279
Skype: dr.matthew.west
matthew.west@xxxxxxxxxxxxxxxxxxxxxxxxx
http://www.informationjunction.co.uk/
http://www.matthew-west.org.uk/ (05)
This email originates from Information Junction Ltd. Registered in England
and Wales No. 6632177.
Registered office: 2 Brookside, Meadow Way, Letchworth Garden City,
Hertfordshire, SG6 3JE. (06)
>
> I have also repeatedly heard statements like - "Since OWL uses logic,
> all we have to check is the assertions. The reasoner will take care or
> the rest." Yes, I know it is absurd, but there is a lot of
> misunderstanding out there.
>
> This may be too fine a detail at this stage of planning, but -
> accepting John's generalisation - any compilation step in the
> implementation methodology requires additional steps in the life cycle
> when modifying or "debugging" the ontology. This may be self evident
> to many of you, but, empirically, it is not obvious to many who develop
> ontologies using techniques involving a reasoning or compilation step.
>
> Regards
>
> Alan
>
>
>
> On 18 Dec 2012, at 18:24, Obrst, Leo J. wrote:
>
> > John, James, and all,
> >
> > Of course you must evaluate what kind of reasoner you will use, and
> generally that includes knowledge compilation. But reasoning can indeed
> point out flaws or inadequacies in the ontology, a true part of
> ontology evaluation.
> >
> > Another issue that must be raised is so-called "glass box vs. black
> box" evaluation, e.g., from [1]:
> >
> > "One issue in evaluating ontologies is whether to perform glass box
> > (component-based) vs. black box (task-based) evaluation, the latter
> > usually applied to ontologies that are tightly integrated with an
> > application performing specific tasks [36]."
> >
> > 36. Hartmann, Jens; Peter Spyns; Alain Giboin; Diana Maynard; Roberta
> > Cuel; Mari Carmen Suárez-Figueroa. 2005. D1.2.3 Methods for ontology
> > evaluation. EU-IST Network of Excellence (NoE) IST-2004-507482 KWEB
> Deliverable D1.2.3 (WP 1.2).
> >
> > Thanks,
> > Leo
> >
> > [1] Obrst, Leo; Werner Ceusters; Inderjeet Mani; Steve Ray; Barry
> Smith. 2007 The Evaluation of Ontologies: Toward Improved Semantic
> Interoperability. Chapter in: Semantic Web: Revolutionizing Knowledge
> Discovery in the Life Sciences, Christopher J. O. Baker and Kei-Hoi
> Cheung, Eds., Springer, 2007.
> >
> >
> >> -----Original Message-----
> >> From: ontology-summit-bounces@xxxxxxxxxxxxxxxx [mailto:ontology-
> >> summit-bounces@xxxxxxxxxxxxxxxx] On Behalf Of John F Sowa
> >> Sent: Tuesday, December 18, 2012 1:12 PM
> >> To: ontology-summit@xxxxxxxxxxxxxxxx
> >> Subject: Re: [ontology-summit] Reasoners and the life cycle
> >>
> >> Leo and James,
> >>
> >> That is highly misleading as a requirement:
> >>
> >> Leo
> >>> Many ontology projects either have automated reasoning as a
> >>> requirement now or they will in the future, as applications evolve
> >>> and new applications arise to use/reuse the ontology.
> >>
> >> James D
> >>> If this read "Many projects which involve an ontology have/will
> have
> >>> also a requirement for a reasoner ..." I'd be happy. But ontologies
> >>> and reasoners are different things. It is plausible to include
> >>> "reasoner-suitability" as a criterion in evaluating ontologies,
> certainly.
> >>> ... But, since an ontology is not a reasoner, it's irrelevant for
> >>> the purpose of evaluating ontologies.
> >>
> >> Yes. And I would emphasize just *one* of many criteria.
> >>
> >> Much more important is *compilation* from a very general, highly
> >> expressive ontology into formats that are appropriate for different
> >> purposes. Among them are database constraints, which can be stated
> >> in full first-order logic and evaluated in polynomial time.
> >>
> >> Cyc uses compilation methods very heavily for translating a logic
> >> (CycL) that is even more expressive than FOL into formats that are
> >> appropriate for different kinds of reasoning engines that are
> >> tailored for different kinds of problems.
> >>
> >> Cyc has been in the ontology business for 28 years, and they have
> >> implemented the largest formal ontology on the planet. It should be
> >> cited as an example of a *good* ontology: very general and stated in
> >> very expressive logic designed for ease of translation to formats
> for
> >> an open-ended variety of reasoners.
> >>
> >> That criterion does not imply that restricted logics, such as OWL
> DL,
> >> are necessarily bad. But they should be put in the category of
> >> special- purpose ontologies designed for just one narrow kind of
> reasoner.
> >>
> >> For more detail, see the following article which was published in a
> >> journal for which Jim Hendler was the editor. Jim said that he
> liked
> >> the article very much:
> >>
> >> http://www.jfsowa.com/pubs/fflogic.pdf
> >> Fads and fallacies about logic
> >>
> >> For another example, see the following note about a system that
> >> successfully used an *undecidable* logic in mission critical
> >> applications. They did use OWL for the type hierarchy, but most of
> >> the ontology was in a more expressive logic.
> >>
> >> John
> >>
> >> -------- Original Message --------
> >> Subject: Mission Critical IT
> >> Date: Fri, 14 Dec 2012 10:47:52 -0500
> >> From: John F Sowa <sowa@xxxxxxxxxxx>
> >> To: '[ontolog-forum] ' <ontolog-forum@xxxxxxxxxxxxxxxx>
> >> CC: Michel Vanden Bossche
> >>
> >> Dear Michel,
> >>
> >> Thanks for the information about the SWESE system.
> >>
> >> I'm sending this note to Ontolog Forum because it has a larger
> >> audience than the original list, Ontology Summit. I renamed the
> >> subject line to a pun on the name of your company and the elusive
> >> goal that the SW and AI in general have been groping toward -- with
> limited success.
> >>
> >> Following is the paper about SWESE:
> >>
> >> http://www.missioncriticalit.com/pdfs/swese-2007.pdf
> >> Ontology Driven Software Engineering for Real Life Applications
> >>
> >> I'd like to emphasize some of your points, but I recommend the
> entire
> >> note (copied below) and the above paper.
> >>
> >> MVB
> >>> The implementation is 100% Mercury (a concern for the customer),
> but
> >>> necessitates only 45,000 LOC of which 25,000 were automatically
> >> generated.
> >>
> >> A requirement for mainstream IT: generate the AI stuff
> >> *automatically* from notations that the IT staff and the SMEs can
> read and write.
> >>
> >>> It took 5000 man.days to specify, design, implement, test and
> deploy.
> >>> A similar system, although much simpler (one product at a time, one
> >>> channel, etc.), built in Java EE, took 15,000 md.
> >>
> >> For the record, Mercury is a high-speed logic-programming language:
> >>
> >> http://www.mercury.csse.unimelb.edu.au/information/features.html
> >>
> >> This increase in productivity for implementing the DSL (Domain
> >> Specific
> >> Language) is typical of Prolog and other logic programming
> languages.
> >> In fact, Alain Colmerauer, who designed the first Prolog system,
> >> adapted it from a parser he had originally written for machine
> translation.
> >>
> >>> The system was just starting in production (4000 users) when
> >>> Winterthur was acquired by Axa. Axa viewed our system as much too
> >>> advanced,
> >> considering
> >>> that their IT staff was not competent enough to support it. AAMOF,
> >> Axa has
> >>> just cancelled a 3rd attempt to build a system like ours, using
> >>> BaNCS, a package from TCS (Tata Consultancy Services). They pulled
> >>> the plug after spending 35 million USD.
> >>
> >> That is a sad but familiar story. Our company, VivoMind Research,
> >> LLC, had similar experiences. We implemented the system for oil and
> >> gas exploration described in the Future Directions paper (pp 15-17):
> >>
> >> http://www.jfsowa.com/pubs/futures.pdf
> >>
> >> After we developed that prototype, the oil company spoke with a
> >> larger company that claimed they could do anything we could do.
> They
> >> got a two-year contract with a lot more money than we did. But they
> failed.
> >>
> >>> Mercury is really great but is sociologically impossible to sell
> >>>
> >>> So, we decided to developed a Java backend for Mercury. Today,
> >>> Mercury compiled in Java is even faster than compiling to C.
> >>
> >> That is also a familiar story. What you deliver to customers is
> Java.
> >> They have heard of Java. Their programmers will never look at the
> >> code, but they'll feel comfortable that it's something they "know".
> >>
> >>> Rules (beyond OWL) are absolutely critical. We started with SWRL,
> >>> but real life applications require at least aggregation and
> >>> negation, with a subtle balance between OWA and CWA. So, we worked
> >>> on xSWRL (extended SWRL, a very poor name as this is not SWRL
> >>> anymore, but it simplifies the marketing ;-)).
> >>
> >> That is the major strength of the SW: it's a powerful hype machine.
> >> But what they hype is pitifully weak for serious applications.
> >>
> >>> It's a pity that W3C is not addressing the Business Rules issue
> >>> aggressively. RIF is like OSI: they couldn't decide between
> >>> Production Rules and Logic Rules (OSI couldn't decide between
> >>> connectionless and connectionfull).
> >>
> >> I spoke at a conference on Business Rules. One of the sponsors was
> >> Experian -- a credit bureau that processes huge amounts of data
> about
> >> everybody in the world. Their major language is Prolog. They use
> it
> >> so heavily that they bought Prologia, the company founded by Alain
> >> Colmerauer. Experian is truly mission critical, but they're so
> >> secretive that nobody knows how they specify their rules.
> >>
> >>> Same syndrome: W3C is not ready to bite the bullet, i.e.
> >>> aggressively support logic. They backtrack to Linked Data Object to
> >>> erase the AI from their story. And "Rules Interchange Format" is a
> terrible name.
> >>
> >> I don't blame Tim Berners-Lee. But I do blame the academics who
> >> rammed decidability down the throats of people who had no clue about
> >> what it meant. Look at Tim's original proposal from February 2000:
> >>
> >> http://www.w3.org/2000/01/sw/DevelopmentProposal
> >>
> >> Tim discussed Prolog and many other AI systems. He emphasized
> diversity.
> >> And he proposed SWeLL (Semantic Web Logic Language) for
> communication
> >> among heterogeneous systems. SWeLL was intended as a superset of
> >> propositional, first order, and higher order logic. Unfortunately,
> >> the final report claimed that OWL was a renamed version of SWeLL.
> >>
> >>> Logic gives you the possibility to debug declaratively your model.
> >>> This is what we do with our ODASE workbench: you can validate the
> >>> model with the SME before writing the first LOC.
> >>>
> >>> And, when the system runs, you can trace and explain why the system
> >>> works as it is.
> >>
> >> Yes. And to do that, you need a logic that can express anything
> that
> >> the system uses or does. But no decidable logic can.
> >>
> >>> With our new reasoner, we can price a car contract with more than
> >>> 10 coverages in 83 ms, on a standard Xeon 3.4 GHz, on one core.
> >>> Batch is easily scaled-out on multiple cores.
> >>>
> >>> This system is mission-critical and we move to a phase of
> productization.
> >>
> >> Congratulations. Note the time: 83 milliseconds. That is the kind
> >> of timing that mission critical applications require.
> >>
> >> Decidability only guarantees *finite* times. OWL guarantees finite
> >> time by limiting their models to trees. But the size of a tree
> grows
> >> exponentially with the depth. The algorithms can take longer than
> >> the age of the universe. That's finite, but not mission critical.
> >>
> >>> fundamental objective: better software requires better science
> >>> first, sound engineering then, and real-life experimentation with
> >>> real
> >> customers.
> >>
> >> Memo to W3C: Don't standardize anything until *after* there is at
> >> least one real live customer that is happy with the results.
> >>
> >> John
> >>
> >> -------- Original Message --------
> >> Subject: Re: [ontology-summit] Potential Tracks for Ontology Summit
> >> 2013
> >> Date: Fri, 14 Dec 2012 12:12:29 +0100
> >> From: Michel Vanden Bossche
> >>
> >> Dear John,
> >>
> >> Thanks for your mail to Michael [Uschold] and thanks to Michael for
> >> copying me.
> >>
> >> First, I read many of your papers and I share your views. The last
> >> paper I read is "Future Directions for Semantic Systems".
> >>
> >>> I repeat the question: What mission-critical applications has
> >>> anybody developed with that platform? Are they actually being used
> >>> on a daily basis? For any significant time (at least a year) after
> >>> the software was delivered?
> >>
> >> 1. SWESE system
> >>
> >> The system described in our SWESE paper was really "mission-
> critical".
> >> That project started in July 2004 and was completed, successfully in
> >> October 2007. It is an eInsurance system with the following
> objectives:
> >>
> >> - customer centric
> >> - multi products
> >> - multi channels
> >> - fast turn around time of new products (2 weeks max)
> >> - fully web
> >> - < 3 sec response time for pricing (8 products, 5 variants each)
> >> - etc.
> >>
> >> It is ontology driven, using OWL (just released in February 2004).
> >> The ontology defines the P&C (Property and Casualty) insurance
> domain
> >> and insurance products as instances of product definitions in the
> >> ontology. The UI is Ajax (we used a kind of JSP for Mercury that we
> >> named MSP). The navigation is defined, in the ontology, using YAOWL,
> >> a variant of YAWL (van der Aalst), a CPN based workflow. RBAC (Role
> >> Based Access Control) defines Roles and Powers ontologically.
> >>
> >> Rating/pricing is executed by a DSL that was imposed by the customer
> >> (Winterthur) so that pricing in the mid-tier (our system) was
> >> identical with the mainframe pricing (zOS, COBOL, DB2). We couldn't
> >> price on the mainframe calling services because the mainframe died
> >> due to the many CICS calls related to multiple products and multiple
> >> variants. We reimplemented the DSL written in COBOL in Mercury (it
> >> took us 3 weeks) and the response time went from more than 1 minute
> >> to
> >> 350 ms (on an HP Intel Xeon Box of that time).
> >>
> >> The implementation is 100% Mercury (a concern for the customer), but
> >> necessitates only 45,000 LOC of which 25,000 were automatically
> generated.
> >>
> >> It took 5000 man.days to specify, design, implement, test and
> deploy.
> >> A similar system, although much simpler (one product at a time, one
> >> channel, etc.), built in Java EE, took 15,000 md.
> >>
> >> The system was just starting in production (4000 users) when
> >> Winterthur was acquired by Axa. Axa viewed our system as much too
> >> advanced, considering that their IT staff was not competent enough
> to
> >> support it. AAMOF, Axa has just cancelled a 3rd attempt to build a
> >> system like ours, using BaNCS, a package from TCS (Tata Consultancy
> >> Services). They pulled the plug after spending 35 MUSD.
> >>
> >> 2. Lessons learned
> >>
> >> a. Mercury is really great but is sociologically impossible to sell
> >>
> >> So, we decided to developed a Java backend for Mercury. Today,
> >> Mercury compiled in Java is even faster than compiling to C. This
> >> illustrates the fact that a great language with a great compiler
> >> which can leverage fundamental CS research (e.g. Abstract Data
> Interpretation) wins.
> >>
> >> Since then, we have also developed a C# backend, which works fine
> but
> >> is not yet as mature.
> >>
> >> We have also built, as an exercise, an Erlang backend. This one is
> >> only alpha.
> >>
> >> We think that we need three layers:
> >>
> >> - logic at the modeling level
> >> - logic programming
> >> to consume the model
> >> to mitigate with the accepted languages of the day
> >> - Java, C# today and something else in the future (unknown)
> >>
> >> The problem is to transcend time.
> >>
> >> b. Code generation
> >>
> >> We recognized the importance of combining code generation (Business
> >> API in Java or C# to access the ontology) and interpretation.
> >>
> >> c. Rules
> >>
> >> Rules (beyond OWL) are absolutely critical. We started with SWRL,
> but
> >> real life applications require at least aggregation and negation,
> >> with a subtle balance between OWA and CWA. So, we worked on xSWRL
> >> (extended SWRL, a very poor name as this is not SWRL anymore, but it
> >> simplifies the marketing ;-)).
> >>
> >> We observed that Pellet performance for SWRL was not adequate.
> >> Relying on a PR engine (JESS...) was not an option because we would
> >> loose declarativeness. So, we built our own reasoners, trading
> >> completeness for performance ; this is not a problem in real-life.
> As
> >> usual, sound engineering requires a good balance between science and
> pragmatism.
> >>
> >> We now exceed the performance requirements imposed on us by our last
> >> customer (see later).
> >>
> >> It's a pity that W3C is not addressing the Business Rules issue
> >> aggressively. RIF is like OSI: they couldn't decide between
> >> Production Rules and Logic Rules (OSI couldn't decide between
> >> connectionless and connectionfull).
> >>
> >> Same syndrom: W3C is not ready to bite the bullet, i.e. aggressively
> >> support logic. They backtrack to Linked Data Object to erase the AI
> >> from their story. And "Rules Interchange Format" is a terrible name.
> >>
> >> Silk is going in the right direction, but it's maybe too ambitious
> >> for the moment.
> >>
> >> d. Process
> >>
> >> Ontologies are "Sein" but real applications require "Sein und Zeit".
> >> Again, besides OWL-S (a very imperative approach), W3C is not
> >> addressing that issue.
> >>
> >> We moved from the imperative approach (CPN) to a declarative
> approach
> >> combining OWL, xSWRL and LTL. In practice, may processes issues are
> >> related to Data flow problems, much less than to Control flow
> problems.
> >>
> >> e. Model "debugging"
> >>
> >> Logic gives yoiu the possibility to debug declaratively your model.
> >> This is what we do with our ODASE workbench: you can validate the
> >> model with the SME before writing the first LOC.
> >>
> >> And, when the system runs, you can trace and explain why the system
> >> works as it is.
> >>
> >> f. IT ego
> >>
> >> There is a really big problem with IT people. Their expertise is
> like
> >> the one of the Byzantine priests. They know everything related to
> >> their practical domain of expertise, but if they don't know
> something
> >> (logic, OWL...) they'll fight like hell to avoid being not experts
> >> anymore. So, they are ultra conservatives under the costume of
> modernists.
> >>
> >> This is a difficult issue. We have the feeling that we can now deal
> >> with them without antagonizing them too much ;-)
> >>
> >> 3. Last project
> >>
> >> We just completed the first phase of a project for AVIVA (another
> >> insurance company): externalizing the product definition and the
> >> pricing (product factory and rating engine).
> >>
> >> With our new reasoner, we can price a car contract with more than 10
> >> coverages in 83 ms, on a standard Xeon 3.4 GHz, on one core. Batch
> is
> >> easily scaled-out on multiple cores.
> >>
> >> This system is mission-critical and we move to a phase of
> productization.
> >>
> >> We are now investigating a parallel implementation of our reasoners.
> >> A language like Mercury makes that much less complex than Java or C#
> >> (maybe Java 8 will change that).
> >>
> >> 4. Conclusion
> >>
> >> I started the "logic dream" in 1978 with Prolog. We moved to Mercury
> >> in 1995. We had our first commercial application written in Mercury
> >> deployed in 2000. It is not really ontology driven (although there
> is
> >> a DAML like model), there are no rules (ŕ la SWRL), but there is a
> >> CPN at the heart. It still works after many changes, with a TCO that
> >> is 5 times better than similar systems (Case Management). Our first
> >> ODIS was built experimentally in 2003 and our first commercial
> system
> >> delivered in 2006. And we are still at the beginning...
> >>
> >> The biggest issue is "Software Innovation": IT people are frighten.
> >> Sales and Marketing is a NP complete problem.
> >>
> >> So, it needs time, a lot of time and the capability of not deviating
> >> from the fundamental objective: better software requires better
> >> science first, sound engineering then, and real-life experimentation
> >> with real customers.
> >>
> >> Tough... and you don't have VC happy with a 30 year business plan.
> >>
> >> Hope it helps,
> >>
> >> Michel
> >>
> >>
> >>
> >> ___________________________________________________________
> >> ______
> >> Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
> >> Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-
> >> summit/
> >> Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
> >> Community Files:
> >> http://ontolog.cim3.net/file/work/OntologySummit2013/
> >> Community Wiki: http://ontolog.cim3.net/cgi-
> >> bin/wiki.pl?OntologySummit2013
> >> Community Portal: http://ontolog.cim3.net/wiki/
> >
> > _________________________________________________________________
> > Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
> > Subscribe/Config:
> > http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
> > Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
> > Community Files:
> http://ontolog.cim3.net/file/work/OntologySummit2013/
> > Community Wiki:
> > http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
> > Community Portal: http://ontolog.cim3.net/wiki/
>
> -----------------------
> Alan Rector
> Professor of Medical Informatics
> School of Computer Science
> University of Manchester
> Manchester M13 9PL, UK
> TEL +44 (0) 161 275 6149/6188
> FAX +44 (0) 161 275 6204
> www.cs.man.ac.uk/~rector
> www.co-ode.org
> http://clahrc-gm.nihr.ac.uk/
>
>
>
>
>
>
> _________________________________________________________________
> Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
> Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-
> summit/
> Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
> Community Wiki: http://ontolog.cim3.net/cgi-
> bin/wiki.pl?OntologySummit2013
> Community Portal: http://ontolog.cim3.net/wiki/ (07)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013
Community Portal: http://ontolog.cim3.net/wiki/ (08)
|