Re: [ontology-summit] Reasoners and the life cycle

["ses@xxxxxxx" <ses@xxxxxxx>]

A reasoner can be a useful tool for evaluation during the development life cycle. 

Obvious uses are:

1) Unit tests to verify that some desired entailments are inferred, and that some  undesired entailments are not.

2) Inferences can be generated (with a bias towards  those using newly asserted statements). These can then be presented to SMEs for yes/no/it-depends/wtf validation. These responses can be used as additional unit or integration tests.

3) TMS can give warnings when an assertion causes major ripples (one plank at a time *while staying afloat*).

[slightly off topic for summit: if you look at the source code to opencyc that is part of larkc, and thus under an open license (apache), you can see that the cost evaluation and  selection of removal modules etc. is done at query time.  
Not much precompliation is done; there certainly is some forward chaining, and code can be fired at assertion time, but e.g. a set of horn clauses that could readily be expressed in pure Prolog do not get converted into some analog of WAM (prolog is a high level WAM assembler). 
With its meta-logical mechanisms for asserting that the complete extent of a predicate is known, this  optimization would not be implausible. 
It's a shame that the code requires decompilation from java to subl, then pattern matching to attempt reverse macroexpansion]

Simon
----- Reply message -----
From: "John F Sowa" <sowa@xxxxxxxxxxx>
Date: Tue, Dec 18, 2012 1:12 pm
Subject: [ontology-summit] Reasoners and the life cycle
To: <ontology-summit@xxxxxxxxxxxxxxxx>

Leo and James,

That is highly misleading as a requirement:

Leo
> Many ontology projects either have automated reasoning as a requirement
> now or they will in the future, as applications evolve and new
> applications arise to use/reuse the ontology.

James D
> If this read "Many projects which involve an ontology have/will have also
> a requirement for a reasoner ..." I'd be happy. But ontologies and
> reasoners are different things. It is plausible to include
> "reasoner-suitability" as a criterion in evaluating ontologies, certainly.
> ... But, since an ontology is not a reasoner, it's irrelevant for
> the purpose of evaluating ontologies.

Yes.  And I would emphasize just *one* of many criteria.

Much more important is *compilation* from a very general, highly
expressive ontology into formats that are appropriate for different
purposes.  Among them are database constraints, which can be stated
in full first-order logic and evaluated in polynomial time.

Cyc uses compilation methods very heavily for translating a logic
(CycL) that is even more expressive than FOL into formats that
are appropriate for different kinds of reasoning engines that
are tailored for different kinds of problems.

Cyc has been in the ontology business for 28 years, and they have
implemented the largest formal ontology on the planet.  It should
be cited as an example of a *good* ontology: very general and stated
in very expressive logic designed for ease of translation to formats
for an open-ended variety of reasoners.

That criterion does not imply that restricted logics, such as OWL DL,
are necessarily bad.  But they should be put in the category of special-
purpose ontologies designed for just one narrow kind of reasoner.

For more detail, see the following article which was published in
a journal for which Jim Hendler was the editor.  Jim said that he
liked the article very much:

    http://www.jfsowa.com/pubs/fflogic.pdf
    Fads and fallacies about logic

For another example, see the following note about a system that
successfully used an *undecidable* logic in mission critical
applications.  They did use OWL for the type hierarchy, but most
of the ontology was in a more expressive logic.

John

-------- Original Message --------
Subject: Mission Critical IT
Date: Fri, 14 Dec 2012 10:47:52 -0500
From: John F Sowa <sowa@xxxxxxxxxxx>
To: '[ontolog-forum] ' <ontolog-forum@xxxxxxxxxxxxxxxx>
CC: Michel Vanden Bossche

Dear Michel,

Thanks for the information about the SWESE system.

I'm sending this note to Ontolog Forum because it has a larger audience
than the original list, Ontology Summit.  I renamed the subject line
to a pun on the name of your company and the elusive goal that the SW
and AI in general have been groping toward -- with limited success.

Following is the paper about SWESE:

    http://www.missioncriticalit.com/pdfs/swese-2007.pdf
    Ontology Driven Software Engineering for Real Life Applications

I'd like to emphasize some of your points, but I recommend the entire
note (copied below) and the above paper.

MVB
 > The implementation is 100% Mercury (a concern for the customer), but
 > necessitates only 45,000 LOC of which 25,000 were automatically 
generated.

A requirement for mainstream IT:  generate the AI stuff *automatically*
from notations that the IT staff and the SMEs can read and write.

 > It took 5000 man.days to specify, design, implement, test and deploy.
 > A similar system, although much simpler (one product at a time, one
 > channel, etc.), built in Java EE, took 15,000 md.

For the record, Mercury is a high-speed logic-programming language:

    http://www.mercury.csse.unimelb.edu.au/information/features.html

This increase in productivity for implementing the DSL (Domain Specific
Language) is typical of Prolog and other logic programming languages.
In fact, Alain Colmerauer, who designed the first Prolog system, adapted
it from a parser he had originally written for machine translation.

 > The system was just starting in production (4000 users) when Winterthur
 > was acquired by Axa. Axa viewed our system as much too advanced, 
considering
 > that their IT staff was not competent enough to support it. AAMOF, 
Axa has
 > just cancelled a 3rd attempt to build a system like ours, using BaNCS,
 > a package from TCS (Tata Consultancy Services).  They pulled the plug
 > after spending 35 million USD.

That is a sad but familiar story.  Our company, VivoMind Research, LLC,
had similar experiences.  We implemented the system for oil and gas
exploration described in the Future Directions paper (pp 15-17):

    http://www.jfsowa.com/pubs/futures.pdf

After we developed that prototype, the oil company spoke with a larger
company that claimed they could do anything we could do.  They got a
two-year contract with a lot more money than we did.  But they failed.

 > Mercury is really great but is sociologically impossible to sell
 >
 > So, we decided to developed a Java backend for Mercury. Today,
 > Mercury  compiled in Java is even faster than compiling to C.

That is also a familiar story.  What you deliver to customers is Java.
They have heard of Java.  Their programmers will never look at the code,
but they'll feel comfortable that it's something they "know".

 > Rules (beyond OWL) are absolutely critical. We started with SWRL,
 > but  real life applications require at least aggregation and negation,
 > with a subtle balance between OWA and CWA. So, we worked on xSWRL
 > (extended SWRL, a very poor name as this is not SWRL anymore, but
 > it simplifies the marketing ;-)).

That is the major strength of the SW:  it's a powerful hype machine.
But what they hype is pitifully weak for serious applications.

 > It's a pity that W3C is not addressing the Business Rules issue
 > aggressively. RIF is like OSI: they couldn't decide between
 > Production Rules and Logic Rules (OSI couldn't decide between
 > connectionless and connectionfull).

I spoke at a conference on Business Rules.  One of the sponsors
was Experian -- a credit bureau that processes huge amounts of data
about everybody in the world.  Their major language is Prolog.  They
use it so heavily that they bought Prologia, the company founded by
Alain Colmerauer.  Experian is truly mission critical, but they're
so secretive that nobody knows how they specify their rules.

 > Same syndrome: W3C is not ready to bite the bullet, i.e. aggressively
 > support logic. They backtrack to Linked Data Object to erase the AI
 > from their story. And "Rules Interchange Format" is a terrible name.

I don't blame Tim Berners-Lee.  But I do blame the academics who rammed
decidability down the throats of people who had no clue about what it
meant.  Look at Tim's original proposal from February 2000:

    http://www.w3.org/2000/01/sw/DevelopmentProposal

Tim discussed Prolog and many other AI systems. He emphasized diversity.
And he proposed SWeLL (Semantic Web Logic Language) for communication
among heterogeneous systems.  SWeLL was intended as a superset of
propositional, first order, and higher order logic.  Unfortunately,
the final report claimed that OWL was a renamed version of SWeLL.

 > Logic gives you the possibility to debug declaratively your model.
 > This is what we do with our ODASE workbench: you can validate the
 > model with the SME before writing the first LOC.
 >
 > And, when the system runs, you can trace and explain why the system
 > works as it is.

Yes.  And to do that, you need a logic that can express anything
that the system uses or does.  But no decidable logic can.

 > With our new reasoner, we can price a car contract with more than
 > 10 coverages in 83 ms, on a standard Xeon 3.4 GHz, on one core.
 > Batch is easily scaled-out on multiple cores.
 >
 > This system is mission-critical and we move to a phase of productization.

Congratulations.  Note the time:  83 milliseconds.  That is the kind
of timing that mission critical applications require.

Decidability only guarantees *finite* times.  OWL guarantees finite
time by limiting their models to trees.  But the size of a tree grows
exponentially with the depth.  The algorithms can take longer than
the age of the universe.  That's finite, but not mission critical.

 > fundamental objective: better software requires better science first,
 > sound engineering then, and real-life experimentation with real 
customers.

Memo to W3C:  Don't standardize anything until *after* there is at least
one real live customer that is happy with the results.

John

-------- Original Message --------
Subject: Re: [ontology-summit] Potential Tracks for Ontology Summit 2013
Date: Fri, 14 Dec 2012 12:12:29 +0100
From: Michel Vanden Bossche

Dear John,

Thanks for your mail to Michael [Uschold] and thanks to Michael for
copying me.

First, I read many of your papers and I share your views. The last
paper I read is "Future Directions for Semantic Systems".

 > I repeat the question:  What mission-critical applications has anybody
 > developed with that platform?  Are they actually being used on a daily
 > basis?  For any significant time (at least a year) after the software
 > was delivered?

1. SWESE system

The system described in our SWESE paper was really "mission-critical".
That project started in July 2004 and was completed, successfully in
October 2007. It is an eInsurance system with the following objectives:

- customer centric
- multi products
- multi channels
- fast turn around time of new products (2 weeks max)
- fully web
- < 3 sec response time for pricing (8 products, 5 variants each)
- etc.

It is ontology driven, using OWL (just released in February 2004). The
ontology defines the P&C (Property and Casualty) insurance domain and
insurance products as instances of product definitions in the
ontology. The UI is Ajax (we used a kind of JSP for Mercury that we
named MSP). The navigation is defined, in the ontology, using YAOWL, a
variant of YAWL (van der Aalst), a CPN based workflow. RBAC (Role
Based Access Control) defines Roles and Powers ontologically.

Rating/pricing is executed by a DSL that was imposed by the customer
(Winterthur) so that pricing in the mid-tier (our system) was
identical with the mainframe pricing (zOS, COBOL, DB2). We couldn't
price on the mainframe calling services because the mainframe died due
to the many CICS calls related to multiple products and multiple
variants. We reimplemented the DSL written in COBOL in Mercury (it
took us 3 weeks) and the response time went from more than 1 minute to
350 ms (on an HP Intel Xeon Box of that time).

The implementation is 100% Mercury (a concern for the customer), but
necessitates only 45,000 LOC of which 25,000 were automatically generated.

It took 5000 man.days to specify, design, implement, test and deploy.
A similar system, although much simpler (one product at a time, one
channel, etc.), built in Java EE, took 15,000 md.

The system was just starting in production (4000 users) when
Winterthur was acquired by Axa. Axa viewed our system as much too
advanced, considering that their IT staff was not competent enough to
support it. AAMOF, Axa has just cancelled a 3rd attempt to build a
system like ours, using BaNCS, a package from TCS (Tata Consultancy
Services). They pulled the plug after spending 35 MUSD.

2. Lessons learned

a. Mercury is really great but is sociologically impossible to sell

So, we decided to developed a Java backend for Mercury. Today, Mercury
compiled in Java is even faster than compiling to C. This illustrates
the fact that a great language with a great compiler which can
leverage fundamental CS research (e.g. Abstract Data Interpretation) wins.

Since then, we have also developed a C# backend, which works fine but
is not yet as mature.

We have also built, as an exercise, an Erlang backend. This one is
only alpha.

We think that we need three layers:

- logic at the modeling level
- logic programming
      to consume the model
      to mitigate with the accepted languages of the day
- Java, C# today and something else in the future (unknown)

The problem is to transcend time.

b. Code generation

We recognized the importance of combining code generation (Business
API in Java or C# to access the ontology) and interpretation.

c. Rules

Rules (beyond OWL) are absolutely critical. We started with SWRL, but
real life applications require at least aggregation and negation, with
a subtle balance between OWA and CWA. So, we worked on xSWRL (extended
SWRL, a very poor name as this is not SWRL anymore, but it simplifies
the marketing ;-)).

We observed that Pellet performance for SWRL was not adequate. Relying
on a PR engine (JESS...) was not an option because we would loose
declarativeness. So, we built our own reasoners, trading completeness
for performance ; this is not a problem in real-life. As usual, sound
engineering requires a good balance between science and pragmatism.

We now exceed the performance requirements imposed on us by our last
customer (see later).

It's a pity that W3C is not addressing the Business Rules issue
aggressively. RIF is like OSI: they couldn't decide between Production
Rules and Logic Rules (OSI couldn't decide between connectionless and
connectionfull).

Same syndrom: W3C is not ready to bite the bullet, i.e. aggressively
support logic. They backtrack to Linked Data Object to erase the AI
from their story. And "Rules Interchange Format" is a terrible name.

Silk is going in the right direction, but it's maybe too ambitious for
the moment.

d. Process

Ontologies are "Sein" but real applications require "Sein und Zeit".
Again, besides OWL-S (a very imperative approach), W3C is not
addressing that issue.

We moved from the imperative approach (CPN) to a declarative approach
combining OWL, xSWRL and LTL. In practice, may processes issues are
related to Data flow problems, much less than to Control flow problems.

e. Model "debugging"

Logic gives yoiu the possibility to debug declaratively your model.
This is what we do with our ODASE workbench: you can validate the
model with the SME before writing the first LOC.

And, when the system runs, you can trace and explain why the system
works as it is.

f. IT ego

There is a really big problem with IT people. Their expertise is like
the one of the Byzantine priests. They know everything related to
their practical domain of expertise, but if they don't know something
(logic, OWL...) they'll fight like hell to avoid being not experts
anymore. So, they are ultra conservatives under the costume of modernists.

This is a difficult issue. We have the feeling that we can now deal
with them without antagonizing them too much ;-)

3. Last project

We just completed the first phase of a project for AVIVA (another
insurance company): externalizing the product definition and the
pricing (product factory and rating engine).

With our new reasoner, we can price a car contract with more than 10
coverages in 83 ms, on a standard Xeon 3.4 GHz, on one core. Batch is
easily scaled-out on multiple cores.

This system is mission-critical and we move to a phase of productization.

We are now investigating a parallel implementation of our reasoners. A
language like Mercury makes that much less complex than Java or C#
(maybe Java 8 will change that).

4. Conclusion

I started the "logic dream" in 1978 with Prolog. We moved to Mercury
in 1995. We had our first commercial application written in Mercury
deployed in 2000. It is not really ontology driven (although there is
a DAML like model), there are no rules (à la SWRL), but there is a CPN
at the heart. It still works after many changes, with a TCO that is 5
times better than similar systems (Case Management). Our first ODIS
was built experimentally in 2003 and our first commercial system
delivered in 2006. And we are still at the beginning...

The biggest issue is "Software Innovation": IT people are frighten.
Sales and Marketing is a NP complete problem.

So, it needs time, a lot of time and the capability of not deviating
from the fundamental objective: better software requires better
science first, sound engineering then, and real-life experimentation
with real customers.

Tough... and you don't have VC happy with a 30 year business plan.

Hope it helps,

Michel


 
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/   
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013  
Community Portal: http://ontolog.cim3.net/wiki/ 




_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/   
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/  
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2013/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013  
Community Portal: http://ontolog.cim3.net/wiki/     (01)