[Top] [All Lists]

Re: [ontolog-forum] Fwd: MOVED: Re: [ontology-summit] Hackathon: BACnet

To: "[ontolog-forum]" <ontolog-forum@xxxxxxxxxxxxxxxx>
From: "Barkmeyer, Edward J" <edward.barkmeyer@xxxxxxxx>
Date: Thu, 21 Mar 2013 01:07:31 -0400
Message-id: <63955B982BF1854C96302E6A5908234417DC9B29DA@xxxxxxxxxxxxxxxxxxxxxxxxxx>
John,    (01)

I read this clarification to say that two humans of reasonable education and 
intelligence who read a text written by a third will agree on the 
interpretation of about 50% of what was written.
Or perhaps that they will be unable to write down what they think the text said 
in such a way that it is easy to determine whether they agree.  It seems to me 
that all of that may be true, and all of it is irrelevant to processing 
restricted natural language.      (02)

 It is easy to process unrestricted NL if the penalty for misinterpretation is 
low, e.g., searches will miss useful information or include irrelevant 
information.  If your objective is the formal capture of the writer's intent, 
and there is a substantial penalty for misinterpretation (!), e.g. erroneous 
activation of some process, or use of structurally unsound building materials, 
it is not easier or better to process unrestricted NL.  The reason for formal 
languages is to phrase things in such a way that the interpretation is 
unambiguous.    (03)

The view that my project takes is that restricted NL is written by knowledge 
engineers who understand how to write it and what it means.  The idea of the 
restricted NL is that the statements written by the knowledge engineers can be 
READ by the domain expert, so that s/he can VERIFY that the captured intent 
matches his/her intent.  That the restricted NL sentences then have a formal 
logic interpretation that is generated by the RNL compiler completes the 
circuit between the intent of the 'domain expert/practitioner' and the formal 
logic text used directly by the reasoning machine.  (This is just one more of 
my assertions that knowledge engineering is a discipline practiced by trained 
engineers.  The idea that a health care specialist or finance specialist or 
automotive engineer can easily use some black magic NL tool to capture his/her 
intent formally is hogwash, and always has been a goal not worth pursuit.)  The 
problem with knowledge engineering is the same problem as with software 
engineering -- the expert in the domain can't read and understand the formal 
language, and thus the transfer of intent relies on the communications skills 
of both parties. Restricted natural language creates a vehicle by which the 
knowledge engineer can capture his understanding of the intent of the domain 
expert, and the domain expert can read what he captured.  The knowledge 
engineer must learn to write the language, and it should be a subset of natural 
language that is readable by the domain expert.  If some specifications are 
hard to express, they were doubtless hard to express in "unrestricted" natural 
language as well, and it may be necessary to factor them into several simpler 
specifications, and perhaps introduce intermediate concepts.    (04)

You might be amused to know that the aerospace industry developed a standard 
for how to write clear requirements in a "restricted" natural language, for the 
purpose of conveying requirements clearly to other engineers.  Apart from a 
requirement to define your terms, the specification suggests a base vocabulary 
with definitions, and provides a number of rules for simply writing unambiguous 
English for different categories of requirements.  It does not define a 
controlled English that one could guarantee to process into formal logic, but 
it is likely that high quality NLP tooling that incorporated heuristics based 
on the guidelines in the standard would have a very high rate of correct 
interpretation.  The intent of the standard is that human engineers of 
reasonable education and intelligence will also have a very high rate of 
correct interpretation, precisely because misinterpretation can have a very 
high cost.    (05)

See: http://www.asd-ste100.org/    (06)

-Ed    (07)

Edward J. Barkmeyer                       Email: edbark@xxxxxxxx
National Institute of Standards & Technology
Engineering Laboratory -- Systems Integration Division
100 Bureau Drive, Stop 8263               Office: +1 301-975-3528
Gaithersburg, MD 20899-8263               Mobile: +1 240-672-5800
From: ontolog-forum-bounces@xxxxxxxxxxxxxxxx 
[ontolog-forum-bounces@xxxxxxxxxxxxxxxx] On Behalf Of sowa@xxxxxxxxxxx 
Sent: Wednesday, March 20, 2013 6:14 AM
To: [ontolog-forum]
Subject: Re: [ontolog-forum] Fwd: MOVED: Re: [ontology-summit] Hackathon: 
BACnet Ontology    (08)

Ali,    (09)

I was trying to be brief.  I'll try to clarify the point.    (010)

>> I believe that it is easier to process unrestricted NL as written by
>> humans who are writing for other humans than it is to correct the errors
>> in the artificial languages written by humans who are writing for
>> machines.    (011)

> Do you mean that it would be easier to process by machines as well? Easier
> to process by whom and how?    (012)

Short answer:  the  error rates are so high that it's useless for both 
computers and people to try to make sense out of what they get.    (013)

When highly trained professional annotators add semantic markup to texts, a 
level of agreement among annotators of 95% is unusually high.  In most cases, 
it's much lower.    (014)

When people with a typical college degree and a modest amount of instruction 
try to annotate texts according to some standard, 50% agreement is high.  In 
many cases, flipping a coin would give comparable results.    (015)

I have very little faith in those annotations and even less faith in the 
attempts by most people to express what they're trying to say in any artificial 
notation.    (016)

I admit that people can learn to write computer programs -- but that is only 
because the computer is an unforgiving taskmaster.  People either give up or 
they persist until they are rewarded by getting something useful from the 
machine.    (017)

John    (018)

Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J    (019)

<Prev in Thread] Current Thread [Next in Thread>