Rich Cooper wrote:
> Hi Ed,
>
> The reason I used an ROI analysis is that ROI is the key decision driver for
> a company choosing to fund one R&D project compared to funding another
> candidate project. The database administrator isn't R&D; she's overhead G&A
> because all parts of the business are exposed in the database, so ROI
> analysis isn't an appropriate justification for her value/cost analysis to
> the business. For Ms Database, the minimum cost with least risk exposure is
> a decision predicate for when to keep her and when to give out the pink
> slips to all database expenses.
>
> But for a private company, R&D funding is paid for by the stockholders,
> through reinvestment of profits within tax regulations. They want to know
> what ROI to expect on every dollar they reinvest, what the project cash flow
> looks like monthly on the revenues versus expenses for R&D project schedule,
> and other methods they can use to explain their budgets to their
> stockholders. (01)
Yes. What I said is, you don't get to talk about RoI for the fraction
of the project that is capturing someone's domain knowledge. The
stockholders will neither understand that, nor care about it. You talk
about RoI for the project, which is re-engineering some part of your
business, and may or may not be called R&D. And then you judge what
approach and allocation of resources maximizes the probability of
project success relative to cost and budget. Your critique of Mike's
and my view of resource allocation was based on your presumption that
one of the domain expert and the knowledge engineer can educate himself
in the other's trade well enough to produce a high probability of
success at clearly lower cost. And it is my experience that the
standard deviation on that probability of success is very large using
your proposed approach. So if you count sigmas, you are talking about a
crap shoot. (02)
> That is simply the private company version of what you had to
> explain to NIST auditors showing how the money was spent. But in NIST's
> case, there is no direct way to measure R&D due to the enormous complexity
> of the federal budget.
> (03)
NIST management doesn't answer for the federal budget, it answers for
600M$ of it (0.03%), and my management answers for a small fraction of
that (50 people out of 3300). My management only cares what my unit's
program of work accomplishes with its 5 million dollars (inclusive of
overheads), and they need to understand how a knowledge engineering
project benefits the manufacturing and/or construction industries,
because that is our mission. (04)
My problem is capturing agreements about supply chain behavior and
information exchange, so that you can use software to detect missing and
contradictory information before it results in a serious delivery foulup
(like a shipment sitting on a dock in Los Angeles or Liverpool for 4
weeks waiting for a correct customs form). A colleague is using
knowledge engineering to effect exchange of information on the electric
power grid, which has agreed to use 3 different standards to carry
essentially the same information (thereby demonstrating the relative
efficiency of industrial decisions over governmental ones). And we
haven't found that freight forwarders and power system operators are up
to learning knowledge engineering technologies. (05)
You are quite right, however, that there is no real way to measure the
impact of R&D in a short period of time, or in isolated projects. It is
like investing in penny stocks -- you have to kiss a lot of frogs to
find a prince. (06)
-Ed (07)
--
Edward J. Barkmeyer Email: edbark@xxxxxxxx
National Institute of Standards & Technology
Manufacturing Systems Integration Division
100 Bureau Drive, Stop 8263 Tel: +1 301-975-3528
Gaithersburg, MD 20899-8263 Cel: +1 240-672-5800 (08)
"The opinions expressed above do not reflect consensus of NIST,
and have not been reviewed by any Government authority." (09)
> -Rich
>
> Sincerely,
> Rich Cooper
> EnglishLogicKernel.com
> Rich AT EnglishLogicKernel DOT com
> 9 4 9 \ 5 2 5 - 5 7 1 2
> -----Original Message-----
> From: ontolog-forum-bounces@xxxxxxxxxxxxxxxx
> [mailto:ontolog-forum-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Ed Barkmeyer
> Sent: Friday, March 11, 2011 9:49 AM
> To: Rich Cooper
> Cc: [ontolog-forum]
> Subject: Re: [ontolog-forum] Using controlled natural languages for ontology
>
>
> Rich,
>
> you wrote:
>
>> John, Simon, Ed, Godfrey et al,
>>
>>
>>
>> Putting together all your comments, I conclude the following:
>>
>>
>>
>> The knowledge that gets represented in 90% of the KE/DE projects
>> covers about one hour of the domain expert's knowledge and one hour of
>> the knowledge engineer's modeling skills, though the two are still
>> askew until another hour of debate and explanation, during which each
>> expert educates the other about the most basic items of knowledge
>> which represents the least common denominator of each expert's wisdom.
>>
>>
>>
>> If you give some thought to the precise phrasing I used, the point may
>> get across that the two "experts" are paid three hours to record just
>> one hour of knowledge as ROI for the time spent. A less well paid
>> nonexpert can then work three times as long to get one hour's work
>> done after s/he studies the recorded hour of knowledge for a few hours.
>>
>>
>>
>> That doesn't sound like a successful project to me.
>>
>>
>
> Interesting. If you give similar thought to it, a Database
> Administrator is paid a handsome salary to do nothing but ensure that
> the computer systems maintain the limited knowledge of the business that
> other experts and practitioners commit to it. It is a purely supportive
> function (in the SCOR terminology) and is not on any revenue-producing
> path. So does his job and his salary have no RoI?
>
> Making ontologies to capture knowledge is not an end in itself, and has
> no RoI per se. It is a necessary step on some path that leads to
> effective use of that knowledge without the further involvement of the
> domain expert. And the presumption is that the RoI comes from that
> effective use.
>
>
>> I would just let the nonexpert work without the extra knowledge
>> indoctrination, at which point s/he produces one hour's work per hour
>> of pay if the same principles of analysis hold for these two paragraphs.
>>
>>
>
> Well, you are not alone. I know a number of companies who have paid
> 'non-experts' to do just that, and gotten highly varied results, ranging
> from the ridiculous to the sublime. Sometimes a non-expert can learn
> quickly and have valuable insights and produce wonders, and sometimes a
> non-expert can learn half of what is needed and do something at best
> clumsy and at worst noxious. NIST itself has contracted for at least
> two administrative support products that fall into this latter category,
> and for this reason, that the software engineers only generally
> understood the application.
>
> There is an old military adage that a job that requires one competent
> O-4 (major) is better assigned to three O-3s (captains) in the
> expectation that one of them will be able to do the job well, whereas an
> arbitrary O-4 has only a 50-50 chance. ;-) It is all about the
> statistical distribution of learning and insight.
>
>
>> So the ROI doesn't justify it. Its not simply that it doesn't meet
>> some arcane metric of goodness for ontologies, or knowledge, or
>> slickness of presentation, or rigidity of logical premises; it simply
>> doesn't pay off.
>>
>>
>
> There is no payoff in creating an ontology. The RoI comes from the
> value of its usage. By their fruits shall ye know them...
>
> -Ed
>
> P.S. You are arguing business case with someone who has to provide one
> every year, if he is to continue doing knowledge engineering at all.
> NIST is not an academic institution, visible similarities
> notwithstanding. ;-)
>
>
>
>> -Rich
>>
>>
>>
>> Sincerely,
>>
>> Rich Cooper
>>
>> EnglishLogicKernel.com
>>
>> Rich AT EnglishLogicKernel DOT com
>>
>> 9 4 9 \ 5 2 5 - 5 7 1 2
>>
>>
>>
>> -----Original Message-----
>> From: ontolog-forum-bounces@xxxxxxxxxxxxxxxx
>> [mailto:ontolog-forum-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Godfrey Rust
>> Sent: Thursday, March 10, 2011 10:12 AM
>> To: [ontolog-forum]
>> Subject: Re: [ontolog-forum] Using controlled natural languages for
>> ontology
>>
>>
>>
>> Spot on, Mike. Knowledge engineer needs to know just enough about the
>>
>> domain, and domain expert needs to know just enough about the modelling
>>
>> methods for you to overlap just enough in the middle. It is time
>> consuming
>>
>> and usually hard work, but it certainly pays off. When they don't overlap
>>
>> enough, a great deal of money and opportunity can eventually vanish down
>>
>> that gap.
>>
>>
>>
>> Godfrey
>>
>>
>>
>> ----- Original Message -----
>>
>> From: "Mike Bennett" <mbennett@xxxxxxxxxxxxxxx>
>>
>> To: "[ontolog-forum]" <ontolog-forum@xxxxxxxxxxxxxxxx>
>>
>> Sent: Thursday, March 10, 2011 6:04 PM
>>
>> Subject: Re: [ontolog-forum] Using controlled natural languages for
>> ontology
>>
>>
>>
>>
>>
>>
>>> Well I would put it the other way around - a knowledge engineer
>>>
>>> working with domain experts. I have managed to achieve some
>>>
>>> results with this. The key is for the knowledge engineer to
>>>
>>> recognize that they are not an expert, but that they know enough
>>>
>>> to put forward a model which is largely complete and correct
>>>
>>> (what the techie folks often call a "strawman"), and then
>>>
>>> facilitate a session which involves changing that model, within
>>>
>>> its stated formalism, until it is a complete and correct record
>>>
>>> of the knowledge of the domain experts.
>>>
>>> For example, I just came off a session modeling loans
>>>
>>> semantically. It turns out that if we define a loan formally as
>>>
>>> "some amount of money extended by some party to some other party,
>>>
>>> under agreed terms and for a given time", then some of the things
>>>
>>> we call loans are not really this but are a commitment to make
>>>
>>> this sort of thing available in the future under certain
>>>
>>> conditions - a thing actually called a Credit Facility. So for
>>>
>>> example construction loans, student loans and many mortgage
>>>
>>> agreements are really this kind of thing and not the kind of
>>>
>>> thing formally axiomatized as a "Loan". By presenting this in a
>>>
>>> formal, structured, graphical way, yes it takes a while to get
>>>
>>> every SME to fully grasp the simplicity of the underlying set
>>>
>>> theory and the fact that this does not represent some inscrutable
>>>
>>> technical design - but to the extent that they do understand this
>>>
>>> and contribute, we end up with a representation of the domain
>>>
>>> knowledge that could not have been achieved by some "hero" techie
>>>
>>> type doing all the work. So for instance today we deleted some
>>>
>>> terms that we have been given by some data modelers, because we
>>>
>>> were able to identify formal meaningful terms that already
>>>
>>> existed in the model which matched the intended meanings of those
>>>
>>> data elements.
>>>
>>> The secret is to know just enough to present something that's
>>>
>>> nearly right, and then come to the table with humility and
>>>
>>> well-directed questions for the SMEs. And of course having a
>>>
>>> model format which does not require them to learn some language.
>>>
>>> Mike
>>>
>>> On 10/03/2011 17:21, Rich Cooper wrote:
>>>
>>>> I have yet to see a domain expert working with a knowledge engineer who
>>>>
>>>> produces quality results. The few good examples I have seen are where
>>>>
>>>> the
>>>>
>>>> knowledge engineer IS a domain expert.
>>>>
>>>> Remember that in every domain, there is no Ulysses. Every expert has
>>>>
> an
>
>>>> experience that is unique, personal, and not very communicable in
>>>>
>>>> language.
>>>>
>>>> The domain expert doesn't have the "aha" reaction of suddenly seeing a
>>>>
>>>> simplification that the knowledge expert does.
>>>>
>>>> So I disagree with this long held, but factually incorrect, assumption.
>>>>
>>>> JMHO,
>>>>
>>>> -Rich
>>>>
>>>> Sincerely,
>>>>
>>>> Rich Cooper
>>>>
>>>> EnglishLogicKernel.com
>>>>
>>>> Rich AT EnglishLogicKernel DOT com
>>>>
>>>> 9 4 9 \ 5 2 5 - 5 7 1 2
>>>>
>>>> -----Original Message-----
>>>>
>>>> From: ontolog-forum-bounces@xxxxxxxxxxxxxxxx
>>>>
>>>> [mailto:ontolog-forum-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Ed
>>>>
>> Barkmeyer
>>
>>
>>>> Sent: Thursday, March 10, 2011 8:58 AM
>>>>
>>>> To: [ontolog-forum]
>>>>
>>>> Subject: Re: [ontolog-forum] Using controlled natural languages for
>>>>
>>>> ontology
>>>>
>>>> Simon Spero wrote:
>>>>
>>>>> On Sun, Mar 6, 2011 at 11:01 PM, John F. Sowa<sowa@xxxxxxxxxxx
>>>>>
>>>>> <mailto:sowa@xxxxxxxxxxx>> wrote:
>>>>>
>>>>> On 3/6/2011 10:39 PM, Zhuk, Yefim wrote:
>>>>>
>>>>> > I'd think of CNL as an intermediate step towards ontology...
>>>>>
>>>>> It's more like an alternate notation for logic that makes
>>>>>
> comments
>
>>>>> readable by both the humans and the computer.
>>>>>
>>>>> A controlled natural language has a formally defined mapping to
>>>>>
>>>>> and from some version of logic. Its main advantage is that
>>>>>
>>>>> it can be read as if it were ordinary language.
>>>>>
>>>>> There may be some small differences in ease of reading between CNL
>>>>>
>>>>> and regular NL, but these do not appear to be important.
>>>>>
>>>>> Tobias Kuhn (until recently a student of Norbert Fuchs) has some
>>>>>
>>>>> interesting results on the understandability of controlled natural
>>>>>
>>>>> language in his dissertation (see Chapter 5 in Kuhn (2010) for info).
>>>>>
>>>>> Also, as part of his work on ACEWiki Tobias built a native java
>>>>>
>>>>> implementation of ACE, making it easier to use without having to
>>>>>
>>>>> install prolog).
>>>>>
>>>>> Simon
>>>>>
>>>>> * Tobias Kuhn. /Controlled English for Knowledge Representation/.
>>>>>
>>>>> Doctoral thesis, Faculty of Economics, Business Administration
>>>>>
>>>>> and Information Technology of the University of Zurich, 2010.
>>>>>
>>>>> [PDF
>>>>>
> <http://attempto.ifi.uzh.ch/site/pubs/papers/doctoral_thesis_kuhn.pdf>|BibTe
>
>>>> X
>>>>
> <http://attempto.ifi.uzh.ch/site/pubs/papers/bibtex/doctoral_thesis_kuhn.bib
>
>>>>> ]
>>>>>
>>>> In our experience the problem isn't intelligibility, unless the
>>>>
>>>> expressions become extraordinarily convoluted. The problem is that the
>>>>
>>>> average domain expert naturally _writes_ a different language and takes
>>>>
>>>> some training to learn to write the controlled language. Further, I
>>>>
>>>> would add, the domain expert is usually reluctant to 'waste his/her
>>>>
>>>> time' doing so. So the practice is still knowledge engineer working
>>>>
>>>> with domain expert to create the ontology. The primary advantage of
>>>>
>>>> using the CNL as a means of expression for _most of_ the ontology is
>>>>
>>>> that it allows the domain expert to read, understand and validate that
>>>>
>>>> part. I say 'most of', because there are usually technical
>>>>
>>>> considerations in the formulation of the ontology that the domain
>>>>
> expert
>
>>>> should not be expected to understand -- that is the domain of the
>>>>
>>>> knowledge engineer.
>>>>
>>>> [Experts tend to be annoyed when the CNL interpreter complains about
>>>>
>>>> what they wrote, especially since its diagnostics only usually identify
>>>>
>>>> the syntactic point(s) at which it became confused, and its guidance
>>>>
> for
>
>>>> what might have been meant is not often helpful. The worst cases,
>>>>
>>>> however, are those in which what the expert writes is unambiguously
>>>>
>>>> parsed by the CNL intepreter, but the interpretation it makes is not at
>>>>
>>>> all what was intended. My favorite recent example was:
>>>>
>>>> The surface must be contained between two planes that are 0.25mm
>>>>
>>>> apart.
>>>>
>>>> The CNL interpreter understood the constraint to refer to two distinct
>>>>
>>>> instances of a class of object described as 'plane that is-apart by
>>>>
>>>> 0.25mm'! We needed to have the ontology in place to determine that
>>>>
> that
>
>>>> interpretation was not comprehensible (there is no such binary
>>>>
>>>> relation). And OBTW, the correct expression of that rule in the CNL
>>>>
> was
>
>>>> 'extraordinarily convoluted'.]
>>>>
>>>> -Ed
>>>>
>>> --
>>>
>>> Mike Bennett
>>>
>>> Director
>>>
>>> Hypercube Ltd.
>>>
>>> 89 Worship Street
>>>
>>> London EC2A 2BF
>>>
>>> Tel: +44 (0) 20 7917 9522
>>>
>>> Mob: +44 (0) 7721 420 730
>>>
>>> www.hypercube.co.uk
>>>
>>> Registered in England and Wales No. 2461068
>>>
>>> _________________________________________________________________
>>>
>>> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
>>>
>>> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
>>>
>>> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
>>>
>>> Shared Files: http://ontolog.cim3.net/file/
>>>
>>> Community Wiki: http://ontolog.cim3.net/wiki/
>>>
>>> To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
>>>
>>> To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx
>>>
>>> ______________________________________________________________________
>>>
>>> This email has been scanned by the MessageLabs Email Security System.
>>>
>>> For more information please visit http://www.messagelabs.com/email
>>>
>>> ______________________________________________________________________
>>>
>>
>>
>>
>>
>>
>>
>>
>> _________________________________________________________________
>>
>> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
>>
>> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
>>
>> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
>>
>> Shared Files: http://ontolog.cim3.net/file/
>>
>> Community Wiki: http://ontolog.cim3.net/wiki/
>>
>> To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
>>
>> To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx
>>
>>
>>
>>
>
> --
> Edward J. Barkmeyer Email: edbark@xxxxxxxx
> National Institute of Standards & Technology
> Manufacturing Systems Integration Division
> 100 Bureau Drive, Stop 8263 Tel: +1 301-975-3528
> Gaithersburg, MD 20899-8263 Cel: +1 240-672-5800
>
> "The opinions expressed above do not reflect consensus of NIST,
> and have not been reviewed by any Government authority."
>
>
> _________________________________________________________________
> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> Shared Files: http://ontolog.cim3.net/file/
> Community Wiki: http://ontolog.cim3.net/wiki/
> To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
> To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx
>
>
> (010)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx (011)
|