Dear John B,
You wrote:
I'm Skeptical,
A total of 103 references for
a 16 page paper seems a bit out of whack. And his approach relies on inputs to
the NN's that, "A biologically-plausible deep learning algorithm should
take advantage of the particularities of the reinforcement learning setting."
That amounts to saying that an appropriately sized corpus is required.
Yes, it sounds like news blather. And his writing is more
opaque than appropriate.
Further, it is always a red
flag for me is when someone coins a new term or acronym such as
"KICKS", when there are better ways to explain the algorithm. I
believe he presents a reasonable overview of some useful techniques that would
be better conceived using semantics.
But there are no semantics in the voice input stream. It's
just numbers from the analog converter as sampled every so often. To get
to what has become commonly called semantics means to interpret those numbers,
and semantics can indeed help with the error feedback and such. But it
can't do what the NN NLP authors purport to do, which is to deal with the
analog signals and convert them into lexical or dictionary units, often words and
phrases.
The problem is the interpretive interface between the number
series and the
The front end problem is to do pattern recognition so that
you can identify many of the sounds as plosives, fricatives, vowels, whatthehellatives,
... before turning them into words, where semantics analysis of the
accumulating situation knowledge would finally provide the basis on which
semantics can work. Only after that point is semantics appropriate to the
feed forward direction of processing.
Semantics can help. The
reason linguistics provides valuable tools is because natural languages have
undergone thousands of years of adaptation and pruning that embody the deep
learning of the members of the community of interest. We should take advantage
of that learning in a semantic fashion, rather than trying to recreate those
lessons. Humans can provide the right answers if we carefully ask the correct
questions.
-John Bottoms
Concord, MA USA
That is absolutely true; +1. But what we have now
includes dictionaries, thesauri, WordNet, VerbNet, ... , and lots of embodiments
of resources at the lexical level and above. Not at the analog level, to
my knowledge (please correct me if you know otherwise) that can be used to
train these NN NLP machines further.
But the machines that hold a credible dialog with us, a la the
Turing Test, can presently be counted on no fingers. Its zero.
Since we haven't yet got to the depths in how language works for
people, let's keep ALL resources working that provide improvements, as compared
to its objectives. Just keep the budget in reasonable limits and keep
pruning out the non productive approaches until only productive ones remain.
Because, futile as this search has been for so long, a
linguistically competent Q&A agent is essential to get really deep into
artificial intelligence. Once it becomes fairly available to obsessive
users, it will also take one heck of a discovery system, but I already showed how
to do that in the 7,209,923.
Sincerely,
Rich Cooper,
Rich Cooper,
Chief Technology Officer,
MetaSemantics Corporation
MetaSemantics AT EnglishLogicKernel DOT com
( 9 4 9 ) 5 2 5-5 7 1 2
http://www.EnglishLogicKernel.com
From: ontolog-forum-bounces@xxxxxxxxxxxxxxxx
[mailto:ontolog-forum-bounces@xxxxxxxxxxxxxxxx] On Behalf Of John
Bottoms
Sent: Sunday, October 25, 2015 1:08 PM
To: ontolog-forum@xxxxxxxxxxxxxxxx
Subject: Re: [ontolog-forum] Semantics, Representations and Grammars for
Deep Learning - David Balduzzi
I'm Skeptical,
A total of 103 references for a 16 page paper seems a bit out of whack. And his
approach relies on inputs to the NN's that, "A biologically-plausible deep
learning algorithm should take advantage of the particularities of the
reinforcement learning setting." That amounts to saying that an
appropriately sized corpus is required.
Further, it is always a red flag for me is when someone coins a new term or
acronym such as "KICKS", when there are better ways to explain the
algorithm. I believe he presents a reasonable overview of some useful
techniques that would be better conceived using semantics.
Semantics can help. The reason linguistics provides valuable tools is because
natural languages have undergone thousands of years of adaptation and pruning
that embody the deep learning of the members of the community of interest. We
should take advantage of that learning in a semantic fashion, rather than
trying to recreate those lessons. Humans can provide the right answers if we
carefully ask the correct questions.
-John Bottoms
Concord, MA USA
On 10/25/2015 3:32 PM, Rich Cooper wrote:
Dear OntoLogicists,
The subject says it all. This is a 17 page paper with 3
pages more in references about how the neural net crowd seems to have
straightened a curve in getting to natural language in a deeper sense than just
voice recognition. The paper is at:
http://arxiv.org/pdf/1509.08627.pdf
The author seems to have the EE math culture viewpoint leading
to some very interesting ways to learn such odd things as those discussed in Women,
Fire and Dangerous Things by exposure to a large enough number of
samples.
Does anyone have a good tutorial in mind on recent NN to NLP
practice?
Sincerely,
Rich Cooper,
Rich Cooper,
Chief Technology Officer,
MetaSemantics Corporation
MetaSemantics AT EnglishLogicKernel DOT com
( 9 4 9 ) 5 2 5-5 7 1 2
http://www.EnglishLogicKernel.com