John, (01)
> That is most definitely false. (02)
Wow! Whenever I've asked somebody that said they've improved something,
they've always given me some measure. If they are about to work on a problem,
I've always asked them "how do you know when you've made any progress?"
Sometimes they hadn't though much about how to measure how effective their work
will be. (03)
How do *you* measure success? (04)
> As for ontology, this forum shows that the field itself is extremely
>unsettled, and there is no consensus on what a good ontology is or how it
>could be measured. (05)
I'll grant you that ontology as a field is unsettled, but it's not without
descriptive statements that say what it is. Are there no questions that can be
asked that test a persons knowledge of those statements? (06)
Any specific ontology can only be measured as "good" against a specific use
case, and if there is something that is better, there must be a reason it is
better. (07)
> I was thinking of a team of people that I would hire to solve a problem. And
>passing a multiple choice test gives ZERO evidence of problem solving ability. (08)
A wild overgeneralization which tells me you have no confidence in the way
tests are built. I had a physics test that was all multiple choice; the
answers were similar to (a) 73.5, (b) 73.6, (c) 73.7, (d) 89.7, (e) None of the
above. The chances of 1 in 5 of getting the right answer (and there were a lot
of questions) was enough for the professor and saved enormous time reviewing
answers! The answers to essay questions are easily filled with puffery, which
is subject to vast chasms of interpretation, and why engineers have such a hard
time dealing with English majors. :-) (09)
Besides, I wasn't specifically limiting my quest for possible questions to only
multiple choice ones. (010)
> I certainly would not hire somebody whose primary credential was the ability
>to pass a multiple-choice test. (011)
True, and I wouldn't hire anybody without at least a baccalaureate degree, and
I would expect their degree program to include other kinds of questions as well. (012)
> The best criterion is successful performance on real jobs. Second best is an
>interview with somebody in the field. A test is almost worthless. (013)
Please don't say that to your pharmacist. Later you write: (014)
> Doug Lenat did use a simple test, but only to weed out the incompetent. (015)
Weeding out the incompetent is a useful process, it allows 90% of the
applicants to be flushed. Anything that saves me time and money in finding
somebody that can solve my problem is a good thing. (016)
> Not the MCP exams that Microsoft uses. There are many very competent people
>who have an MCP certificate. But that is because they took the exam in self
>defense to avoid looking worse than the idiots whose only knowledge is the
>content of the MCP books. (017)
Like it or not, the industry has developed these tests specifically for that
purpose, and people that don't bother to take those tests *do* look worse than
the idiots, even if they're not. They are being measured. Maybe that specific
measuring tool isn't so great. Maybe they *are* better than all of the other
applicants that have taken the test, but am I really supposed to go through all
of my applicants, hunting for that special someone? (018)
You should take a look at the CCNE process from Cisco and their tests, they are
far from worthless. Please don't throw out the test and certification process
baby with the Microsoft bath water. (019)
> JB> ... the design and development teams saying "we need an ontologist"
>> or he/she goes to lunch with a colleague that says "we've done so much
>> better after we hired an ontologist".
>
> On what planet have you ever heard anybody say anything like that? (020)
I haven't heard that, but are you implying that nobody on this planet has ever
said anything like that? That doesn't bode well for this particular field of
study, and I also don't believe it. There exists somebody somewhere that has
played the role of ontologist and made some improvement to something, and that
improvement was measured and documented. That role involved skills beyond the
systems analyst, maybe it included philosophy, biology, or epistemology. What
were those skills? How do I know you've got them? (021)
> I would ask them the same question that Lou Gerstner asked the VPs who
>reported to him when he was named the CEO of IBM. (022)
Another good "team building" scenario, but I don't need a VP. I don't need an
athlete, I need a wide receiver, and if they can't do a 100 yard dash in X
seconds, they're out. If they ran better than that at the last track meet, and
I trust the time keepers, I don't have to test them for that. They were
measured. If they say their times have improved, that was measured too, or I
say they haven't. (023)
Maybe I need a new kind of person, a lot like the systems analysts that have
been around for 30 years, but someone with a different perspective. (When I
worked for a small consulting firm we had a psychiatrist on staff, not for his
computer skills, but because he could connect with the customers in a way the
technical staff couldn't.) (024)
Maybe I need an ontologist. (025)
> He gave all prospective candidates a list of six sentences in English, which
>he asked them to translate to FOL.
> But this test was just to verify a minimal competence. It didn't produce a
>number that he used to rank the candidates. (026)
Excellent! If they couldn't answer any of the questions, they were in serious
trouble. If they answered them all, they were clearly on to the next round of
testing. If the sentences were carefully constructed by people that understood
some of the traps that people doing English -> FOL translation can fall into,
and the responses were independently verified as accurate and complete, that
would be even better, I could skip that part of the interview. (027)
If they don't want to be bothered with taking the test then I have more than
enough applicants that are, and they're just doing me a favor because I don't
want somebody with that attitude. (028)
Joel (029)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx (030)
|