[Top] [All Lists]

Re: [ontolog-forum] Ontologist Aptitude Test?

To: "[ontolog-forum]" <ontolog-forum@xxxxxxxxxxxxxxxx>
From: John Bottoms <john@xxxxxxxxxxxxxxxxxx>
Date: Fri, 18 Dec 2009 14:02:02 -0500
Message-id: <4B2BD1AA.4040803@xxxxxxxxxxxxxxxxxx>
Joel,    (01)

The primary purpose of testing is to bin students into scores of 
increasing difficulty. This entails the need to determine the difficult 
of the questions used in the test. This can be done by interview, essay 
or multiple choice once the difficulty of each test item is know.    (02)

The most readily process available for the creation of items is to have 
them suggested by domain experts who are looking at the most common 
misconceptions, coverage and salience. From these and other sources they 
create the items. At this point is is unlikely that the domain experts 
understand the difficulty of each item. This information is then 
determined by a field trial that compares the performance of each items 
against the broad responses of the students tested in the field trial.    (03)

As test items are evaulated atainst the field test norms, items are 
discarded that do not perform as desired. In the development of the 
early tests it might be useful to use a larger number of essay questions 
in order to determine coverage and salience. Essay or interview testing 
typically takes as much time to evaluate as it does to administer the 
test. So, an hour test will require an hour of analysis for each test.
Later tests may move to more efficient methods of testing such as 
multiple response. After the field trials the test items are returned to 
the domain experts for review.    (04)

In determining the test items the domain experts need to consider 
whether they are testing for practical knowledge of existing tools or 
methodologies versus validating an understanding of the conceptual 
understanding of ontologies and their use in systems. My preference is 
for a mixture of both the "appliance operator" approach and the
"appliance designer" approach since both are required in most ontology work.    (05)

Suggested items (these assume an essay response)
1. Describe the differences between taxonomies, categories and ntologies.    (06)

2. Describe the components of an ontology and how each is used in 
systems. Include a description of the type of system used as the example.    (07)

3. Discuss the difference between FOL and higher logic in the 
architecture of a system that uses ontologies.    (08)

4. Describe the contributions of Aristotle in the development of 
knowledge systems and comment on their relevance today.    (09)

5. Describe tools that can be used in the creation and maintenance of an 
ontology.    (010)

6. Explain how graphs can be used in problem solving associated with 
ontologies. What impact does this have on the design of the ontology.    (011)

7. Explain how an ontology is updated and the issues that may arise 
during the update.    (012)

There, that should take an hour! These are just off the top of my head 
but may give you some starting points.    (013)

-John Bottoms
  (past adjunct prof. Graduate School of CS, Northeastern Univ.)    (014)

Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx    (015)

<Prev in Thread] Current Thread [Next in Thread>