John
Great questions
to some extent, answers may be canonical - there are standard generic knowledge sources
but largely I would say that there are several schools of thought with differring answers
would the scores differ if answer is according to school a as opposed to school b?
would the right answer be in the eye of the person marking the essay, or would you provide
guidlines of how to mark these?
Some of these answers I would probably address saying
according to x answer is y
according to z answer is ...
etc
but that may require having the citations at hand when writing the essay
no?
PDM
On Fri, Dec 18, 2009 at 8:02 PM, John Bottoms <john@xxxxxxxxxxxxxxxxxx> wrote:
Joel,
The primary purpose of testing is to bin students into scores of
increasing difficulty. This entails the need to determine the difficult
of the questions used in the test. This can be done by interview, essay
or multiple choice once the difficulty of each test item is know.
The most readily process available for the creation of items is to have
them suggested by domain experts who are looking at the most common
misconceptions, coverage and salience. From these and other sources they
create the items. At this point is is unlikely that the domain experts
understand the difficulty of each item. This information is then
determined by a field trial that compares the performance of each items
against the broad responses of the students tested in the field trial.
As test items are evaulated atainst the field test norms, items are
discarded that do not perform as desired. In the development of the
early tests it might be useful to use a larger number of essay questions
in order to determine coverage and salience. Essay or interview testing
typically takes as much time to evaluate as it does to administer the
test. So, an hour test will require an hour of analysis for each test.
Later tests may move to more efficient methods of testing such as
multiple response. After the field trials the test items are returned to
the domain experts for review.
In determining the test items the domain experts need to consider
whether they are testing for practical knowledge of existing tools or
methodologies versus validating an understanding of the conceptual
understanding of ontologies and their use in systems. My preference is
for a mixture of both the "appliance operator" approach and the
"appliance designer" approach since both are required in most ontology work.
Suggested items (these assume an essay response)
--------------------------------------------
1. Describe the differences between taxonomies, categories and ntologies.
2. Describe the components of an ontology and how each is used in
systems. Include a description of the type of system used as the example.
3. Discuss the difference between FOL and higher logic in the
architecture of a system that uses ontologies.
4. Describe the contributions of Aristotle in the development of
knowledge systems and comment on their relevance today.
5. Describe tools that can be used in the creation and maintenance of an
ontology.
6. Explain how graphs can be used in problem solving associated with
ontologies. What impact does this have on the design of the ontology.
7. Explain how an ontology is updated and the issues that may arise
during the update.
There, that should take an hour! These are just off the top of my head
but may give you some starting points.
-John Bottoms
(past adjunct prof. Graduate School of CS, Northeastern Univ.)
--
Paola Di Maio
**************************************************
“Logic will get you from A to B. Imagination will take you everywhere.”
Albert Einstein
**************************************************
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx (01)
|