Objective testing
An objective test is one in which each question is designed to have an unequivocally
correct answer or set of answers. In contrast a subjective test, for which there is no
uniquely correct answer, examines the ability of the student to respond clearly and
develop an argument. The Multiple Choice Question (MCQ) is the most commonly used type of
objective question. While objective tests are used largely to test knowledge and
comprehension, with careful design they can also assess higher learning outcomes such as
the application of knowledge and analytical skills.
Because they can be marked quickly and easily and can cover a wide range of course
content, objective tests are particularly appropriate for informal self-assessment to give
students ongoing feedback about their progress. They can be used in this way throughout an
undergraduate course. For formal assessment, however, their contribution to final marks is
likely to decline as students progress through their degree programmes and the range of
abilities being examined becomes more comprehensive and demanding.
The use of computers for objective testing
Approaches to computer-assisted assessment (CAA) can be divided into two systems:
automated marking of paper forms, using an optical mark reader (OMR) with hard copy
question paper and an OMR-readable answer form;
computerised marking in which questions are presented and responses assessed entirely
by computer software, with no paper involved.
CAA software can provide immediate supportive feedback for each question, tailored to
the answer given, making this particularly suitable for informal self-assessment. Superior
graphical and visualisation exercises can also be included with the use of multimedia
files such as graphics, sound and video clips. Most software offers a facility for
reporting results and analysing student responses. An increasing number of systems are web
based.
Benefits
It is objective (i.e. highly reliable in the assignment of
marks)
Tests can be marked and returned very speedily
Tests can incorporate a variety of media (images, video, audio)
Relevant feedback can be given automatically (during or after the test)
Randomised selection can be made from large question banks
Flexible access gives students opportunities for self-assessment
Built-in test management (collation, analysis, tabulation, report generation) eases the
administrative burden of assessment
Concerns
Questions tend to address factual content and may
therefore encourage learning of surface detail rather than an appreciation of underlying
concepts
The nature of the process makes it difficult to retrieve mistakes in compiling
questions before the test is taken, placing a heavy duty of responsibility on the question
setter
Formal assessment by computer is subject to concerns over security
Students may obtain the correct answer for the wrong reason or may be encouraged to
reinforce misconceptions about the subject while answering questions incorrectly
Hardware and software resources and support must be available
Producing well designed, appropriate questions involves a considerable investment of
staff time
Objective question types
There are various forms of objective question:
multiple choice: choose the correct answer form a list of alternatives
multiple response: select a number of correct answers from the list
true/false
selection/association: match items from two related lists
assertion/reason: choose the correct reason for an assertion (a special case MCQ)
While these question types can be delivered on paper and then processed by OMR, the
following two types specifically require a computer for input:
text match/gap filling: enter a word, short phrase or number
visual identification/hotspot: move a marker is moved to identify a particular
‘hotspot’ on an image
Designing questions
An MCQ question typically comprises three parts:
stem: the question component;
key: the correct answer;
distractors: incorrect answers provided as alternatives to the key;
and optionally:
feedback: a mark and/or comment reflecting the student’s performance. This
may be presented immediately after the question, at the end of the test or not at all.
Feedback may also be provided on the basis of overall test performance.
For MCQs students might be discouraged from guessing by the imposition of a penalty for
incorrect answers, although it is generally considered that negative marking simply alters
the baseline of results. In any case, the rewards of ‘blind’ guessing diminish
rapidly as the number of questions increases. Other viewpoints hold that
‘intelligent’ guessing may be no bad thing, and that well designed MCQ questions
will lead a genuinely blind guesser astray.
It is self-evident that the questions should reflect the aims and objectives of the
course and be appropriate to the abilities of the students for whom they are intended. A
pre-test during which questions can be tried out, edited and reformulated is considered
essential. Beyond that:
stem: ensure that it is concise and unambiguous, avoiding negatives (‘Which
of the following is NOT…?’) and grammatical clues to the key
key: this should be the same length as the distractors and its position in any
list should be randomised using the software (unless it is a numeric answer in which case
answers should be in ascending order, avoiding any systematic placement of the key)
distractors: these should be apparently realistic alternatives that, as far as
practicable, cover the full range of options and easily identify misconceptions
feedback: this should be appropriate, helpful, encouraging, varied and
unpatronising
Implementing tests
Students find that objective tests require considerable concentration and it is
probably wise to limit the test content to no more than one hour’s duration, although
imposition of a time limit for self-assessment purposes is usually counterproductive. The
use of open tests to assess coursework is subject to abuse through collaboration unless
supervised, but this is true of conventional activities executed out of class as well.
Encouraging this sort of student collaboration may even be considered desirable. The
provision of feedback under such conditions is also a significant issue: withholding
information can compromise other educational objectives. Using CAA in examination
conditions requires careful preparation including back-up plans in case of hardware or
software problems. Step-by-step tutorials and check lists are available for summative
assessment planning.
Question banks
Writing good objective questions can be a lengthy and laborious exercise. It is usually
faster to edit existing questions. Some question banks are available commercially,
generally associated with text books. It is worth finding out whether the textbooks you
use for your courses have associated questions. It is also possible to adapt questions
found in textbooks for objective testing. Otherwise, contact colleagues at other
institutions to see whether they are willing to work with you to produce questions for
similar courses. This may best be done via a relevant learned society. Your CTI Centre can
probably point you in the right direction.
Resources
Computer-assisted assessment centre: http://caacentre.ac.uk
Computer-assisted assessment mailing list: http://www.mailbase.ac.uk/lists/computer-assisted-assessment/
For information on OMRs see Data & Research Services: http://www.drs.co.uk
For information on computerised systems see Question Mark
Designer and Perception: http://www.qmark.com or
TRIADS: http://www.derby.ac.uk/assess/talk/quicdemo/html/
For Web-assisted assessment see CASTLE: http://www.le.ac.uk/castle; the Web-assisted
assessment mailbase list: http://www.mailbase.ac.uk.
A tutorial on setting effective objective tests is available from the CTI Centre for
Land Use and Environmental Sciences: http://www.clues.abdn.ac.uk:8080/caa/caatut.html
A protocol for the implementation of summative
computer-assisted assessment examinations is at: http://www.clues.abdn.ac.uk:8080/caa/protocol.html
Original
Authors
Eamonn Twomey
Jacqui Nicol
Christina Smart
Version: 4.1.10.1
