Assessing using multiple-choice questions

Choose a university

Select a UK institution to view location-specific information throughout this website.

 

  1. Introduction.
  2. Types of question.
  3. Advantages and disadvantages.
  4. Hints, tips and suggestions.
  5. Further information.

pdf Download PDF version here

1. Introduction

Multiple choice questions, or MCQs for short, are used in many degree programmes in the UK. They are commonly used to test acquisition of knowledge and basic understanding, although enthusiasts are keen to demonstrate that, with careful setting, they can also be used to test higher order thinking skills, such as application and analysis.  Also, MCQs have proved to be popular as a way of providing assessment for large cohorts of students as much of the marking can be automated and so feedback and results can be provided quickly for even the largest class sizes.  

There are several computer programmes that can be used to store, set and mark MCQ tests.  A bank of questions can be created in the computer and then the questions can be asked in a fixed or random order.  Students respond by keying in numbers, letters or words and the software can react with an appropriate result and feedback.  More sophisticated programmes can select questions based on the previous answers that the student gives - setting more demanding questions if answers are correct, or easier questions about the same topic if answers are incorrect.  Alternatively, the questions can be printed on paper and students record their answers on specially designed forms that can be read by a machine - an optical mark reader (OMR).  This machine can recognise which answers are correct and which are not, so the tests can be marked quickly and accurately.

 

2. Types of question

An MCQ is a question in which students are asked to select one answer from a set of given options.  There are many different formats that can be used in MCQ tests - here are some of the most common:

Question stem

What is the name of the flag of the United Kingdom?

A     British accord

B     Union Joe

C     Union Jack

D     George Cross

E     United Jack

The student answers the question by selecting a letter A to E, and entering this either on a form provided for the purpose or at a computer keyboard.  In this example, the correct option (the key) is C.  The other incorrect options are referred to as distractors.  Often the difficulty of a question is determined by how realistic the distractors are.

True / False

In a True / False question the student simply chooses between two options and selects the answer they believe to be true.  However there are also True / False questions where there are many possible true answers and students are asked to indicate all of the answers that are true from the list of options.

Drag and Drop

The students are given a diagram or picture to label and a list of possible labels.  They would either write the correct label on the diagram, or on a computer literally click on the selected name and drag it to the site they wish to label.

Hotspots

The students highlight an area on a diagram, map or image (either on paper or on the computer screen).  For example, students could be asked to indicate the position of a tumour on an x-ray or the location of a receptor site on a diagram of a cell wall etc.

Ranking or sequencing

The students are asked to order or sequence items in a list according to specific requirements, eg putting events in date order or steps in a biochemical pathway in sequence.

 

3. Advantages and disadvantages

It is often said that MCQs take ages to write and prepare but are then quick and easy to administer.  Question authors usually aim to produce a bank of questions so that each year a new subset of questions can be used in the examination.  If the assessment is summative (counts towards the student's overall mark/grade) and paper-based then it becomes important to collect in examination papers and so try to prevent question knowledge travelling between cohorts of students.

If the assessment is primarily for formative purposes, ie to provide the students with ongoing feedback about their progress and understanding, then online MCQ tests can be very useful.  Not only can the students attempt tests when they are ready to do so and as many times as they wish, but question authors can provide feedback statements that are released in response to incorrect answers - to explain why this isn't correct and advise the student on further study.

MCQ tests and examinations are often seen as being very broad-based assessment methods that provide a wide coverage of the syllabus (whereas essay questions tend to provide a much more limited sample of coverage).  As a result, students cannot choose to avoid certain areas or cover topics superficially.

MCQ tests and examinations are good at testing knowledge and question authors tend to favour memory testing questions; it is very difficult to test synthesis and design.  These abilities, along with creativity and writing skills, need to be tested in more discursive forms of assessment.

 

4. Hints, tips and suggestions

Problems encountered in MCQ tests and examinations are often the result of poorly designed questions.  This may be because the list of options:

  • Includes give-away clues to the answer, or allows students to rule out several obviously ‘wrong answers' easily; this could be as simple as making the correct answer the longest of all of the options.
  • Does not test the abilities and attributes required by the intended learning outcomes.
  • Encourage surface learning and over-reliance on memory.
  • Confuse and frustrate students who have a deeper understanding.

In an attempt to stop students guessing some assessors have introduced a system called ‘negative marking'.  Here marks are deducted for every incorrect answer as well as being awarded for correct answers.  Such an approach discourages risk taking and guess-work.  Other approaches include the adoption of mathematical strategies to normalise scores or setting high pass marks.  Optical mark reader software and computer-based MCQ programmes include facilities to analyse the responses that students give to questions, and so provide you with feedback on the quality of the questions.  If, for example, most students answer a question correctly, even those that do badly overall in the test, this might suggest that the question is not well designed; the distractors are not distracting, or the key is too obvious.

 

5. Further information

Designing and Managing MCQs.  The CASTLE Toolkit, University of Leicester.  (This is a super online handbook that covers the fundamentals of MCQ use and more advanced topics such as effective scoring and statistical manipulation of mark profiles and norms.)

Brown, G.; Bull, J. and Pendlebury, M., 1997.  Assessing Student Learning in Higher Education.  London: Routledge.  In particular, see chapter 4 - Methods and Strategies: an overview.  (A well respected text that gives much background and great insight into the UK approach to assessment.)

McKenna, C. and Bull, J., 1999.  Designing effective objective questions: an introductory workshop.  A useful guide to question design, with examples of tested questions demonstrating alternative formats.