Multiple Choice Questions

Multiple choice questions on a quiz, exam, or even as a clicker question during a lecture are a simple way to assess learning, and can be used for different levels on Bloom’s Taxonomy. They are most appropriate for assessing learning objectives focused on recall, understanding, and application, though it is possible to write questions that assess analysis and evaluation. Learning objectives focused on creation should be measured using other assessment methods.

Question Structure

Multiple choice questions consist of the stem and the alternatives (i.e. choices). Alternatives include both the correct answer and the distractors (i.e., incorrect answers).

Example Question:

1. What is most likely to happen to the reliability of the scores for a multiple-choice test, where the number of alternatives for each item is changed from three to four? [this component of the multiple choice question is the stem]
   1. It will decrease
   2. It will increase
   3. It will stay the same
   4. There is no basis for predicting

The four answer choices are the alternatives. The second choice (B) is the correct answer, while the other three incorrect options (A, C, and D) are the distractors.

Tips for Preparing Items

• Revisit your learning objectives to determine topics/target levels of Bloom’s taxonomy
• Do not use multiple choice questions for learning outcomes better assessed by other methods
• Draft potential items when preparing lessons/lectures
• Cover topics that students have questions on in class
• Consider common mistakes students make on homework or assignments
• Write the stem first, then the correct answer, followed by the distractors
• Keep in mind that items can and should be revised before a final test  or quiz is produced
• Questions on your quiz or exam should cover each topic in your unit and different levels of Bloom’s Taxonomy, with a particular emphasis on application

The Stem: Best Practices

• Question formats are more effective than incomplete statements
• Minimize the number of words where possible/do not include irrelevant material
• Avoid negative phrasing (i.e. which of the following is not true…) unless the learning outcome necessitates it (e.g. avoidance of dangerous practices)
• A stem is good if a proficient student is able to answer before reading the alternatives

The Alternatives: Best Practices

• All should be plausible, clear and concise
• Include distractors based on common student misconceptions
• Alternatives should be independent and mutually exclusive
• Homogenous in content
• Free from clues to the correct answer
• Minimize use of “all of the above” or “none of the above”
• Presented in logical order, with the location of the correct answer random throughout the test

Writing Higher Order Multiple Choice Questions

• Use Bloom’s Taxonomy categories and associated action words (learn more by completing the resource guide Creating Learning Objectives) focusing on applying, analyzing and evaluating.
• Use specific examples for which students will use the information. This can include use of scenarios or cases for which students must make a choice, asking students to identify a concept based on an example, or asking students to make conclusions or predictions based on discipline appropriate visuals.
• Require multilogical thinking or “thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a …problem” (Morrison and Free, 2001)

Multiple Choice Questions for Application, Analysis, and Evaluation

Writing multiple choice questions to assess application

To move from understanding to applying you shift the focus from definitions to execution. At the apply level, a student must use their knowledge of a rule or concept in a novel situation. Application questions should always include a scenario that students have not yet been introduced to.

To adapt a question to assess  application, follow these three steps:

1. Introduce a novel scenario: Instead of asking about a concept in the abstract, put it into a specific, real-world context the student has not already seen in their course content (e.g. readings, videos, lectures, etc.).
   1. Shift the question from “What is the formula for area?”, to “a homeowner wants to tile a circular patio with a diameter of 12 feet…”
2. Require procedural knowledge: Word the question so students must follow a  multi-step process: identify the relevant rule, select the correct information from the scenario, and determine the outcome.
   1. Shift from “identify a metaphor” to “which sentence in the following paragraph uses a metaphor to emphasize the character’s fear?"
3. Use Functional Distractors: Distractors (wrong answers) should represent common mistakes made during the application process (e.g., using the wrong formula, rule or process; typical calculation error, etc.)

Sample Question Evolution
Here is how a single concept—Newton’s Second Law—evolves as it moves into the apply category:

1. Recall/Understand
   1. Which of the following best defines Newton's Second Law?
   2. Focus: Recognize a definition
2. Apply (basic)
   1. How much force is needed to accelerate a 5kg bowling ball at…
   2. Focus: Uses a formula in a simple word problem.
3. Apply (advanced)
   1. A 1,200kg car is traveling at 20m/s. To stop the car in 5 seconds, how much force must the brakes apply?
   2. Focus: Identifies variables and executes a multi-step process.

The Stem: action verbs & key phrases
To trigger application-level thinking, start your questions with these types of prompts:

• "Calculate..."
• "Predict the outcome if..."
• "How would you solve..."
• "Which rule applies to..."
• "Based on the provided chart, determine..."

Writing multiple choice questions to assess analysis and evaluation

To move from remembering or understanding to analyzing or evaluating  in a multiple-choice format, you stop asking what something is and start asking how it works or why one option is superior to another. Like for application questions, you need to create scenarios that require students to think more deeply about a concept.

Assess analysis (Breaking it Down)
Analysis requires students to identify patterns, determine how parts relate to one another, or distinguish between fact and inference.

• Provide a new set of data, a short case study, or a complex diagram students have not been exposed to prior
• The shift: Instead of "Which of these is a symptom of X?", ask "Based on this patient’s lab results and history, which finding is the most likely cause of their current crisis?"
• Key Action Verbs: Categorize, contrast, correlate, diagram, or discriminate

Assess evaluation (Judging the Value)
Evaluation requires making judgments based on criteria and standards. In a multiple-choice setting, this usually involves choosing the best, most effective, or most ethical option among several plausible ones.

• Create a "Compelling Distractor" environment. All four options should be technically "correct" actions, but only one fits the specific constraints or priority of the scenario.
• The shift: Instead of "What is one benefit of solar energy?", ask, "Given a city with 300 days of sun but limited land mass and a high budget, which energy proposal is the most sustainable long-term solution?"
• Key Action Verbs: Appraise, critique, justify, prioritize, or support.

Sample Question Evolution

1. Recall/Understand
   1. Define the term Confirmation Bias.
   2. Focus: Recognize a definition.
2. Analysis
   1. In the attached transcript of a debate, which statement best illustrates Confirmation Bias
   2. Focus: Finding patterns in new data
3. Evaluation
   1. Which of the following experimental designs would most effectively minimize the impact of Confirmation Bias?
   2. Focus: Judging the quality of a solution.

References

Bothell, T. 14 Rules for writing multiple choice questions. Presented at the Brigham Young University Annual Conference, 2001. 

Brame, C. Writing good multiple choice test questions. Vanderbilt University Center for Teaching. 

Brame, C. (2022). Writing higher order multiple choice questions. Vanderbilt University Center for Teaching. 

Cheung, D and Bucat, R. How can we construct good multiple-choice items? Presented at the Science and Technology Education Conference, Hong Kong, June 20-21, 2002. 

Morrison, Susan and Free, Kathleen. Writing multiple-choice test items that promote and measure critical thinking. Journal of Nursing Education 40: 17-24, 2001.

Zimmaro, D. (2010). Writing good multiple-choice exams. University of Texas Center for Teaching and Learning.