Chemistry teachers frequently make decisions about curriculum and assessment. A curriculum can be summarized as a list of measurable learning objectives, or expectations of what students should know and be able to do. An assessment is designed to help teachers determine whether or not students have achieved certain learning objectives.

One type of assessment task that is used in both formal and informal assessments in chemistry is the multiple-choice question. However, some teachers have never been trained in how to write multiple-choice items. This article will discuss the characteristics of multiple-choice items as well as some guidelines and strategies for writing them.

Figure 1: Components of a multiple-choice item

Components of a Multiple-Choice Item

The components of a multiple-choice item are listed in Figure 1. The learning objective represents something that the student should know and be able to do. The stimulus, which is optional, may be a graph, diagram, or data table. The stem is the question itself. The options include both the correct answer and the incorrect answers, the latter of which are known as distractors. Figure 2 represents a multiple-choice item that I wrote for Learning Objective 1.3 from the AP Chemistry curriculum. The AP Chemistry Course and Exam Description includes a total of 117 learning objectives.

Figure 2: A sample multiple-choice item

Guidelines for Writing Multiple-Choice Items

Guidelines for writing multiple-choice items have been studied in research literature.^2,3 A few of these guidelines are listed in Figure 3.

Figure 3: Guidelines for writing multiple-choice items

When a well-written multiple-choice item is aligned with a single learning objective, this item can provide information about how well students have mastered the objective. In order to get more “mileage” from an item, a teacher might choose to write a question that covers several learning objectives. However, if students answer such a question incorrectly, it is difficult to identify which specific learning objective that they are struggling to understand. An example of this situation appears in Figure 4. The multiple-choice item on the left side of the figure requires the student to know how to write the chemical formula for a metal hydroxide from its name, how to calculate moles from the given information of volume and molarity, and how to convert from moles of base into moles of acid for a neutralization reaction. The item on the right has been edited so that the question focuses only on the ability to calculate moles of hydroxide ions from the given information of volume, molarity, and chemical formula.

Figure 4: Comparing questions that focus on multiple vs. single learning objectives

Another example of an attempt to assess student understanding of several concepts within a single question is the complex multiple-choice item, which is also known as a Type-K multiple-choice item. This type of question usually presents three or more statements labeled with Roman numerals (I., II., III., etc.). Students must decide whether each statement is true or false. The letter options (A, B, C, D) for this multiple-choice item are presented as various combinations of these statements. An example question, which appears in the student guide for the SAT Subject Tests⁴, is shown in Figure 5.

This type of question has several disadvantages. If it is graded on an “all-or-none” basis, students who can identify some (but not all) of the true statements receive no partial credit. In addition, depending on the patterns listed in the options, this type of item may provide helpful clues to a student who is test-savvy, but has only partial knowledge of the information presented⁵. In a comparative review of multiple-choice items, this type of item was determined to be more difficult and exhibited lower discrimination than a single-answer item⁶. Discrimination is a measure of how well an item is able to distinguish between examinees who are knowledgeable and those who are not. Complex multiple-choice items are found on a variety of standardized tests, including the SAT Subject Tests. They have appeared on the AP Chemistry exam prior to 2014, but this type of item is no longer used.

Figure 5: A complex multiple-choice item

Students who are unsure of the correct answer to a question may simply choose the longest option. An example of this situation is shown in Figure 6. In the original question, students may be biased toward choice (D) because it is longer than the other options. In the edited version of the question, each option is about the same length.

Figure 6: Example of editing answer options to be roughly the same length

Another guideline for writing multiple-choice items is to ensure that all of the distractors are plausible. An example of this situation is shown in Figure 7. Based on the carbon-to-hydrogen ratio, choice (C) is an impossible empirical formula. Choice (D) is not impossible, but seems rather unlikely. It is preferable that each empirical formula appears reasonable. An improved list of options for this question might be (A) CH, (B) CH₂, (C) CH₃, and (D) CH₄. A multiple-choice item should be connected to a specific learning objective. If students use a simple heuristic to eliminate some of the distractors, they might get the correct answer without fully understanding the concepts associated with that learning objective.

Figure 7: Example of a multiple-choice question in which not all of the distractors are plausible

When writing the distractors for a multiple-choice item, it is helpful to be familiar with common student misconceptions. The stimulus of the item shown in Figure 8 indicates that C₃H₈ has a higher boiling point than CF₄. Both of these molecules are nonpolar based on the symmetrical arrangement of the bond dipoles. The only intermolecular attractive forces that these molecules experience are London dispersion forces. Choice (D) correctly states that the dispersion forces in C₃H₈ are stronger than those in CF₄. Choice (A) is based on the misconception that any molecule containing a hydrogen atom is capable of forming hydrogen bonds. Choice (B) is based on an error regarding the polarity of C₃H₈. Choice (C) is based on the confusion associated with intramolecular attractive forces (i.e., covalent bonds) versus intermolecular attractive forces.

Figure 8: Example of how to use common student errors when writing distractors

Figure 9: Example of multiple-choice item focusing on conceptual understanding

Figure 10: Example of multiple-choice item asking students to make a conclusion based on experimental data

Higher-Order Cognitive Skills

One reason that multiple-choice items are frequently used to measure student performance is that they are easy to administer and score. However, some critics may assume that multiple-choice items are only useful for measuring simple recall of facts or basic knowledge. It is desirable to construct multiple-choice items that focus on higher-order cognitive skills.^7,8

A multiple-choice item can be written in a way that emphasizes conceptual understanding and critical thinking. Figure 9 features a question that I wrote to assess understanding of the concept that a weak acid is only partially dissociated into ions in solution. Choice (A) is based on the misconception that the definition of a weak acid is related to its solubility instead of its dissociation. Choice (C) is incorrect because it fails to account for the 1-to-1 mole ratio between H₃O⁺ and F^– in a solution of HF. Choice (D) would only be true if HF were classified as a strong acid.

Multiple-choice questions on the AP Chemistry exam focus on conceptual understanding and critical thinking when they ask students to make a conclusion, make a prediction, or select the best explanation for experimental data that is presented. An example of this type of question, which appears in the AP Chemistry Course and Exam Description,¹ is shown in Figure 10. Choice (B) is correct because it supports the law of conservation of energy, whereas choices (A) and (D) are inconsistent with this law. It is important to consider the equation q = mcΔT in which q = heat, m = mass, c = specific heat capacity, and ΔT = the change in temperature. Choice (C) is incorrect because it fails to recognize that there is an inverse relationship between the magnitudes of c and ΔT.

Conclusion

In this article, a few guidelines for writing multiple-choice items have been discussed. A comprehensive list of guidelines has been summarized in the research literature.²,³ Teachers who struggle to find good multiple-choice items may find it necessary to modify existing questions or to write their own questions “from scratch.”

The process of designing multiple-choice items includes selecting a specific learning objective and writing a stem that is phrased clearly. There should be only one correct answer, and it is helpful to consider common student misconceptions when writing the distractors. There may be situations in which teachers decide that a free-response question is more appropriate than a multiple-choice question for the purpose of gathering information about what students know and are able to do.

Building assessments and writing questions can be challenging, but it can become easier when teachers collaborate with their colleagues and share ideas. The creation of high-quality assessments can provide useful information about student learning and help teachers make important decisions about instruction.

References

1. The College Board. AP Chemistry Course and Exam Description (Effective Fall 2014), available online at media.collegeboard.com/digitalServices/pdf/ap/ap-chemistry-course-and-exam-description.pdf (accessed August 6, 2018).
2. Haladyna, T. M., et al. A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education, 2002, 15, 309–334.
3. Paniagua, M. A.; K. A. Swygert. Constructing Written Test Questions For the Basic and Clinical Sciences (4th ed.), National Board of Medical Examiners, 2016; available online at info.nbme.org/rs/343-IIB-826/images/IWW%20Gold%20Book_new%20cover_web.pdf (accessed August 6, 2018).
4. The College Board. 2017-2018 The SAT Subject Tests Student Guide, The College Board, 2017, p. 27; available online at https://collegereadiness.collegeboard.org/pdf/sat-subject-tests-student-guide.pdf (accessed August 6, 2018).
5. Burton, S. J., et al. How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Brigham Young University Testing Services and The Department of Instructional Science, 1991; available online at https://testing.byu.edu/handbooks/betteritems.pdf (accessed August 6, 2018).
6. Haladyna, T. M. and S. M. Downing. A Taxonomy of Multiple-Choice Item-Writing Rules. Applied Measurement in Education, 1989, 2(1), 37−50.
7. Domyancich, J. M. The Development of Multiple-Choice Items Consistent with the AP Chemistry Curriculum Framework To More Accurately Assess Deeper Understanding. Journal of Chemical Education, 91, 1991, 1347–1351.
8. Towns, M. H. Guide To Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments. Journal of Chemical Education, 1991, 91, 1426–1431.

Photo credit:
(article cover) sasirin pamai/Bigstockphotos