Recent changes in the AP Chemistry curriculum have included a strong focus on a conceptual understanding of chemistry. Dr. Eric Mazur¹, a physics professor at Harvard, developed Peer Instruction (PI) for his physics courses to engage his students in learning challenging physics concepts. Dr. Julie Schell², currently at UT Austin, has worked with Mazur to develop the process and introduce it to the educational community. Professor Schell describes PI as a “research-based teaching method that leverages the power of social interaction to drive learning.”³ In this article, I am going to describe how you can leverage the power of social interaction through Peer Instruction, share some data on its efficacy as a learning opportunity, and provide some guidelines for writing higher-quality concept questions.

How peer instruction works

Peer instruction can be distilled into four basics parts: (1) learn, (2) respond, (3) pair/share, and (4) respond again. Peer instruction begins with an initial learning phase that includes the challenging concept. Since I teach using a flipped learning approach, this is delivered via online videos — although traditional lectures, inquiries, or other methods can also work well. The second step is to ask students to individually answer a multiple-choice question. I make sure my students understand that this is a low-risk, stress-free activity so that they will try hard without a fear of failure.

The results of first responses will drive how the teacher should proceed. Once at least roughly 80% of students (please note that all of the percentages used here should be seen as approximations) have responded, the teacher can do a quick estimate of the percent of students who answered correctly. If more than 70% of the students answered correctly, the teacher may want to stop the PI process and quickly explain the correct answer or re-teach the concept to the small group of students who do not understand the concept. If fewer than 30% of students get the answer correct, this may signal to the teacher that he or she needs to completely re-teach the concept in a large group setting.

If the percentage of students answering correctly is between 30-70%, PI will be a successful option. Students are now asked to pair up with another student — ideally, someone whose answer differed from their own. Now it’s time for students to implement AP’s two favorite words, explain and justify, to answer the same question.Students should be encouraged to not only justify their choices, but also explain them at the molecular level whenever possible. Once students respond with their consensus answers, the teacher again decides on the best course of action. Typically, the percentage of correct answers increases. I have had a few times when it has actually decreased (at which point I ponder aloud whether I may have some future politicians in my midst!)

©Kasia Bialasiewicz/Bigstock.com

At this point, you may be asking about the method for collecting student responses. There are a variety of ways to have students respond to questions — from the low-tech to high-tech. On the low-tech end, one can simply have students hold one, two, three, or four fingers against their chest to represent their answer choices. Or students can simply convert a square piece of paper into a colorful response card. Alternately, a few decks of playing cards can readily be turned into a response system. Simply assign answer choices with a card value such as “ace,” “king,” etc.

On the higher-tech end, this evaluation process can be made easier with the use of an e-response system (such as those that accompany Smart Boards). There are also a number of options for using cell phones. I myself use TI-Nspire^TM Navigators^TM (a system of handheld devices connected wirelessly to my computer) when I do PI.

What research says about peer instruction

If you have been in education for a decade or more, you have likely been exposed to a multitude of different teaching strategies. One of the first things I say whenever I’m going through professional development is “show me the data.” Our time is too valuable to invest in strategies that do not have a strong possibility of increased learning.

Fortunately, Mazur⁴ has published an extensive amount of data indicating a significant increase in learning in both algebra-based and calculus-based physics at Harvard. One of his metrics was a comparison of pre- and post-test results. A traditionally taught calculus-based class realized an 8% improvement. Implementation of PI resulted in a 14-25% improvement. In his algebra-based course, the pre- to post-test improvement went from 21% for a traditionally taught group to 33% after the implementation of PI.

Meanwhile, Troy Faulkner⁵ and his colleagues saw increases of up to 17% for their course proficiency averages in algebra II, pre-calculus, and calculus. A big win was the decrease in failure rates from almost 14% to 6% in geometry when PI was added as a strategy. Computer science professors Cynthia Bailey Lee and Beth Simon, from Stanford University and UC San Diego, respectively, present more data to support these observations.⁶ PI has also shown impressive results when it was included in The Monash Critical Thinking Study.⁷

Writing concept questions (ConcepTests)

Hopefully, the short survey above has convinced you to give PI a shot in your class. Unfortunately, this is where we experience the real barrier to the implementation of PI in AP Chemistry: the concept questions. Current, well-written questions are in short supply and are often needed for assessments.

Interestingly, learning how to write higher-quality multiple-choice questions has never been part of the professional development offered at the schools in which I have taught. For this reason, I want to share what I have learned over the years from my own experience and research. First of all, realize that you don’t need to start from scratch. Older AP questions provide a valuable starting point. Test banks from textbooks can also be helpful. I have seen a wide range in quality from the books I have used.

Julie Schell has noted, “ConcepTests turn the Peer Instruction classroom into what I like to call a higher-order thinking sandbox, where students can, in a low-stakes environment, build expert-level cognitive skills that are conduits to the Holy Grail of learning — knowledge transfer (the ability to apply prior knowledge to solve problems in or navigate new, unfamiliar contexts).”⁸ This should be our goal with our questions.

There are three main parts to a multiple choice question. The stem is the question itself, and the choices include the correct answer and distractors. The first thing to consider is the question, or stem. We are not including calculation-based questions in PI. We really want to focus on concepts that are typically confusing for students.

A stem should have a problem clearly stated and should be linked to only one specific objective. Compare the two examples below. The question and answers on the left includes multiple objectives. The stem on the right zeros in on only one of the objectives.

As you are culling through your sources, avoid “negative” questions. These questions include phrases such as “never,” “not,” and “all true except.” They are not only confusing, but also give away pieces of true information that might be helpful on other questions. The question below is from a test bank that accompanies a popular text used for AP Chemistry.⁹ Note how much information is revealed in the distractors.

The PI questions must lead to higher order, critical thinking, especially if they are to aid in success in AP Chemistry. We are not seeking simple recall for this learning opportunity. Compare the following two questions, for example. The one on the left is more appropriate for tenth-grade chemistry and requires only memorization. The one on the right involves not only the ability to describe what happens, but also to explain the concept at the molecular level.

As you are seeking multiple-choice questions, make sure to re-write ones that are fill-in-the-blank. Fill-in-the-blank questions often deal with lower-order thinking concepts, and also increase the cognitive load of the question, using up valuable working memory space.¹⁰ As psychology professor Jon Mueller has observed, “A test is a task that requires considerable conscious attention. So, it is important to remove any elements of a test item that might distract or unnecessarily increase the cognitive load a student encounters.”¹¹

Finally, keep your questions focused, and avoid using extraneous information. I have to admit that I love themed tests. We have done Lord of the Rings, The Matrix, and Big Bang Theory tests. While they are fun, they use up valuable reading time, and are especially challenging for English language learners.

Let’s now turn our attention to distractors. ETS recently reduced the number of distractors from five to four. A key goal of AP tests is to discriminate, or differentiate, the ability of students in order to help colleges place them in the correct classes. Statistically, five distractors do not improve a question’s ability to discriminate well — and in fact, some have argued that three distractors is optimal.¹² What makes this part of question writing challenging is that all of the distractors must be plausible. Choice “4” below is clearly not plausible (it is also a lower-order thinking question). In this case it would be better to have only three possible answers. Do not give in to the temptation to include “all of the above” or “none of the above.” When students select these answers, it is more difficult to determine what they know and with which concepts they are struggling.

Finally, make sure the distractors are homogeneous. When three choices include kinetic molecular theory and one includes collision theory, many students will play “odd man out.” For this reason, you also want to make sure to edit distractors so they are similar in length. In the question below, many students will choose option “2” simply because it is longer.

Peer Instruction is an engaging and effective way to improve learning for an AP chemistry class. Hopefully, finding higher-quality concept questions will not deter you from trying it with your students. I strongly recommend working with other teachers to develop questions. If you are the only AP chemistry teacher in your district, I would encourage you to seek out virtual colleagues.

References

1. The Mazur Group, http://mazur.harvard.edu/research/detailspage.php?...
2. Turn to Your Neighbor, the Official Peer Instruction Blog, https://blog.peerinstruction.net
3. Schell, J., https://blog.peerinstruction.net/2013/08/26/the-6-...
4. Crouch, C.H. and Mazur, E., “Peer Instruction: Ten years of experience and results,” American Journal of Physics 69, No. 9, (September 2001). http://mazur.harvard.edu/publications.php?function.
5. Does the Peer Instruction Flipped Learning Model Really Work? https://sites.google.com/site/troyfaulknerprofessi...
6. Lee, C.B. and Simon, B., http://www.peerinstruction4cs.org
7. Results from critical thinking tests, http://artsonline.monash.edu.au/peer-instruction-i...
8. Schell, J., https://blog.peerinstruction.net/2012/05/29/what-i...
9. Brown, T.E. and LeMay, H.E., Chemistry: The Central Science, 12e (Edinburgh: Pearson Education, 2014) Test Bank
10. Gillmor, S.C.; Poggio, J. and Embretson, S., “Effects of Reducing the Cognitive Load of Mathematics Test Items on Student Performance.” Numeracy. 2015, 8. Available at: http://scholarcommons.usf.edu/numeracy/vol8/iss1/a...
11. Mueller, J. Authentic Assessment Toolbox, http://jfmueller.faculty.noctrl.edu/toolbox/tests/gooditems.htm
12. Rodriguez, M. “Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research.” Educational Measurement: Issues and Practice. 2005. http://www.performancetest.org/documents/RodriqguesEdMeasurement3option.pdf