Tag Archives: testing

Multiple Choice Questions

A multiple choice question begins with a stem or lead-in that is addressed by a correct response chosen from a list of alternatives. Writing a good multiple choice question that elicits an answer based on knowledge, not guessing or misunderstanding, is an art. For example:

Who was the twentieth president of the United States?

  1. Rutherford B. Hayes
  2. James A. Garfield
  3. Chester A. Arthur
  4. Grover Cleveland

This question tests recall of the twentieth president. The stem is parsimonious, including only the ideas and words necessary to answer the question. The “distractors” are parallel, possible answers–all presidents from around the same time. Compare to this question:

Choosing the first president of the United States was a tremendous responsibility. He would set precedents for subsequent office holders. The Electoral College unanimously elected George Washington who had led the colonies to victory against the British. James Madison, who was married to Dolly, was the fourth president. Who was the twentieth?

  1. James Brown
  2. LeBron James
  3. James Garfield
  4. James Bond

In this question, the stem is overwritten with information you don’t need to answer the question correctly. Irrelevant information may be testing your reading comprehension more than your twentieth president knowledge. Even if you know the correct answer, you may get it wrong because you can’t get through the reading.

The distractors are implausible. If the correct answer is embedded in a group of possibilities that are totally outlandish, you will get the right answer not because you’ve learned it, but because you can use general knowledge to eliminate the others. That’s a bad question.

If written correctly, a multiple choice question can be very effective at proving mastery in Bloom’s elementary cognitive categories of remembering and understanding, and to a lesser extent in the third category, applying (see previous blog Learning Objectives in Higher Education).

According to Cathy Davidson, educator Frederick J. Kelly introduced multiple choice tests in 1914.  They were intended to improve the equality of grading. Teacher bias as well as individual differences such as wealth or poverty would not prevent a student from being graded correctly. Multiple choice questions also made grading less time-consuming for teachers, freeing them to do more instruction. Incorporated in standardized tests, multiple choice questions allowed us to compare student proficiency in different areas of the country. Good goals, right?

Don’t we share these goals today: To evaluate students without bias. To give them equal opportunity to learn despite where they live or learn. To free instructors to have more time to teach and interact with students. So why are multiple choice questions criticized so much?

Davidson says it’s because we try to use multiple choice questions in areas where they don’t work such as

….problem solving, collaborative thinking,  interdisciplinary thinking, complex analysis, the ability to apply learning to other problems, complexity…creativity, imagination, originality…

Demonstration of these types of learning, Bloom’s applying, analyzing, evaluating, and creating, requires more than picking out the right answer if there even is a “right” answer. Kelly created multiple choice questions to measure basic skills important to twentieth century American work and citizenship. He admitted that they only tested “lower-order” thinking.

Extending the multiple choice format to measure higher-order thinking results in many flawed questions. Piled one on top of the other in repetitive quizzes or long tests, these ill-conceived items become anxiety-provoking, deadening experiences for students. In this context, they are weak indicators of student learning achievement.

Through digital programming we have the potential to create robust profiles of students showing how they process, retain, and apply information. This gives us the opportunity to approach the challenge of assessing student performance from a fresh perspective, one that may even use testing rarely. Let’s start by identifying the problem we want to solve: How do we make sure that students have learned what they need to learn to be successful in the world?

Now to test your understanding:

Which statement best describes this blog writer’s point of view?

  1. Multiple choice questions are easy to write.
  2. Multiple choice questions test critical thinking.
  3. We should rethink how we assess learning.
  4. We should never use multiple choice questions.

Are You Ready for Critical Thinking?

Definitions of critical thinking vary greatly. My feelings about critical thinking are reflected in a scene from the movie Rudy, the story of a working-class young man who has to overcome many obstacles to achieve his goal of playing football at Notre Dame. After yet another disappointment, Rudy asks the local priest for advice. The priest replies, “In thirty-five years of religious studies I’ve come up with only two hard incontrovertible facts, there is a God and I’m not him.”

We all agree that critical thinking exists and we know it is not repeating facts. After that, it’s up for grabs. My favorite part of critical thinking is creativity, building on what you are hearing, reading, or learning, and making connections, usually clever, enlightening, or imaginative.

Although educators say critical thinking should be encouraged, in reality, it is often discouraged, ignored, or put off so that educators can achieve their own goals. Saying “That’s interesting but,” “I will take questions at the end if I have time” “We’re going to address that in next week’s class” are all ways to delay spontaneous critical thinking.

As part of Jumpstart’s Read for the Record,  I read Loren Lang’s Otis about a heroic farm tractor to a group of preschoolers. The sound of the tractor, “putt puff puttedy chuff,” was repeated many times as he did his work.

After the reading, the teacher started singing “Old Macdonald Had A Farm:”  “On his farm he had a cow…with a moo moo, here, and a moo moo there…” Then she asked the children to call out suggestions for what was on the farm. One boy kept saying “tractor.” “After being ignored three times, he stopped. In a few more grades, he will probably not even bother to make a suggestion as teachers continue to ignore his critical thinking.

As critical thinking does, the boy’s thoughts stimulated mine. What’s on a modern farm?

A milking machine, with a chugga, chugga, here, and a chugga, chugga there

A harvester, with a tatata, tatata, here, and a tatata, tatata there

A banker, with a foreclosure, foreclosure, here, and a foreclosure, foreclosure, there

A frustrated farmer, with a fuck, fuck, here, and a fuck, fuck there

We need to be realistic about critical thinking if we are going to encourage it. Ironically, those who oppose it are correct when they stated that teaching critical thinking results in: “challenging the student’s fixed beliefs and undermining parental authority” (Texas Republican 2012 Platform, later retracted). See Stephen Colbert for critical thinking on that!

I support teaching critical thinking, but we need to think about how we will handle the consequences. Based on articles critical of general testing, my son Ross put his name on a middle-school standardized test and handed it in without answering any questions. His teacher called me; she was extremely upset about the “terrible” action he had taken. I thought, I guess this is not a good day for critical thinking.

Educators, if you are serious about critical thinking, start looking ahead, get ready to be made uncomfortable and to think on your feet.  Students who think critically will rock your world!