Tag Archives: Bloom’s Taxonomy

Multiple Choice Questions

Education Strategy, , , , , , ,

A multiple choice question begins with a stem or lead-in that is addressed by a correct response chosen from a list of alternatives. Writing a good multiple choice question that elicits an answer based on knowledge, not guessing or misunderstanding, is an art. For example:

Who was the twentieth president of the United States?

  1. Rutherford B. Hayes
  2. James A. Garfield
  3. Chester A. Arthur
  4. Grover Cleveland

This question tests recall of the twentieth president. The stem is parsimonious, including only the ideas and words necessary to answer the question. The “distractors” are parallel, possible answers–all presidents from around the same time. Compare to this question:

Choosing the first president of the United States was a tremendous responsibility. He would set precedents for subsequent office holders. The Electoral College unanimously elected George Washington who had led the colonies to victory against the British. James Madison, who was married to Dolly, was the fourth president. Who was the twentieth?

  1. James Brown
  2. LeBron James
  3. James Garfield
  4. James Bond

In this question, the stem is overwritten with information you don’t need to answer the question correctly. Irrelevant information may be testing your reading comprehension more than your twentieth president knowledge. Even if you know the correct answer, you may get it wrong because you can’t get through the reading.

The distractors are implausible. If the correct answer is embedded in a group of possibilities that are totally outlandish, you will get the right answer not because you’ve learned it, but because you can use general knowledge to eliminate the others. That’s a bad question.

If written correctly, a multiple choice question can be very effective at proving mastery in Bloom’s elementary cognitive categories of remembering and understanding, and to a lesser extent in the third category, applying (see previous blog Learning Objectives in Higher Education).

According to Cathy Davidson, educator Frederick J. Kelly introduced multiple choice tests in 1914.  They were intended to improve the equality of grading. Teacher bias as well as individual differences such as wealth or poverty would not prevent a student from being graded correctly. Multiple choice questions also made grading less time-consuming for teachers, freeing them to do more instruction. Incorporated in standardized tests, multiple choice questions allowed us to compare student proficiency in different areas of the country. Good goals, right?

Don’t we share these goals today: To evaluate students without bias. To give them equal opportunity to learn despite where they live or learn. To free instructors to have more time to teach and interact with students. So why are multiple choice questions criticized so much?

Davidson says it’s because we try to use multiple choice questions in areas where they don’t work such as

….problem solving, collaborative thinking,  interdisciplinary thinking, complex analysis, the ability to apply learning to other problems, complexity…creativity, imagination, originality…

Demonstration of these types of learning, Bloom’s applying, analyzing, evaluating, and creating, requires more than picking out the right answer if there even is a “right” answer. Kelly created multiple choice questions to measure basic skills important to twentieth century American work and citizenship. He admitted that they only tested “lower-order” thinking.

Extending the multiple choice format to measure higher-order thinking results in many flawed questions. Piled one on top of the other in repetitive quizzes or long tests, these ill-conceived items become anxiety-provoking, deadening experiences for students. In this context, they are weak indicators of student learning achievement.

Through digital programming we have the potential to create robust profiles of students showing how they process, retain, and apply information. This gives us the opportunity to approach the challenge of assessing student performance from a fresh perspective, one that may even use testing rarely. Let’s start by identifying the problem we want to solve: How do we make sure that students have learned what they need to learn to be successful in the world?

Now to test your understanding:

Which statement best describes this blog writer’s point of view?

  1. Multiple choice questions are easy to write.
  2. Multiple choice questions test critical thinking.
  3. We should rethink how we assess learning.
  4. We should never use multiple choice questions.

Learning Objectives in Higher Education

Education Strategy, , , , , , , ,

Not everything that can be counted counts, and not everything that counts can be counted.     Albert Einstein

Learning is a cognitive activity. According to the Psychology Dictionary, cognition encompasses “the mental processes in gaining knowledge and comprehension.”

In the 1950’s, the University of Chicago’s Benjamin Bloom invented a cognitive taxonomy which was embraced by American educators. Revised in 2001 by Lorin Anderson and David Krathwohl, the taxonomy became more dynamic, reflecting the interactivity of contemporary education contexts. The list has six categories. It moves from the simplest form of cognition, knowledge/remembering, to the most complex, synthesis/evaluation/creating.*

For educators, Bloom provided a scientific way to categorize learning. If educators could connect Bloom’s taxonomy to teaching and learning, they would have a much more sophisticated way of measuring instructional and student accomplishment.

At the same time that Bloom was creating his taxonomy, the idea of writing formal learning objectives in education was gaining traction. Bernard Bull does a great job blogging on the history of learning objectives   Learning objectives are statements that describe an act that can be measured. A learning objective always includes a verb and a specific goal to be achieved, showing learning through performance. Typical objectives assume “the student will be able to”:

  • Solve one-variable equations.
  • Describe the importance of audience in public speaking.
  • Compare the strategies of Lee and Grant during the Civil War.

K-12 uses cognitive categories and learning objectives to frame teaching, student content, and test questions. The core of the Common Core involves cognitive taxonomy and learning outcomes.

Learning objectives and their measurement are very concrete. They focus on cognitive growth in the present. So there is a contradiction when calls for accountability and job training require higher education institutions to develop learning objectives.

Colleges have traditionally provided students with a broad education meant to resonate in the future. And, they have set as their goals helping students become better people, responsible citizens in one way or another. The recent news about university responsibility in judging behaviors such as rapes on campus is rooted in this type of higher education mission.

We expect college graduates to be prepared for dealing with life outside their majors.We need to preserve the cultivation of this type of learning in higher education. Learning objectives may be reasonable and helpful in some curricula, but they are not the answer to creating a higher education environment that helps students succeed in life as well as work. We cannot allow learning objectives advocates to make learning all about what you can assess, what you can “see” now.

We need to resist the push in higher education to make the learning that can easily be measured the heart of 21st century education. We need to find ways to emphasize and reward learning that “can’t be counted” as much as that which “can be counted.”

*

Bloom’s Original/Revised Description
Knowledge/Remembering Recall facts and basic concepts.
Comprehension/Understanding Connect concepts through interpretation and organization.
Application/Applying Solve problems using acquired knowledge.
Analysis/Analyzing Use evidence, knowledge and data to draw conclusions, infer, and conclude.
Synthesis/Evaluating Create a plan or product based on elaborated ideas/Present opinions based on criteria.
Evaluation/Creating Assess based on criteria/Present new ideas or solutions by innovative organization of evidence