Category Archives: Education Strategy

Multiple Choice Questions

Education Strategy, , , , , , ,

A multiple choice question begins with a stem or lead-in that is addressed by a correct response chosen from a list of alternatives. Writing a good multiple choice question that elicits an answer based on knowledge, not guessing or misunderstanding, is an art. For example:

Who was the twentieth president of the United States?

  1. Rutherford B. Hayes
  2. James A. Garfield
  3. Chester A. Arthur
  4. Grover Cleveland

This question tests recall of the twentieth president. The stem is parsimonious, including only the ideas and words necessary to answer the question. The “distractors” are parallel, possible answers–all presidents from around the same time. Compare to this question:

Choosing the first president of the United States was a tremendous responsibility. He would set precedents for subsequent office holders. The Electoral College unanimously elected George Washington who had led the colonies to victory against the British. James Madison, who was married to Dolly, was the fourth president. Who was the twentieth?

  1. James Brown
  2. LeBron James
  3. James Garfield
  4. James Bond

In this question, the stem is overwritten with information you don’t need to answer the question correctly. Irrelevant information may be testing your reading comprehension more than your twentieth president knowledge. Even if you know the correct answer, you may get it wrong because you can’t get through the reading.

The distractors are implausible. If the correct answer is embedded in a group of possibilities that are totally outlandish, you will get the right answer not because you’ve learned it, but because you can use general knowledge to eliminate the others. That’s a bad question.

If written correctly, a multiple choice question can be very effective at proving mastery in Bloom’s elementary cognitive categories of remembering and understanding, and to a lesser extent in the third category, applying (see previous blog Learning Objectives in Higher Education).

According to Cathy Davidson, educator Frederick J. Kelly introduced multiple choice tests in 1914.  They were intended to improve the equality of grading. Teacher bias as well as individual differences such as wealth or poverty would not prevent a student from being graded correctly. Multiple choice questions also made grading less time-consuming for teachers, freeing them to do more instruction. Incorporated in standardized tests, multiple choice questions allowed us to compare student proficiency in different areas of the country. Good goals, right?

Don’t we share these goals today: To evaluate students without bias. To give them equal opportunity to learn despite where they live or learn. To free instructors to have more time to teach and interact with students. So why are multiple choice questions criticized so much?

Davidson says it’s because we try to use multiple choice questions in areas where they don’t work such as

….problem solving, collaborative thinking,  interdisciplinary thinking, complex analysis, the ability to apply learning to other problems, complexity…creativity, imagination, originality…

Demonstration of these types of learning, Bloom’s applying, analyzing, evaluating, and creating, requires more than picking out the right answer if there even is a “right” answer. Kelly created multiple choice questions to measure basic skills important to twentieth century American work and citizenship. He admitted that they only tested “lower-order” thinking.

Extending the multiple choice format to measure higher-order thinking results in many flawed questions. Piled one on top of the other in repetitive quizzes or long tests, these ill-conceived items become anxiety-provoking, deadening experiences for students. In this context, they are weak indicators of student learning achievement.

Through digital programming we have the potential to create robust profiles of students showing how they process, retain, and apply information. This gives us the opportunity to approach the challenge of assessing student performance from a fresh perspective, one that may even use testing rarely. Let’s start by identifying the problem we want to solve: How do we make sure that students have learned what they need to learn to be successful in the world?

Now to test your understanding:

Which statement best describes this blog writer’s point of view?

  1. Multiple choice questions are easy to write.
  2. Multiple choice questions test critical thinking.
  3. We should rethink how we assess learning.
  4. We should never use multiple choice questions.

Think Outside the Box

Education Strategy, , ,

“Think outside the box” has become a way of asking people to be creative. The idiom is based on a psychological concept, functional fixedness:  Very simply, when we learn about something, the meaning and its first associations get stuck in our memories and prevent us from thinking about it differently. The experiment that proved this included observing whether participants would use an actual box in a new way.

Recently, Nicholas Kristof described great examples of thinking outside the box in his NY Times column on HeroRats: Three-foot long rats are being taught to sniff out mines in minefields resulting in detection that is twenty times faster and safer than humans. And, the same type of rats are being used to diagnose tuberculosis about one hundred times faster than humans.

While I’m amazed at what the rats can do, I’m more amazed that someone actually thought of doing this. Bart Weetjens, founder of Apopo, the non-profit that trains and disperses the rats, got the idea by connecting his experience with rats as pets to what he learned about gerbils used for detecting smells. Incredible, right?

For most of us, functional fixedness would have gotten in the way of creating HeroRats. If you learned that rats are harmful rodents, you probably wouldn’t think that they could be used to do good. Similarly, if you learned that detecting disease is done through lab tests, you wouldn’t think that using a non-technological solution is a better way to recognize it.

While functional fixedness can get in the way of new ideas, fear of risk-taking or just inexperience can keep us from acting on them. So if we want to encourage better ways of doing things, we need strategies to teach students that it’s okay to present something unconventional, to be creative.

In Finding a Place for Creative Assignments, Maryellen Weiner gives examples of three strategies in higher education courses: learner-centering the course; requiring a poem describing what students want to get out of a course; and having students present their reflections on the process of learning in the class.

I would like to add another strategy: freestyling. Freestyling is associated with rappers who build rhymes off of random words from the audience or in competition with each other. This is all done in real time; it’s instant free association and creativity. And it’s fun!

In class, the students are the audience. Suppose each student is asked to write a random word or phrase that seems to have nothing to do with the subject of the class on a piece of paper. The suggestions are put into a receptacle. Then groups or individuals each pick one out. They are given a specific amount of time to prepare a brief presentation (not in rhyme) connecting the chosen words to a general class study topic such as the Depression in US History or depression in Psychology or the Renaissance in Art History.

My hypothesis: This type of exchange will grease students’ inner wheels of creativity and lead to a lively, productive discussion. If you try it, let me know how it works out.

Learning Objectives in Higher Education

Education Strategy, , , , , , , ,

Not everything that can be counted counts, and not everything that counts can be counted.     Albert Einstein

Learning is a cognitive activity. According to the Psychology Dictionary, cognition encompasses “the mental processes in gaining knowledge and comprehension.”

In the 1950’s, the University of Chicago’s Benjamin Bloom invented a cognitive taxonomy which was embraced by American educators. Revised in 2001 by Lorin Anderson and David Krathwohl, the taxonomy became more dynamic, reflecting the interactivity of contemporary education contexts. The list has six categories. It moves from the simplest form of cognition, knowledge/remembering, to the most complex, synthesis/evaluation/creating.*

For educators, Bloom provided a scientific way to categorize learning. If educators could connect Bloom’s taxonomy to teaching and learning, they would have a much more sophisticated way of measuring instructional and student accomplishment.

At the same time that Bloom was creating his taxonomy, the idea of writing formal learning objectives in education was gaining traction. Bernard Bull does a great job blogging on the history of learning objectives   Learning objectives are statements that describe an act that can be measured. A learning objective always includes a verb and a specific goal to be achieved, showing learning through performance. Typical objectives assume “the student will be able to”:

  • Solve one-variable equations.
  • Describe the importance of audience in public speaking.
  • Compare the strategies of Lee and Grant during the Civil War.

K-12 uses cognitive categories and learning objectives to frame teaching, student content, and test questions. The core of the Common Core involves cognitive taxonomy and learning outcomes.

Learning objectives and their measurement are very concrete. They focus on cognitive growth in the present. So there is a contradiction when calls for accountability and job training require higher education institutions to develop learning objectives.

Colleges have traditionally provided students with a broad education meant to resonate in the future. And, they have set as their goals helping students become better people, responsible citizens in one way or another. The recent news about university responsibility in judging behaviors such as rapes on campus is rooted in this type of higher education mission.

We expect college graduates to be prepared for dealing with life outside their majors.We need to preserve the cultivation of this type of learning in higher education. Learning objectives may be reasonable and helpful in some curricula, but they are not the answer to creating a higher education environment that helps students succeed in life as well as work. We cannot allow learning objectives advocates to make learning all about what you can assess, what you can “see” now.

We need to resist the push in higher education to make the learning that can easily be measured the heart of 21st century education. We need to find ways to emphasize and reward learning that “can’t be counted” as much as that which “can be counted.”

*

Bloom’s Original/Revised Description
Knowledge/Remembering Recall facts and basic concepts.
Comprehension/Understanding Connect concepts through interpretation and organization.
Application/Applying Solve problems using acquired knowledge.
Analysis/Analyzing Use evidence, knowledge and data to draw conclusions, infer, and conclude.
Synthesis/Evaluating Create a plan or product based on elaborated ideas/Present opinions based on criteria.
Evaluation/Creating Assess based on criteria/Present new ideas or solutions by innovative organization of evidence

 

Extrinsic Versus Intrinsic Motivation

Education Strategy, , , , , ,

Educating students in school relies on motivating them to learn. Studies and discussions on the meaning, application, and benefits of extrinsic and intrinsic motivation are happening as digital programs are created and instructors are incorporating them into their curriculums.

Very broadly, extrinsic motivation comes from outside in the form of stimuli or rewards such as ranking, salary, badges, and praise. Intrinsic motivation is prompted by inner feelings such as curiosity, need for belonging, and satisfaction. How extrinsic and intrinsic motivation can sometimes relate to each other is the moral of a story that has made the rounds in psychology circles for years:

A group of children go every day after school to chant anti-Semitic remarks in front of a store owned by a Jewish man. One day the man comes out to the children and says, “This is what you call yelling? I can hardly hear you. If you promise to yell more loudly, I will pay you each a dime.”

The children accept and every day, after they yell for a while, the store owner gives each a dime. After about two weeks, the store owner comes out and says, “You’ve been doing a great job; but business has been bad lately, and I can only pay you each a nickel.” The children protest, but they accept the reduced payment.

After another couple of weeks,  the store owner comes out to say, “I’m sorry, but I can’t afford you anymore. You’re welcome to continue, but I won’t be able to pay you.” The children reply, “You think we are going to do this for nothing? No way,” and they never come back again.

What has happened here? The intrinsic motivation of fun or satisfaction, (no matter how twisted) that spurred the children to act in the first place was replaced by extrinsic motivation (money) and when the external reward disappeared, so did the original intrinsic motivation. This is a simplistic analysis, but the point is clear: Be careful with extrinsic rewards.

Schools have already tied learning to extrinsic rewards: grades, test scores, medals, badges, diplomas. Isn’t that enough? These rewards have robbed many students of a love of learning. We can’t let that continue.

The world of phenomenal change that we live in makes lifelong learning more important than ever. Ditto for intrinsic motivation. We need to make sure that students desire learning without all the bells and whistles.

In the past, a higher education graduate could choose a career and plan to thrive in it through experience and some professional development. Now, we are expected to learn new technologies and ways to communicate every year. Instructors are expected to integrate technology as they teach.

The best preparation for success is knowing how to learn and to want to keep learning. Creating a deep consonance between internal satisfaction and learning in students will ensure that they are able to initiate and navigate the changes at the core of 21st century life.

For a more data-based scientific explanation of how extrinsic motivation can ruin intrinsic motivation see “Extrinsic Rewards and Intrinsic Motivation in Education: Reconsidered Once Again” by Deci, Koestner, and Ryan.

Failing is for Everyone

Education Strategy, , , ,

The world of entrepreneurs is running hot these days. Innovation is at the heart of entrepreneurship and to be successful that innovation must fill a need. Successful innovators identify a problem to solve.  Examples: Venmo, an electronic wallet app (Wouldn’t it be great if, after a shared dinner, we could pay each other back through our phones?), Tinder, a quick dating app (Wouldn’t it be easier to just hook up with people without all the requirements of on-line dating?), or Teachley, an EdTech company basing programs on cognitive science (How can teachers use app data to make faster, better decisions about student learning needs?).

Entrepreneurs know that they should “fail fast” because it’s a waste of time and money working on something that ultimately is not going to succeed. Entrepreneurs work fast, hard, and cheaply to create basic prototypes called MVP’s (Minimally Viable Products) that they can try out on friends, family, and market segments to find out quickly: Is this a product that people want? Will this work? How can I make it better? The product may or may not turn out to be something to pursue.

To an entrepreneur, failure is part of the process, often a badge of honor. Many successful entrepreneurs include “I failed X times and I learned from my failures” in their speeches to help motivate others. Success follows from failure if you are able to accept failure and learn from it.

Learning is a personal process of innovation. When you learn, you encounter, try out, and adapt new knowledge and ways of thinking. So why are students punished for failure: bad grades, not getting a diploma, embarrassment. Those who fail consistently often develop a form of learned helplessness: Psychologists have found that if people fail enough times without understanding, they will just give up trying. School for many children is a dead end of failing or a deflating experience of average accomplishment.

How can we turn students into learning entrepreneurs, learning from their mistakes, and trying again? One way is through transparency. We should incorporate ongoing review with students of their learning pathways. Explain how learning A leads to B. When they fail A, suggest other ways to get to B.

Another solution is to make problem solving and hard work the center of learning. Carol Dweck, Stanford University, has shown colossally positive results with children in failing schools by changing their attitudes to a “growth mindset,” a belief that abilities are not fixed, they can be developed through investing time and energy.

In her TEDx talk, Dweck gives an example of a Chicago school where students who fail are given a “not yet” rather than an F. The “not yet” signals that they are on a trajectory, they are on a journey. Failure is not a stop; it’s not a reason to give up. It’s just a bump in the road. Figure out how to get past it and you will keep moving forward. That’s a lesson we all should learn.

Read more on what educators can learn from entrepreneurial perspective on failure in Kathleen Parker’s Washington Post article, “How we succeed by failing.”