Robert Bjork: Remembering, Forgetting, and Desirable Difficulties

At this week’s Biology Leadership Conference (#BLC8), we were fortunate to hear a talk given by Dr. Robert Bjork, a cognitive psychologist (UCLA) who conducts research on remembering, forgetting, and desirable difficulties.  This is going to be a loooong post, so I’ll allow those of you less interested to skip past this one.  For those interested in remembering…forgetting…and desirable difficulties, please click to continue reading…

Dr. Bjork gave us a very interesting presentation on his research and the intricacies of our human memory system, which he described as a symbiosis of forgetting, remembering, and learning – a remarkable interaction. Equally amazing, says Dr. Bjork, are all of our – the users of the system – misunderstandings as to how memory and learning works. Our judgment, explains Bjork, about what we will remember or forget is not very reliable. He went onto describe important peculiarities of the human memory system:

– A remarkable capacity for storing information (might as well assume it’s unlimited)

– It doesn’t work like a tape or a CD where the more you store, the less room you have.  With our brains, the more you learn (attempt to store) the more room there is.

– A highly fallible retrieval process

– Retrieving information from memory is a dynamic process that alters the subsequent state of the memory

– What is accessible in memory is heavily influenced by environment, interpersonal, emotion, and body-state cues.

–  What you retrieve now becomes even more accessible in the future.

– Conditions that produce forgetting, create opportunities for additional learning

Dr. Robert Bjork at the BLC8

Dr. Bjork presented example after example of “common wisdom” about effective studying, ideas we hold dear about the best way to learn, that his lab’s research has brought into question.  Take for example, context change (studying in more than one place). Common wisdom has it that we should find a quiet, consistent place to study – a library, a dedicated office or corner of the room. But no.  Bjork’s research shows that context change – studying in place A, then place B, and then place C actually enhances the learning.  In fact, in general, conditions of instruction that appear to create difficulties for the earner, to slow the rate of apparent learning, often optimize long-term retention and transfer.

Dr. Bjork and his research colleagues have dubbed these difficulties “desirable difficulties” – an apparent oxymoron that nicely sums up the gist of his work.  Other examples of introduced desirable difficulties that can enhance retrieval and transfer include spacing study sessions, using tests as learning events, and providing contextual interference during learning (e.g. interleaving rather than blocking practice).

To illustrate, Dr. Bjork summarized a fascinating study by Baddeley and Longman (1979) involving the training schedule for British postal workers. The workers had to be trained in a new, complex keyboard procedure.  As an experiment, some of the workers learned the new skills in widely spaced sessions, while some were trained in massed sessions (that is, one session quickly following another).  Our instincts would tell us that the “massed” workers would have better retention results but it was the spaced-session workers who out-performed the massed.  Even more interesting than that, when the workers were asked how satisfied they were with their training, there was a complete negative correlation between their results and their satisfaction rating. Those that did the worst were the most satisfied with their instruction.

Dr. Bjork refers to this phenomenon as “the illusion of knowing” – that is, we think we know more than we do.  He brought out another experiment that tested four different study (“S”) and testing (“T”) patterns with learners.  Here were the four patterns:

1. S S S S —T

2. S S S T—T

3. S S T T –T

4. S T T T — T

Of those options, which do you predict will perform best?  Most of us (including me) predict #1, the four sequential study sessions, understanding, as we do, that repetition helps us to remember.  But in fact, it’s #4.  The group with only one study session, followed by three short testing sessions – and then a final assessment – outperformed the other three groups.

Hooboy.  At this point I was concerned that Dr. Bjork was on the “high- stakes testing” bandwagon (which doubtless would have caused the entire room of thoughtful, earnest college faculty to cross their arms and close their ears). But far from it. Dr. Bjork, who has obviously thought long and hard about learning, urged us to think of testing as pedagogy; to use testing as a vehicle for learning. As he describes it, tests potentiate subsequent learning. Even time spent coming up with a wrong answer helps the learner get to the correct answer – it facilitates the amping and encoding process.

So how do we make practice tests that constitute good pedagogy?  Bjork’s advice: make the testing experiences low stakes, frequent, and when designing the test, select distracters carefully.  Choose competing alternative answers so that they trigger retrieval of other information (that might be tested later).  In designing an objective test in this manner, we help the student beyond the performance on that one individual item.

Dr. Bjork also talked with us about the notion of interleaved vs. blocked learning.  As the names imply, interleaved learning means mixing up the material you are studying.  So, for instance, if you are trying to learn U.S. capital cities, don’t just rehearse the names for an hour – study them for a few minutes, then turn to another topic, and back again to the capital cities. Interleave your study sessions on one topic with other work on another. In his research, he’s shown that blocked learners have poorer performance and their judgment (prediction) about how well they will do with the task in 24 hours is waaaay off.

This is completely counterintuitive to our experience in life. When you think of it, “blocking” is everywhere in our world – military drills, training for athletic skills, workplace learning, learning to read and write, learning a foreign language, the very fabric of our K-12 education system.  It’s our instinct to break a task down into component parts, and then repeat them in order over and over again.

You would think that there might be some situations where blocking would be the best approach. For instance, induction or the need to pick out what unifies a category or a concept.  But no.  Bjork showed us a fascinating study where learners looked at paintings by three different artists.  The control group subjects were shown the paintings in sequence (massed) and the experimental group were shown the paintings in a spaced (interleaved) manner.  Then all of the subjects were shown a new painting, by one of the artists, and asked to identify it.  Those that examined the paintings in a spaced/interleaved way were better able to identify the unknown painting. A similar experiment was conducted, asking subjects to identify different species of butterflies with the same results.

So why is this?  What is happening in our brains when we “mass” versus “interleaf” our learning?  Dr. Bjork speculates that the massing gives you a false sense of security –  you think you’re getting fluent.  By contrast, the interleaving sets up some anxiety, things are out-of-order, and the  interleaving highlights how the paintings differ.

But are there some other conditions, or other learners, who might benefit from massing or blocking – say, for instance, the elderly or the very young?

He went on to describe similar experiments with elderly subjects and with 3 year olds. The 3 year olds, they were asked to identify made-up toys – wugs and blickets (made up toys) – either through blocked exposure or interleaved exposure.  The same effect was observed – there were substantial benefits to the learner with the interleaved approach.

So, what are the implications here for the design of instruction?  Dr. Bjork recommends variation, interleaving, spacing, and using tests as learning events.  These results also have implications for the evaluation of instruction – how do our students evaluate their education?  We ask them about their satisfaction but how actionable are their opinions?  Just consider for a moment how many ways our students’ course organization expectations conflict with these new insights.

Here are some examples of common attitudes and assumptions that Dr. Bjork explains will get in our way:

– We misunderstand the meaning and the role of errors in learning (optimal instruction should always include making errors)

– Aptitude is over appreciated

– The assumption that efficient learning is easy (or best) learning

– Over-attributing differences in performance to innate difference in ability

– Over reliance on “styles of learning” (which Dr. Bjork says has no scientific basis)

Dr. Bjork speculated with us about why the notion of different “learning styles” is such an attractive one.  It’s because it’s comforting.  It’s an explanation for why I’m not doing so well.  As in… “If only the instructor would teach to my style, then I would do just fine.”  To illustrate his point, Bjork shared this satirical article from The Onion, lamenting the needs of nasal-learners.

copyright of The Onion

To wrap up his talk, Dr. Bjork cited another fascinating learning experiment conducted by Elizabeth Newton (1990), which he referred to as a “parable of teaching”.  In Newton’s experiment she designated subjects as either “listeners” or “tappers”.  The tappers would tap out a well-known melody for the listeners and ask them to identify the tune they’d just tapped out.  To demonstrate, Dr. Bjork tapped out a rhythm on the podium.  All 90 of us sat dumbfounded, unable to identify the melody…what was it?  “You don’t recognize that?” asked Bjork.  “It’s so obvious!!!” (in a playful, mocking tone) “Couldn’t you pick out the familiar tune of Row, Row, Row Your Boat??!!”  But no, we couldn’t.  A childhood tune that we’ve all sung countless times was indecipherable to us. But to Dr. Bjork, the tapper, the answer was completely obvious.  He was hearing the tune in his head as he tapped and fully expected that we would be hearing it too.

In Newton’s experiment, the tappers try to predict whether the listener would succeed in naming the melody.  By this point, I’m sure you won’t be surprised to learn that the listener’s actual success rate was much, much worse than the prediction on the part of the tapper.

At the conclusion of Bjork’s wonderful talk, Dr. Mary Pat Wenderoth (University of Washington) shared some strategies she’s used to apply the principles behind Dr. Bjork’s research to her introductory biology course at University of Washington.  Here were her suggestions:

– Be transparent with students – share this research information with them and give them some insight to their own learning patterns and insight into the optimal conditions for learning.

– By sharing this knowledge with her students, she’s found that it gives her a commonly understood vocabulary to use when talking with her students

– Switch from study groups to “testing groups”.

– Assign one-page pictorial summary sheets, due at the end of the week.  She’s learned from experience to allow them to work on these in groups, to encourage free form (whatever works for you), and to not give them an example summary sheet (when she did that, they all mimicked the form of the example).  Here’s an example of one of her student’s summary sheets:

-Assign a “Learning Paragraph” at the end of each week.  This is a one-to-two paragraph assignment, where they reflect on their learning and make connections to everyday life.  She has them submit the Learning Paragraph electronically in a Google Form-type set up, so that she can quickly scan the submitted paragraphs.  The primary benefit here is, of course, for the student to help them reflect on their own learning, but she also can monitor how their insights and patterns change over the course of the semester.

What a treasure trove of insights to consider and new methods to try. In our work as educators, it will serve us well to remember that befuddled feeling of the listener as we “tap out” information that is so powerfully clear to us a long-time tappers.


Filed under Reflections on Teaching

22 responses to “Robert Bjork: Remembering, Forgetting, and Desirable Difficulties

  1. jchoigt

    Great summary, Robin! I’m passing this on to all our instructors. I’d love to see some sharathon discussions next year about the ways we tried to incorporate “desirable difficulty” into our classes.

  2. rheyden

    Thanks, Jung. Yes, yes! Ongoing discussion about the practicalities of this – great idea.

  3. GenericMerrill

    Great summary Robin! Thanks. I like Jung’s idea, too. But then the pressure is on to do all the things I learned about. Interestingly, I’ve been toying for some time with having students create what I call “cheat sheets”. Put whatever you want on one piece of paper and you can bring it to the exam. Idea is pretty much like Mary Pat’s: they have to go through the thought process of “what is important” and get it written/drawn.

    • jchoigt

      Hi John,
      I have allowed students to create and bring a crib sheet – a single 8.5 x 11 in sheet of paper with whatever they have written/drawn, to the exam for the cells and energy metabolism unit. My rule is that it has to be hand-drawn (no photocopies or prints). It allows me to ask more conceptual and what-if questions, and I think it’s a great review exercise for them.

  4. rheyden

    Hi John, Good to see you here – and glad you liked the Bjork summary. Love the “cheat sheet” activity idea. Just the act of creating it would be an assist for most students. All the careful weighing and judging about what’s “sheet-worthy”, right?

    • GenericMerrill

      I think the process of building the cheat sheet is more important than any benefit of using it during the exam. The student might be more likely to think the reverse, however, and therefore put more effort into building it! I believe that it becomes a mental “picture” that the student can conjure up on demand while agonizing over an exam question: “Oh man, I know this… Oh yeah, I had it drawn in the lower left corner… Ah, now I remember! Got it!”

  5. Lenora

    I always chide myself if I go down a “rabbit trail” – something I refer to as a Life Lesson – Biology is infused with them! But this research suggests that the interuptions in a stream of teaching may actually be beneficial. I definitely agree with the idea that changing study locations can be beneficial. I think this is true for an individual, small group and even an entire class. Field trip anyone???

  6. rheyden

    Well said, Lenora. And I like the idea of going down a “rabbit trail”. Who knew that all that stuff we berated ourselves over was actually good for us?! ha!

  7. Part of being a good learner is recognizing that being wrong is not such undesirable part of the process. True, testing makes us think this is a weakness, but in the scheme of things, it is what keeps us learning.

    Thanks for an excellent summary, Robin!

  8. Pingback: How to Fail Smarter: The Goldilocks Rule « The Talent Code

  9. Pingback: December 30th, 2011 Weekly Update | Hillsboro HEAT Swim Team

  10. Really interesting stuff. Thanks for summarizing this for us. I’ve tweeted out a few of the really good points. yhx

  11. rheyden

    Thanks for the tweet and the props, Michael.

  12. Hi Robin,
    Your post came to me from a friend as a result of a rousing discussion on Facebook over this article:

    I think I met you as a friend of Ethel’s/at a professional biology conference/user of Campbell (switched books recently though). I have heard Dr. Wenderoth talk at a regional biology ed conference as well. We should get all these great minds in the same place to talk about biology education in the context of ed psych!

    • rheyden

      Hi Stacey! Great to see you here. Yes, I remember meeting you at the Biology Leadership Conference. Dr. Wenderoth is really terrific, isn’t she? I see from your Facebook page that you also know Brad WIlliamson – seems you and I know a lot of the same biology ed folks. I’m looking for a good speaker on doing scienced education research for biology instructors – any suggestions for me? And thanks for the link to the Clark/Kirschner article – yes, I can well imagine that that generated a ton of discussion!

  13. Hi there.
    Does the article that Stacey linked advocate a different approach to what Dr robert Bjork suggests?.

    So Does Staceys artcile suggest that for learning you need to study worked examples, and not burden yourself with problem solving, whereas Dr Bjork suggests problem solving and testing is key for memory retrieval.

  14. Pingback: Veronica Yan: More on Learning and Desirable Difficulties | Jung's Biology Blog

  15. Pingback: Retrieval Practice and Ed Tech | Stepping Stones

  16. Pingback: Independent Learning week 3: ‘Desirable Difficulties’ part 2

  17. Pingback: Learning Methods: Interleaving – A Way to Learn Faster and Better -

  18. Pingback: Ask A Researcher #1 - Dr. Robert Bjork - The Effortful Educator

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s