Combining retrieval, spacing, and feedback boosts STEM learning — from retrievalpractice.org

Punchline:
Scientists demonstrated that when college students used a quizzing program that combined retrieval practice, spacing, and feedback, exam performance increased by nearly a letter grade.

—-

Abstract
The most effective educational interventions often face significant barriers to widespread implementation because they are highly specific, resource intense, and/or comprehensive. We argue for an alternative approach to improving education: leveraging technology and cognitive science to develop interventions that generalize, scale, and can be easily implemented within any curriculum. In a classroom experiment, we investigated whether three simple, but powerful principles from cognitive science could be combined to improve learning. Although implementation of these principles only required a few small changes to standard practice in a college engineering course, it significantly increased student performance on exams. Our findings highlight the potential for developing inexpensive, yet effective educational interventions that can be implemented worldwide.

In summary, the combination of spaced retrieval practice and required feedback viewing had a powerful effect on student learning of complex engineering material. Of course, the principles from cognitive science could have been applied without the use of technology. However, our belief is that advances in technology and ideas from machine learning have the potential to exponentially increase the effectiveness and impact of these principles. Automation is an important benefit, but technology also can provide a personalized learning experience for a rapidly growing, diverse body of students who have different knowledge and academic backgrounds. Through the use of data mining, algorithms, and experimentation, technology can help us understand how best to implement these principles for individual learners while also producing new discoveries about how people learn. Finally, technology facilitates access. Even if an intervention has a small effect size, it can still have a substantial impact if broadly implemented. For example, aspirin has a small effect on preventing heart attacks and strokes when taken regularly, but its impact is large because it is cheap and widely available. The synergy of cognitive science, machine learning, and technology has the potential to produce inexpensive, but powerful learning tools that generalize, scale, and can be easily implemented worldwide.

Keywords: Education. Technology. Retrieval practice. Spacing. Feedback. Transfer of learning.

 

 
 

Introducing several new ideas to provide personalized, customized learning experiences for all kinds of learners! [Christian]

From DSC:
I have often reflected on differentiation or what some call personalized learning and/or customized learning. How does a busy teacher, instructor, professor, or trainer achieve this, realistically?

It’s very difficult and time-consuming to do for sure. But it also requires a team of specialists to achieve such a holy grail of learning — as one person can’t know it all. That is, one educator doesn’t have the necessary time, skills, or knowledge to address so many different learning needs and levels!

  • Think of different cognitive capabilities — from students that have special learning needs and challenges to gifted students
  • Or learners that have different physical capabilities or restrictions
  • Or learners that have different backgrounds and/or levels of prior knowledge
  • Etc., etc., etc.

Educators  and trainers have so many things on their plates that it’s very difficult to come up with _X_ lesson plans/agendas/personalized approaches, etc.  On the other side of the table, how do students from a vast array of backgrounds and cognitive skill levels get the main points of a chapter or piece of text? How can they self-select the level of difficulty and/or start at a “basics” level and work one’s way up to harder/more detailed levels if they can cognitively handle that level of detail/complexity? Conversely, how do I as a learner get the boiled down version of a piece of text?

Well… just as with the flipped classroom approach, I’d like to suggest that we flip things a bit and enlist teams of specialists at the publishers to fulfill this need. Move things to the content creation end — not so much at the delivery end of things. Publishers’ teams could play a significant, hugely helpful role in providing customized learning to learners.

Some of the ways that this could happen:

Use an HTML like language when writing a textbook, such as:

<MainPoint> The text for the main point here. </MainPoint>

<SubPoint1>The text for the subpoint 1 here.</SubPoint1>

<DetailsSubPoint1>More detailed information for subpoint 1 here.</DetailsSubPoint1>

<SubPoint2>The text for the subpoint 2 here.</SubPoint2>

<DetailsSubPoint2>More detailed information for subpoint 2 here.</DetailsSubPoint2>

<SubPoint3>The text for the subpoint 3 here.</SubPoint3>

<DetailsSubPoint3>More detailed information for subpoint 3 here.</DetailsSubPoint1>

<SummaryOfMainPoints>A list of the main points that a learner should walk away with.</SummaryOfMainPoints>

<BasicsOfMainPoints>Here is a listing of the main points, but put in alternative words and more basic ways of expressing those main points. </BasicsOfMainPoints>

<Conclusion> The text for the concluding comments here.</Conclusion>

 

<BasicsOfMainPoints> could be called <AlternativeExplanations>
Bottom line: This tag would be to put things forth using very straightforward terms.

Another tag would be to address how this topic/chapter is relevant:
<RealWorldApplication>This short paragraph should illustrate real world examples

of this particular topic. Why does this topic matter? How is it relevant?</RealWorldApplication>

 

On the students’ end, they could use an app that works with such tags to allow a learner to quickly see/review the different layers. That is:

  • Show me just the main points
  • Then add on the sub points
  • Then fill in the details
    OR
  • Just give me the basics via an alternative ways of expressing these things. I won’t remember all the details. Put things using easy-to-understand wording/ideas.

 

It’s like the layers of a Microsoft HoloLens app of the human anatomy:

 

Or it’s like different layers of a chapter of a “textbook” — so a learner could quickly collapse/expand the text as needed:

 

This approach could be helpful at all kinds of learning levels. For example, it could be very helpful for law school students to obtain outlines for cases or for chapters of information. Similarly, it could be helpful for dental or medical school students to get the main points as well as detailed information.

Also, as Artificial Intelligence (AI) grows, the system could check a learner’s cloud-based learner profile to see their reading level or prior knowledge, any IEP’s on file, their learning preferences (audio, video, animations, etc.), etc. to further provide a personalized/customized learning experience. 

To recap:

  • “Textbooks” continue to be created by teams of specialists, but add specialists with knowledge of students with special needs as well as for gifted students. For example, a team could have experts within the field of Special Education to help create one of the overlays/or filters/lenses — i.e., to reword things. If the text was talking about how to hit a backhand or a forehand, the alternative text layer could be summed up to say that tennis is a sport…and that a sport is something people play. On the other end of the spectrum, the text could dive deeply into the various grips a person could use to hit a forehand or backhand.
  • This puts the power of offering differentiation at the point of content creation/development (differentiation could also be provided for at the delivery end, but again, time and expertise are likely not going to be there)
  • Publishers create “overlays” or various layers that can be turned on or off by the learners
  • Can see whole chapters or can see main ideas, topic sentences, and/or details. Like HTML tags for web pages.
  • Can instantly collapse chapters to main ideas/outlines.

 

 

Top Trends in Active and Collaborative Learning — from thesextantgroup.com by Joe Hammett

Excerpts:

My daughter is a maker. She spends hours tinkering with sewing machines and slime recipes, building salamander habitats and the like. She hangs out with her school friends inside apps that teach math and problem solving through multi-player games. All the while, they are learning to communicate and collaborate in ways that are completely foreign to their grandparent’s generation. She is 10 years old and represents a shift in human cognitive processing brought about by the mastery of technology from a very young age. Her generation and those that come after have never known a time without technology. Personal devices have changed the shared human experience and there is no turning back.

The spaces in which this new human chooses to occupy must cater to their style of existence. They see every display as interactive and are growing up knowing that the entirety of human knowledge is available to them by simply asking Alexa. The 3D printer is a familiar concept and space travel for pleasure will be the norm when they have children of their own.

Current trends in active and collaborative learning are evolving alongside these young minds and when appropriately implemented, enable experiential learning and creative encounters that are changing the very nature of the learning process. Attention to the spaces that will support the educators is also paramount to this success. Lesson plans and teaching style must flip with the classroom. The learning space is just a room without the educator and their content.

 


8. Flexible and Reconfigurable
With floor space at a premium, classrooms need to be able to adapt to a multitude of uses and pedagogies. Flexible furniture will allow the individual instructor freedom to set up the space as needed for their intended activities without impacting the next person to use the room. Construction material choices are key to achieving an easily reconfigurable space. Raised floors and individually controllable lighting fixtures allow a room to go from lecture to group work with ease. Whiteboard paints and rail mounting systems make walls reconfigurable too!.

Active Learning, Flipped Classroom, SCALE-UP, TEAL Classroom, whatever label you choose to place before it, the classroom, learning spaces of all sorts, are changing. The occupants of these spaces demand that they are able to effectively, and comfortably, share ideas and collaborate on projects with their counterparts both in person and in the ether. A global shift is happening in the way humans share ideas. Disruptive technology, on a level not seen since the assembly line, is driving a change in the way humans interact with other humans. The future is collaborative.

 

 

 

From DSC:
Ever notice how effective Ted Talks begin? They seek to instantly grab your attention with a zinger question, a somewhat shocking statement, an interesting story, a joke, an important problem or an issue, a personal anecdote or experience, a powerful image/photo/graphic, a brief demonstration, and the like.

Grabbing someone’s attention is a key first step in getting a piece of information into someone’s short-term memory — what I call getting through “the gate.” If we can’t get through the gate into someone’s short-term memory, we have zero (0) chance of having them actually process that information and to think about and engage with that piece of content. If we can’t make it into someone’s short-term memory, we can’t get that piece of information into their long-term memory for later retrieval/recall. There won’t be any return on investment (ROI) in that case.

 

 

So why not try starting up one of your classes this week with a zinger question, a powerful image/photo/video, or a story from your own work experience? I’ll bet you’ll grab your students’ attentions instantly! Then you can move on into the material for a greater ROI. From there, offering frequent, low-stakes quizzes will hopefully help your students slow down their forgetting curves and help them practice recalling/retrieving that information. By the way, that’s why stories are quite powerful. We often remember them better. So if you can weave an illustrative story into your next class, your students might really benefit from it come final test time!

Also relevant/see:

Ready, set, speak: 5 strong ways to start your next presentation — from abovethelaw.com by Olga Mack, with thanks to Mr. Otto Stockmeyer for this resource
No matter which of these five ways you decide to launch your presentation, ensure that you make it count, and make it memorable.

Excerpts:

  1. Tell a captivating story
  2. Ask thought-provoking questions to the audience
  3. State a shocking headline or statistic
  4. Use a powerful quote
  5. Use silence
    When delivering a speech, a pause of about three or even as many as 10 seconds will allow your audience to sit and quiet down. Because most people always expect the speaker to start immediately, this silence will thus catch the attention of the audience. They will be instinctively more interested in what you had to say, and why you took your time to say it. This time will also help you gather your nerves and prepare to speak.

 

 

 

Reflections on “Inside Amazon’s artificial intelligence flywheel” [Levy]

Inside Amazon’s artificial intelligence flywheel — from wired.com by Steven Levy
How deep learning came to power Alexa, Amazon Web Services, and nearly every other division of the company.

Excerpt (emphasis DSC):

Amazon loves to use the word flywheel to describe how various parts of its massive business work as a single perpetual motion machine. It now has a powerful AI flywheel, where machine-learning innovations in one part of the company fuel the efforts of other teams, who in turn can build products or offer services to affect other groups, or even the company at large. Offering its machine-learning platforms to outsiders as a paid service makes the effort itself profitable—and in certain cases scoops up yet more data to level up the technology even more.

It took a lot of six-pagers to transform Amazon from a deep-learning wannabe into a formidable power. The results of this transformation can be seen throughout the company—including in a recommendations system that now runs on a totally new machine-learning infrastructure. Amazon is smarter in suggesting what you should read next, what items you should add to your shopping list, and what movie you might want to watch tonight. And this year Thirumalai started a new job, heading Amazon search, where he intends to use deep learning in every aspect of the service.

“If you asked me seven or eight years ago how big a force Amazon was in AI, I would have said, ‘They aren’t,’” says Pedro Domingos, a top computer science professor at the University of Washington. “But they have really come on aggressively. Now they are becoming a force.”

Maybe the force.

 

 

From DSC:
When will we begin to see more mainstream recommendation engines for learning-based materials? With the demand for people to reinvent themselves, such a next generation learning platform can’t come soon enough!

  • Turning over control to learners to create/enhance their own web-based learner profiles; and allowing people to say who can access their learning profiles.
  • AI-based recommendation engines to help people identify curated, effective digital playlists for what they want to learn about.
  • Voice-driven interfaces.
  • Matching employees to employers.
  • Matching one’s learning preferences (not styles) with the content being presented as one piece of a personalized learning experience.
  • From cradle to grave. Lifelong learning.
  • Multimedia-based, interactive content.
  • Asynchronously and synchronously connecting with others learning about the same content.
  • Online-based tutoring/assistance; remote assistance.
  • Reinvent. Staying relevant. Surviving.
  • Competency-based learning.

 

The Living [Class] Room -- by Daniel Christian -- July 2012 -- a second device used in conjunction with a Smart/Connected TV

 

 

 

 

 

 

 

We’re about to embark on a period in American history where career reinvention will be critical, perhaps more so than it’s ever been before. In the next decade, as many as 50 million American workers—a third of the total—will need to change careers, according to McKinsey Global Institute. Automation, in the form of AI (artificial intelligence) and RPA (robotic process automation), is the primary driver. McKinsey observes: “There are few precedents in which societies have successfully retrained such large numbers of people.”

Bill Triant and Ryan Craig

 

 

 

Also relevant/see:

Online education’s expansion continues in higher ed with a focus on tech skills — from educationdive.com by James Paterson

Dive Brief:

  • Online learning continues to expand in higher ed with the addition of several online master’s degrees and a new for-profit college that offers a hybrid of vocational training and liberal arts curriculum online.
  • Inside Higher Ed reported the nonprofit learning provider edX is offering nine master’s degrees through five U.S. universities — the Georgia Institute of Technology, the University of Texas at Austin, Indiana University, Arizona State University and the University of California, San Diego. The programs include cybersecurity, data science, analytics, computer science and marketing, and they cost from around $10,000 to $22,000. Most offer stackable certificates, helping students who change their educational trajectory.
  • Former Harvard University Dean of Social Science Stephen Kosslyn, meanwhile, will open Foundry College in January. The for-profit, two-year program targets adult learners who want to upskill, and it includes training in soft skills such as critical thinking and problem solving. Students will pay about $1,000 per course, though the college is waiving tuition for its first cohort.

 

 

 

Why giving kids a roadmap to their brain can make learning easier — from edsurge.com by Megan Nellis

Excerpts:

Learning, Down to a Science
Metacognition. Neuroplasticity. Retrieval Practice. Amygdala. These aren’t the normal words you’d expect to hear in a 15-year-old rural South African’s vocabulary. Here, though, it’s common talk. And why shouldn’t it be? Over the years, we’ve found youth are innately hungry to learn about the inner workings of their mind—where, why and how learning, thinking and decision-making happens. So, we teach them cognitive science.

Over the next three years, we teach students about the software and hardware of the brain. From Carol Dweck’s online Brainology curriculum, they learn about growth mindset, memory and mnemonics, the neural infrastructure of the brain. They learn how stress impacts learning and about neuroplasticity—or how the brain learns. From David Eagleman and Dan Siegel, they learn about the changing landscape of the adolescent brain and how novelty, emotionality and peer relationships aid in learning.

Pulling from books such as Make It Stick and How We Learn, we pointedly teach students about the science behind retrieval practice, metacognition and other strategies. We expressly use them in our classes so students see and experience the direct impact, and we also dedicate a whole class in our program for students to practice applying these strategies toward their own academic learning from school.

 

 

 

Connecting Assessment and Learning — from scholarlyteacher.com by Spencer Benson

Excerpt:

Formative assessments provide information on students’ levels of understanding or performance. Formative assessments are designed to facilitate learning and improving teaching, curriculum/course design, and learning progress. They are typically low stakes and inform the teacher and student about current understanding and practices, primarily to improve understanding and performance. Summative assessments are used to evaluate student knowledge, understanding, or performance. Generally, they involve tests to determine grades and rankings.

Many formative assessments facilitate learning and motivate students: the ubiquitous; “are there any questions?”, 1-minute papers, pop quizzes, on-line quizzes, clicker questions, muddiest point, jeopardy games, etc. Formative assessments are frequently low stakes and often assigned a few points to encourage students to complete them. Formative assessment “sans” points are unfortunately often ignored or not taken seriously. How do we adapt summative assessment that captures the developmental aspects of formative assessments?

Below are four suggestions that have been shown to provide for deeper learning. Two approaches are focused on getting students to think about the content in useful ways and two are designed to help students perform better on traditional assessments.

 

 

NEW: The Top Tools for Learning 2018 [Jane Hart]

The Top Tools for Learning 2018 from the 12th Annual Digital Learning Tools Survey -- by Jane Hart

 

The above was from Jane’s posting 10 Trends for Digital Learning in 2018 — from modernworkplacelearning.com by Jane Hart

Excerpt:

[On 9/24/18],  I released the Top Tools for Learning 2018 , which I compiled from the results of the 12th Annual Digital Learning Tools Survey.

I have also categorised the tools into 30 different areas, and produced 3 sub-lists that provide some context to how the tools are being used:

  • Top 100 Tools for Personal & Professional Learning 2018 (PPL100): the digital tools used by individuals for their own self-improvement, learning and development – both inside and outside the workplace.
  • Top 100 Tools for Workplace Learning (WPL100): the digital tools used to design, deliver, enable and/or support learning in the workplace.
  • Top 100 Tools for Education (EDU100): the digital tools used by educators and students in schools, colleges, universities, adult education etc.

 

3 – Web courses are increasing in popularity.
Although Coursera is still the most popular web course platform, there are, in fact, now 12 web course platforms on the list. New additions this year include Udacity and Highbrow (the latter provides daily micro-lessons). It is clear that people like these platforms because they can chose what they want to study as well as how they want to study, ie. they can dip in and out if they want to and no-one is going to tell them off – which is unlike most corporate online courses which have a prescribed path through them and their use is heavily monitored.

 

 

5 – Learning at work is becoming personal and continuous.
The most significant feature of the list this year is the huge leap up the list that Degreed has made – up 86 places to 47th place – the biggest increase by any tool this year. Degreed is a lifelong learning platform and provides the opportunity for individuals to own their expertise and development through a continuous learning approach. And, interestingly, Degreed appears both on the PPL100 (at  30) and WPL100 (at 52). This suggests that some organisations are beginning to see the importance of personal, continuous learning at work. Indeed, another platform that underpins this, has also moved up the list significantly this year, too. Anders Pink is a smart curation platform available for both individuals and teams which delivers daily curated resources on specified topics. Non-traditional learning platforms are therefore coming to the forefront, as the next point further shows.

 

 

From DSC:
Perhaps some foreshadowing of the presence of a powerful, online-based, next generation learning platform…?

 

 

 

From DSC to teachers and professors:
Should these posters be in your classroom? The posters each have a different practice such as:

  • Spaced practice
  • Retrieval practice
  • Elaboration
  • Interleaving
  • Concrete examples
  • Dual coding

That said, I could see how all of that information could/would be overwhelming to some students and/or the more technical terms could bore them or fly over their heads. So perhaps you could boil down the information to feature excerpts from the top sections only that put the concepts into easier to digest words such as:

  • Practice bringing information to mind
  • Switch between ideas while you study
  • Combine words and visuals
  • Etc. 

 

Learn how to study using these practices

 

 

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

© 2018 | Daniel Christian