From DSC:
I especially appreciated the following excerpt of the dialog between Donald Kirkpatrick (Kirkpatrick’s 4 Levels*) and Jack Phillips (another well-known evaluator of training).  Kirkpatrick proposes the following series of questions:

Kirkpatrick:
Many people see evaluation as something to be done after training. But the art is to arrange your training in such a way that success is guaranteed. We launch a series of questions to see whether we can build a chain of impact up front: How can we make sure people learn? How do we ensure people use their newly acquired skills on the job? How do we know that the application of these skills returns the desired effect?

Kirkpatrick:
That’s very simple. Let me give an example. At Intel, they said, “Let’s take your four levels, start with the last one, and work backwards. What results are we looking for? What behaviors are needed to accomplish those results? What knowledge, skills, and attitudes do people need in order to behave in that way? And how can we do it in such a way that they’ll react favorably?” Answering those questions in that order is at least one less headache.

Excerpt from:

Diederick Stoel.  (2004, January). The evaluation heavyweight match. T + D, 58(1), 46-48.  Retrieved February 18, 2010, from ProQuest Education Journals. (Document ID: 535013771).


* Per Wikipedia:
The four levels of Kirkpatrick’s evaluation model essentially measure:

  • Reaction of student – what they thought and felt about the training
  • Learning – the resulting increase in knowledge or capability
  • Behavior – extent of behavior and capability improvement and implementation/application
  • Results – the effects on the business or environment resulting from the trainee’s performance

I need to catch up to many of you out there…but while reading Kirkpatricks’ Evaluating training programs: The four levels (3rd ed.) last night, I had the following thoughts:

4 Levels:

  • Reaction <– Hypothesis from DSC: K-12 and college/university students’ initial reactions will become immediately more positive if technology is used in effective, engaging ways. Start with a format/media/technique/game/simulation that interests the students the most about that topic (even if this “hook” is not going to be the focus of what you will be covering the majority of the time). For as the Kirkpatricks state on p. 22, “Positive reaction may not ensure learning, but negative reaction almost certainly reduces the possibility of its occurring” (p. 22).
  • Learning
  • Behavior <– In order for change to occur, four conditions are necessary:
    • The person must have a desire to change
    • The person must know what to do and how to do it
    • The person must work in the right climate <– speaks to job families in business world
    • The person must be rewarded for changing <– speaks to incentive systems. How can we build some powerful incentive systems for students? Perhaps use an online system whereby if the student successfully completes the learning objectives, they get whatever they want up to $_____ off of their tuition, or perhaps an iTunes gift certificate for K-12ers. Perhaps Apple and company would be willing to donate ___ songs or the Gates Foundation put forth $___ as a reward system here. Or perhaps if they meet ___ objectives, they earn the right to take another elective within a discipline that they are passionate about.
  • Results

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2005). Evaluating training programs: The four levels (3rd ed.). San Francisco: Berrett-Koehler.

© 2024 | Daniel Christian