Evaluating Training Programs: Kirkpatrick's 4 Levels

The most widely used and popular model for the evaluation of training programs is known as "The Four Levels of Learning Evaluation." The model was defined in 1959 by Donald L. Kirkpatrick in a series of articles that appeared in the US Training and Development Journal. Kirkpatrick redefined the evaluation model with his 1998 book "Evaluating Training Programs: The Four Levels."

The idea behind the model is for an organization to have meaningful evaluation of learning in the organization. The degree of difficulty increases as you move through the levels. However, the knowledge learned regarding the effectiveness of the training program more than compensates for this.

The four levels of the model are:

• Level 1: Reaction
• Level 2: Learning
• Level 3: Behavior
• Level 4: Results

Level 1: Reaction

Kirkpatrick refers to Level 1 as a measure of customer satisfaction. Most of the forms that people fill out at the end of a class or workshop are instruments for measuring Level 1. Here are 8 guidelines that Kirkpatrick recommends to get maximum benefit from reaction sheets:

1. Determine what you want to find out
2. Design a form that will quantify reactions
3. Encourage written comments and suggestions
4. Get a 100 percent immediate response
5. Get honest responses
6. Develop acceptable standards
7. Measure reactions against standards and take the appropriate action
8. Communicate reactions as appropriate.

Level 2: Learning

Kirkpatrick defines learning as the extent to which participants change attitudes, increase knowledge, and/or increase skill as a result of attending a program. So to measure learning we need to determine the following:

• What knowledge was learned
• What skills were developed or improved
• What attitudes were changed

Here are guidelines for evaluating learning:

1. Use a control group if it is practical
2. Evaluate knowledge, skills, and/or attitudes both before and after the program. Use a paper and pencil test to measure knowledge and attitudes and use a performance test to measure skills.
3. Get a 100 percent response
4. Use the results of the evaluation to take appropriate action.

Level 3: Behavior

Level three can be defined as the extent to which a change in behavior has occurred because someone attended a training program. In order for change in behavior to occur, four conditions are necessary:

• The person must have a desire to change
• The person must know what to do and how to do it
• The person must work in the right climate
• The person must be rewarded for changing

Here are some guidelines for evaluating behavior:

1. Use a control group if that is practical
2. Allow time for a change in behavior to take place
3. Evaluate both before and after the program if that is practical
4. Survey and/or interview one or more of the following: trainees, their immediate supervisors, their subordinates and others who often observe their behavior.
5. Get a 100 percent response
6. Repeat the evaluation at appropriate times
7. Consider cost versus benefits

Level 4: Results

This involves measuring the final results that occurred because a person attended a training session. This can include increased production, improved work quality, reduced turnover, etc.

Level four can be difficult because you must determine what final results occurred because of attendance and participation in a training program.

You must also evaluate the conditions that the trainee is operating in. It is important to determine whether the conditions set forth above in Level 3 have been met. If there are conditions in the office that prevent the trainee from using the knowledge that they have learned, than the training can not be faulted for not "doing the job." The problem lies in the conditions that the employee is working in. How many times have you heard, "Oh, forget what you learned in training, that's not how we do it in this office." This is a clear example a a conditional problem with the operating environment.

Here are some guidelines for evaluating results:

1. Use a control group if it is practical
2. Allow time for results to be achieved
3. Measure both before and after the program if it is practical
4. Repeat the measurement at appropriate times
5. Consider cost versus benefit
6. Be satisfied with evidence if proof is not possible

Summary

This is a basic introduction to Kirkpatrick's Four Levels of Training Evaluation. More information is available through:

• Kirkpatrick's 1998 book "Evaluating Training Programs: The Four Levels."
• Chapter 9, " Evaluating Training Programs: The Four Levels" in the ASTD Handbook of Training Design and Delivery
• There are also a number of websites that offer information on the four levels.