It is important to know whether the training that you have given to a team has been effective or not.
Is your audience getting enough out of your training?
And more importantly, does your training have a positive impact on their role and on the organisation?
These are the kinds of questions that are well answered with the help of Kirkpatrick’s 4-Level Training Evaluation Model. That could be a huge sigh of relief for you because for most people, calculating the effectiveness and ROI of training is often a daunting task.
Background to Kirkpatrick’s 4-Level Model of Training Evaluation
For the first time, Donald Kirkpatrick got his model published in 1959, during the time he was a professor at University of Wisconsin. The first update to that came in 1975 and then in 1993 which included his most renowned contribution, “Evaluating Training Programs”.
The interesting thing about the Kirkpatrick Model is every level offers a more accurate measure of effectiveness of training. Initially, Donald with his son James prepared the model. Later on, it was developed by Donald and Wendy Kirkpatrick, his wife.
It wasn’t until 2016 when James and his mother Wendy reviewed and updated the actual theory by introducing “New World Kirkpatrick Model” as an update. The book is called Four Levels of Training Evaluation. One notable change is the focus on significance of making the training applicable to everyone’s day jobs nowadays.
Let’s take a detailed look at all the four levels namely: Reaction, Learning, Behaviour and Results.
Reaction (Level 1):
What’s the number one goal of your training? Most probably it’s that you want them to know that the training is useful. On a simpler level, there are three metrics that help you decide how well they received it:
• How engaged the audience was during your training?
• How much contribution did they put in and how actively did they do that?
• How was their overall reaction to your training?
Not only you can get feedback on the training, it would also help you greatly improve your next training. This is because you can easily identify significant topics that you might have missed by any chance. There are a lot of questions that you can ask from the audience. Such as:
• Do you think the training was successful?
• Would you recommend this training to others in your unit?
• If anything, what would you change about this class, and why?
• In your opinions, what are the strong and weak points of this class?
• Did the training activities/exercises engage you?
• What could be the 3 most valuable stuff that you learned by attending this training?
• How good/bad were the venue?
• How god/bad was the presentation style, in your opinion?
• From this training program, how much do you think you can apply at your job?
• How much and what kind of support might you need when applying the learned knowledge at your job?
Learning (Level 2):
This is a level that emphasizes what the trainees have or have not learned from your training. There is one new and innovative addition in the New World version. It also measures how the trainees think they can do things differently, how confidently they feel in that aspect and what the source of their motivation is. As a result, you can assess how the training has imparted skills and knowledge to them, their self-confidence and determination.
The first step in line is to identify what you want to evaluate. This will streamline the process of measuring how much the trainees have learned. Ensure that there are enough learning objectives in your training, that the they are defined in a measurable way.
Measuring learning effectively
Of course, there are a number of ways to measure learning and it would depend on the objectives. However, as a rule of thumb, it is good to evaluate these areas before and after the training. A good way to do that is to have a conversation or quick question answer session with your trainees. This would help you assess their knowledge level, attitudes and skill level as well.
Behaviour (Level 3):
This is a really important level which helps you comprehend how well people make of their training. In other words, how much they apply the training they have received. Another benefit of this level is that it explores which parts the trainees may need assistance. However, for the behaviour to change, the conditions have to be favourable.
Let’s elaborate this part in more detail. Imagine a classroom where you are giving a training to your colleagues. You would assess them afterwards, right? In one scenario, it is quite possible that they show little change, helping you reach a conclusion that the people learned almost nothing. This renders your training ineffective.
Behavioural Change
In another scenario, it is also likely that the colleagues of yours learned a lot. It might be the case that they learned but the company or organizational culture hinders behavioural change. With the help of existing processes, you may assess there is little to no room to apply new knowledge. What happens as a result? People don’t feel comfortable when they are in a position to use the new knowledge or see few chances to do so. In some cases, people may not even find sufficient time to put the new skills to work.
Required Drivers
If you are facing such negative factors and developmental blocks, Kirkpatrick model presents a solution. Focus on developing processes that boost, encourage and reward positive behavioural changes. Such processes, as per New Kirkpatrick Model, are known as “required drivers”. Simply put, if a colleague or trainee uses the newly learned skill effectively, highlight them and appreciate them for it.
• It is to be noted that efficiently measuring behaviour is a long process that demands patience. Our recommendation is to do it three months after the training. You want to do it as early as possible but not too early sine the trainee’s need to have had the opportunity to apply their learning as well.
At Smart Consult & Research, we believe that conducting observations and interviews are two of the most effective ways to measure behaviour.
However, since they are quite resource demanding only about 15% of those we speak with are doing it. Using a digital tool that automates the evaluation can reduce the workload and release the resources to work with e.g. continuous improvement of the training portfolio
Results (Level 4):
This part is usually regarded as the primary goal of the program. At this level, you determine the overall success of the training based on a number of critical business objectives. These include decreased spending, greater ROI (return on investments), better quality of products and more work efficiency in employees etc.
It is important to note that the level 4 is definitely the most time consuming and costly process. One of the trickiest challenges is to predict which advantages, outcomes and results are associated with the training. Then you also have to think of an effective way to evaluate the outcomes in the longer run.
Possible Risks of Kirkpatrick’s Model
This model has remained very popular since its arrival and is now over 60 years old. However, there has been several updates to it and the way people learn and organizations work has changed the way it works. This is the primary reason why it should be used with care, and applied by experienced researchers – contact Smart Consult & Research for support. For example, even the term “learning” has been changed to “learning and development”.
Tackling Real-World challenges of evaluating learning success with Kirkpatrick’s Model
There are a few challenges of evaluating training effectiveness using Kirkpatrick’s model, a service which Smart Consult specialises in. Let’s take a look at the two main ones:
First Challenge:
As you know the four levels of Kirkpatrick model: Reaction, Learning, Behaviour and Results. Just as you start applying the model to real-life scenarios and problems, you will observe that not all levels of evaluation are always required. Just as an example, sexual harassment training is almost always mandatory. This is mandatory regardless of the ROI, which is why you might want to disregard that level completely. In other scenarios, compliance training is also necessary in order to know that the trainees retain the information and also apply at work. In such scenarios, you should definitely focus on level 2 and 3.
Second Challenge:
The second challenge in today’s world is called “Choosing the right metrics” as required in entry level of this model. For example, emotional response can be measured by the “reaction”. Similarly, the level of information retention may be measured by the “learning level” and the other two levels measure how the new knowledge is transferred to everyday job and measuring ROI respectively. However, there may be hundreds of metrics to use among these 4 levels. It can be a difficult task to choose the right metrics and get the right result. The below tips can help you choose the right metrics and ultimately get deeper insights:
• Asking the stakeholders about the metric is always a good idea. Although, the metric should drive business decisions and also determine the value of program for those who funded them.
• Check the relevancy of the metric and see if the performance indicators are inside the scope of the project.
• Carefully review how the metric can be connected to the key performance indicators (KPI).
• Always ensure that there is enough data available to draw conclusions. Data volume is important for data integrity.
Needless to say, the more data you have for a metric, more likely you are to get accurate insights.
The key point, at the end, is to bring your training process out of the theory and into the reality of what data and evaluation offers. This would be of great help to your learning & development project. Also, you can make strong conclusions to assist the business decisions and of course, develop effective training programs.
Get in Touch to Commission our Services Using any of the Following Frameworks:
- NHS Health and Care Evaluation Services (HaCE) – NHS Leeds CCG
- Lived Experience Service – NHS England and NHS Improvement (NHSE&I)
- Health Systems Support Framework – NHS England and NHS Improvement (NHSE&I)
- GP services and GP Caretaking services – NHS England and NHS Improvement (NHSE&I)
- Research Marketplace DPS – Crown Commercial Service RM6018
- G-Cloud – Crown Commercial Service RM1557.12
- Data SaSiE – Office for National Statistics
Reference Links:
Kirkpatrick Partners. (2021). The Kirkpatrick Model. https://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model
Mind Tools. (n.d.). Kirkpatrick’s Four-Level Training Evaluation Model. https://www.mindtools.com/pages/article/kirkpatrick.htm
- Learning Accelerators. (2020). Kirkpatrick’s Four-Level Model: What Is It and How Can You Use It? https://www.learningaccelerators.com/blog/kirkpatricks-four-level-model-what-is-it-and-how-can-you-use-it
