"So how do you evaluate the success of eLearning that you create?"
As a learning solutions vendor, I’ve been asked this question countless times and have also encountered it in many an RFP.
Proving the effectiveness of training and showing ROI is no walk in the park and still keeps L&D up at night.
Kirkpatrick’s evaluation model has long been the holy grail of training effectiveness measurement to businesses. My response to clients has typically been: If you are referring to Kirkpatrick Levels 1 & 2 it’s not very difficult; ‘smiley sheets’ will tell us about Reaction, and ‘assessments’ can help determine Learning. However, Levels 3 & 4 are not so easy to implement. Arriving at meaningful measures of behavioral change and demonstrating tangible results needs elimination of other influences on individual & business performance. Not to forget it takes months, if not years, to arrive at such conclusions with any certainty. Consequently, while many of our clients do ask, not many venture into it; and requiring organizational commitment and action as it does, it’s not something only we, as vendors, can entirely influence.
It is with this background that I found Professor Robert O. Brinkerhoff’s (who I had an opportunity to listen to at the AITD conference in Melbourne in April 2013) arguments very compelling. The key message from his talk was to Stop Evaluating Training the way we currently tend to.
Rather, he advises organizations to align training with business goals and evaluate all elements that go into achieving those goals with a focus on improving the impact.
According to Professor Brinkerhoff, an alarming 57% of firms failed to effectively implement strategic initiatives over the last 3 years. He also says that research suggests that less than 20% of trainees actually apply their learning in their work in a way that produces concrete and worthwhile results aligned with business strategy. The rest is ‘unrealized value’ of training.
Why is this happening? Why are we unable to use training effectively to achieve organizational goals? Here are some reasons that training commonly fails.
(From Professor Brinkerhoff’s Presentation)
Do any of these apply to your organization? If you find most of your reasons coming from the left or the right column, you’re not alone. Both these list crucial elements in helping training achieve its real goals.
So here’s what Professor Brinkerhoff recommends to improve the effective use of training in our workplaces.
-
Clearly communicate strategic goals and associated linkages throughout the organization. Each individual should know how their task fits into the bigger picture.
-
Don’t just offload training responsibility and accountability onto L&D. Look at training as a business driver rather than a staff benefit, and L&D as a consultant who truly wants to help you improve your department’s performance.
-
Provide training that addresses the knowledge and skills required for the adoption of strategic initiatives and the execution of strategic change.
-
Look at learning events differently. One of the biggest problems is being stuck in a mythical view where ‘training’ is seen as completely encompassed in learning events, based on which returns are expected.
(From Professor Brinkerhoff’s Presentation)
But in reality, training comprises not just the learning events themselves, but also what comes before and after them.
(From Professor Brinkerhoff’s Presentation)
-
Before the learning event: A dialog between managers and trainees to establish an action plan for using the training.
-
After the learning event: Evaluation of training effectiveness through the Success Case Evaluation Method® in which training use and non-use cases are analyzed.
The diagram below summarizes Professor Brinkerhoff’s argument:
You can read more about The Advantage WaySM Process and The Success Case Evaluation Method here.
And here’s Professor Brinkerhoff speaking at the Learning Technologies Show in London in 2013. It’s a 60-minute video but well worth your time.
The conclusion? If we continue to evaluate ONLY training the way we normally do, we’re not getting anywhere.
