Josh Bersin is the principal and founder of Bersin & Associates, with more than 25 years of experience in corporate solutions, training and e-learning. He can be reached at editor@clomedia.com.
As part of our coverage of corporate learning, we conduct ongoing research on trends and best practices in training measurement. This column summarizes three top findings.
1. Training measurement continues to be a challenge.
In a recent survey, only 4 percent of respondents said they had a “world class” strategy in place, and 37 percent had no plan for measurement at all. These numbers have remained consistent for the past three years, showing that the issue of learning measurement continues to challenge most organizations. The good news is that 16 percent of respondents said they had a strong measurement program of some kind — a number that is slightly up from past years, indicating a small but positive trend toward better solutions.
When we look at what people measure, we find that almost 70 percent of the measurements being captured are related to satisfaction. Only 3 percent of respondents said their companies try to measure some form of business impact or ROI. This number hasn’t changed in the past four years.
Executives tell us that their challenges with training measurement are not due to lack of interest, but rather to the difficulty of gaining buy-in to a total measurement approach that can be used across a wide range of programs. For example, GE has dozens of training groups organized under several corporate L&D functions. Without a commonly agreed-upon measurement model, each department has to figure out a measurement approach on its own.
We advise companies to develop an enterprise model, which can start as a simple approach and evolve to become more sophisticated over time. The key is that all training functions adopt the model. An enterprise model saves time and energy and eliminates the need to reinvent the wheel, while also giving learning executives the ability to compare program to program across the enterprise.
2. Measurement tools are still evolving, forcing learning leaders to use a variety of methods to make measurement complete.
In our Impact Measurement Framework, published in 2006, we defined new measures to consider in any training measurement strategy. These include the measures of utility, or usefulness to an employee’s immediate job; alignment, or agreement among managers that the program was urgent and relevant; efficiency, or measurement of total cost for development and delivery; and customer satisfaction, or the raw satisfaction of the manager or executive who is the ultimate customer.
These are easy-to-understand, common-sense measures, but few systems can handle them. To further complicate the issue, many companies are looking to analyze both training- and talent-related measurements. While the benefit of knowing the value and ROI of a given training program is great, understanding how this program impacts skill levels, performance ratings, and technical and leadership readiness would be even better. Ultimately, training measurement should be part of the broader talent measurement strategy.
LMS vendors are stepping up support for measurements, and many now deliver powerful analytic solutions. But the implementation of these tools is usually left up to you.
3. Measurement initiatives must expand to cover informal learning.
As informal learning becomes an integral component of corporate learning strategies, new measurements are needed. Such metrics include activity and resource utilization, which are measured by page hits, clicks and downloads; contribution, which is the frequency of an individual’s contributions to informal learning resources and their value; and feedback, which is the number of comments and corrections to informal learning resources and user ratings.
Such metrics will help you answer a variety of questions, such as: Which documents and resources are most valuable? Which subject-matter experts are contributing the most and are most respected? What media is most useful to which audiences? Why are people finding and using certain things and not others? And what is our efficiency at delivering such materials?
Training measurement must continue to be a top priority for any learning organization. Measurement programs not only provide critical information for decision making, but also force learning executives to consider their entire process of content development and delivery in a holistic and business-oriented way. If you start off simple and grow over time, you will find that much of the sophisticated, advanced approaches to the measurement and management of training come naturally.
