Has something like this ever happened to you? Your learning team creates a dynamite training program for newly hired sales representatives. The program is instructionally sound, interactive, engaging and highly effective in terms of meeting learning objectives. Learner reviews are off the chart. Analysis of a key business indicator – say, average size of sales deal for those who completed the program – shows that the training had a measurable business effect. The vice president of sales requests a meeting with you – to give you a well-deserved pat on the back, you think. Instead, you find out he or she is unhappy with the program. What went wrong?
The above is a classic example of what happens when a learning organization is out of sync with its customer. Although “business alignment” is one of the most commonly used phrases in the learning profession today, too often learning organizations fall short of meeting this goal.
We can all cite a long list of obstacles that stand in the way of business alignment: organizational structures, time constraints, limited budgets and resources, uncooperative managers and supervisors, lack of executive support, etc. But the problem is actually much more basic. The models, processes and measurements that have been institutionalized fail to take business impact and alignment into account from a program’s inception and throughout its life cycle. Consequently, we are left scrambling to justify a program’s value.
In order to truly stay in sync with your customer’s – and your organization’s – needs, you must take a holistic approach that goes beyond the traditional models known by heart. The market needs an approach that views learning as a business-centric, service delivery organization and takes into consideration the influencers and different types of metrics available. A more up-to-date and business-oriented approach also should integrate training and performance measurements.
The Limitations of Kirkpatrick and Other Models
When the Kirkpatrick model was introduced, training organizations comprised primarily stand-up trainers whose jobs were completed at the end of a class. The training department usually operated as an island with limited interaction with business units. Even up through the 1990s, when “corporate universities” became popular, the training organization was viewed largely as a specialized unit within the enterprise. The focus was still formal training, although by then e-learning was becoming widely adopted.
In the last several years, the learning organization has become much more tightly intertwined with business and talent management. Informal approaches to learning – such as referenceware, collaboration, and coaching and mentoring – are bringing learning closer to an employee’s job and occurring over long periods of time, sometimes even years. Today, high-impact learning organizations emphasize performance consulting and even assign dedicated staff resources to lines of business in order to maintain a closer relationship to their internal customers.
Measurement models that focus primarily on the development and delivery of formal training do not reflect the current mission of corporate learning organizations. Commonly, they view measurement as a separate activity, and most do not provide guidance on ensuring ongoing business alignment. Additionally, many models, including the Kirkpatrick model, imply that measurements are hierarchical.
Today’s training environments call for a wide variety of measures, none more important than others. In some cases, one of the most important measures of success is time to market. In others it’s meeting a specified budget or a very specific behavior change. The key to alignment is in understanding the business priorities, determining how measurements truly relate to business impact, and then identifying and using the appropriate and practical metrics.
The Relationship Between Business Impact and Training
Successful business alignment requires that training processes, key influences on impact and measurements are correlated to the business throughout a program’s life cycle. It’s critical to take a holistic view as opposed to one in which these areas are managed and monitored separately.
By understanding how and where processes influence business, when various influencers come into play and the types of metrics that can be used at each phase, you can keep your finger on the pulse of the business at all times.
Problem Definition: The success of any training program hinges on the identification, quantification and definition of the business problem and then ensuring the problem is appropriately prioritized. This phase has two associated training processes: business alignment, which is working with the business to understand and prioritize the problem, and performance consulting, which is the process of unraveling the business problem into its root causes.
As an example, let’s consider a new product with lackluster sales. The learning program manager must make sure this is a recognized problem among sales management and warrants the time, money and effort to solve through training. As part of the business-alignment process, the program manager should quantify revenue shortcomings.
Concurrently, a learning team member is researching the problem’s root causes and developing learning objectives. Interviews might uncover that one cause of low sales is that sales representatives don’t fully understand the value of new features and do not know how to address common sales objections. This is a root cause that clearly can be addressed by a training solution.
Training Solution: Most training resources are focused on this phase. In fact, many learning organizations focus almost exclusively on the design, and too often, at the expense of the other just-as-important phases. Impact influencers in this phase include:
Here’s an example from a major accounting corporation. The company’s CLO introduced a program on project management. It garnered excellent reviews by learners – most had high scores on exit exams. To understand how well the program was affecting actual client work, the CLO launched a survey and series of interviews with attendees several months later. The findings were alarming. Although employees had spent weeks learning new project management skills, few were being reinforced or used in actual client projects. Why? Because managers and the overall business were not aligned in the practice and use of project management.
This example shows how individual performance can be high, while organizational performance is low. Once the problem was identified, the CLO focused on aligning the program with business processes to drive impact.
Using Measurements
Once you understand the interconnection between business and training, you can determine the best measures to assess business impact during each phase of a program. Measurements fall into nine categories:
Satisfaction: Satisfaction measures are an excellent way to measure the quality of the learning experience, how well the program was targeted and how well the program was designed. Satisfaction is a foundational measurement for any program.
Learning: Learning outcomes such as assessment scores are important in evaluating program design, audience fit and the overall learning experience. These measures, however, do not tell us anything about the individual performance or organizational impact. Because most corporate training is a function that uses teaching to improve business performance, training managers should be cautious about using test scores to measure impact. More meaningful learning measurement would be based on on-the-job assessments and measurements of actual job performance under widely changing conditions.
Adoption: While adoption often is viewed simply as “butts in seats,” it is actually an important measure that indicates how well a program was marketed and adopted, as well as how it was received by the audience. For instance, you might target 90 percent of the 350 directors and above in your company for a new leadership development program. If adoption rates fall below your goal, you immediately will know that you have a communications problem or an alignment problem. Adoption is also an important measure used in compliance programs and any mandatory training program.
Efficiency: Nearly 40 percent of organizations routinely measure the costs related to development and delivery of training. Efficiency measures are very actionable because they help you understand factors such as time to market, cost-effectiveness and resource utilization. You should consider factors such as the relationship between costs and program priorities, the value of technology investments and how a program’s cost-per-delivered-hour compares with other training programs within and outside of your organization.
Alignment: Essentially, alignment measures tell you how well programs correlate to critical business objectives and whether you’re investing most of your resources to support strategic rather than operational initiatives. Alignment measures should be among your standard evaluations before, during and after a learning event. Examples include process alignment, management alignment, job-role alignment, competency alignment, financial alignment, and urgency and time alignment.
Individual Performance: Individual performance can be tied directly to the root cause analysis identified through performance consulting. If a quantifiable business problem is identified upfront and documented as a root cause, you can then use appropriate metrics to assess individual performance before and after training. If your organization has a performance management process, data from performance reviews also can be correlated to training.
Organizational Performance: Ultimately, the goal of any training program is to improve workgroup or organizational performance. Organizational performance might or might not derive from individual performance. In many cases, you can measure organizational improvements directly from business metrics already being captured.
Utility: The usefulness of the program to learners can be measured through surveys and interviews. How useful was this course in helping you to improve the efficiency and effectiveness of your job? Will this course help you reduce errors or time spent on certain tasks? Would you recommend this course to others? While qualitative in nature, such questions can give you excellent quantitative results when answers are assigned numeric values.
Attainment of Client Objectives: How well a program meets your client’s objectives is one of the most important measures of all.
This framework is intended to be used for three purposes. First, it gives you a way of categorizing and prioritizing what you do want to measure. No organization can measure every area in detail. You should, however, be able to build a measurement program around at least one metric in each key area. Second, the framework should give you a way to simplify the measurement process. By understanding the concepts behind each area, you easily can implement assessments, evaluations or other measurement processes. Finally, and perhaps most important of all, this framework offers a way of continually assessing the business impact of training from the moment a problem is identified to months or even years after a program or other training solution is delivered.
Leighanne Levensaler directs Bersin & Associates’ research and advisory services in talent management. She can be reached at llevensaler@clomedia.com.
