Practitioners and thought leaders are working together to implement reporting standards in L&D and, ultimately, all facets of talent development. The inspiration for this work is the Generally Accepted Accounting Principles (GAAP) that have provided accountants with standard definitions and statements such as the income statement and balance sheet since 1939.
Like the accounting profession, L&D should have its own reporting principles, standards and defined measures to provide consistent guidance to practitioners and allow for meaningful comparisons among organizations.
Kent Barnett, CEO and founder of KnowledgeAdvisors, and Tamar Elkeles, vice president of Qualcomm’s Corporate Learning Center, launched this effort in fall 2010. They established an executive council composed of leading practitioners and recruited the author of this article to help with the effort. Barnett and Elkeles also established an advisory group of thought leaders in the field, including: Jac Fitz-enz, CEO, Human Capital Source; Rob Brinkerhoff, professor emeritus, Western Michigan University; Jack Phillips, chairman, ROI Institute; Josh Bersin, CEO and president, Bersin & Associates; Frank Anderson, president, Strategic Public Sector Solutions LLC; and Laurie Bassi, CEO of McBassi & Co. ASTD and several accounting firms have endorsed the multi-phase work.
Phase 1 focused on reporting for L&D in the private sector and was completed in March 2011 with the endorsement of the executive council and advisory board. Phase 2 expanded the scope to the government sector, refined the principles and reports from Phase 1 and defined more than 100 measures. It was completed in March 2012. Phase 3 focused on all talent processes — not just L&D — and devised an approach to use in Phase 4 to extend the work of Phase 2 to other talent development processes such as leadership development, talent acquisition and performance management. Phase 3 was completed in April with the launch of Phase 4, which is projected to be completed by December.
Phase 1 and 2 recommendations, created by a small working group under the guidance of the executive council and advisory board, include the following:
• Adopt a set of eight guiding principles.
• Adopt standard measures and definitions.
• Adopt three foundational statements.
• Adopt three executive reports.
The three foundational statements will be the learning industry’s equivalent of the income statement, balance sheet and cash flow statement. The recommended statements for L&D are the business outcomes statement, the effectiveness statement and the efficiency statement. Similar to the three accounting statements, each L&D statement has a standard, defined format, while allowing flexibility in the elements or measures depending on the industry and organization circumstances. Further, the formal, written definitions of L&D measures should drive consistency in use and calculation by all organizations. This consistency in reporting and definitions will not only provide much-needed guidance but also will enable more robust benchmarking across companies.
The three executive reports are highly customizable — each organization chooses which measures to include — and are intended to aid the learning function’s efforts to deliver planned results. The L&D Summary Report provides a high-level view of the learning function’s performance for the CEO and other senior leaders. The other two reports are the L&D Program Report and the L&D Operations Report. The CLO would use these reports to manage L&D programs and overall efficiency of the function.
Figure 1 shows how the measures feed into the three statements, which in turn provide data for the three management reports.
The Three Statements
The Business Outcomes, Effectiveness and Efficiency statements organize and standardize all the important L&D measures like the income statement, balance sheet and cash flow statement do for accounting measures. The statements provide information about the basic measures needed to manage the learning function. Each high-level statement has the same columns: last year’s results, this year’s plan, year-to-date (YTD) results, and YTD results as a percent of plan. The statements may have additional columns such as monthly plan and results, but these are the minimum. These statements should be produced at least quarterly and, in most cases, monthly. Detailed statements may contain higher-frequency data and greater granularity and may omit the plan.
The Business Outcomes Statement (Figure 2) reports the key goals or desired results for the organization and the expected impact of L&D on achieving those goals. The report should include the primary goals of the organization for the year — for example, a 10 percent increase in sales — and any other goals that will be supported by learning. In addition to sales, goals might include the following: financial — profit and cost reduction; operational — quality, productivity or innovation; customer — market share, satisfaction; employee — retention, engagement.
The goals should be presented in descending order of priority. The impact or importance of learning on achieving the goal may be a Kirkpatrick/Phillips level 3, 4 or 5 quantitative measure, a number, or may be expressed qualitatively: high, medium, low. The data should represent primarily the stakeholder’s view. For example, the vice president of sales believes the proposed learning program, properly conceived, designed, implemented and reinforced, should contribute 30 percent of the 10 percent increase in sales, or 3 percent higher sales. If level 4 or 5 measures are not available or not appropriate, a level 3 application rate or other measure may be used as a proxy or indicator for impact.
The Effectiveness Statement contains more familiar measures and focuses on how well or how effectively the business outcomes are achieved through learning. Recommended measures include level 1-5 indicators. All organizations should, at a minimum, report level 1 — satisfaction and a forecast of application and perhaps impact — and 2 — learning — measures. Level 3 — application — is strongly recommended for strategic, costly or visible programs aligned to organizational goals. Levels 4 — impact — and 5 — net dollar benefits or ROI — are recommended for key programs.
The Efficiency Statement reports the activity measures and cost elements that demonstrate how efficiently the outcomes were achieved. Efficiency measures include Level 0 or volume — number of participants, courses and hours; utilization rates — courses, instructors, facilities; budget and opportunity costs; program and vendor management; cycle times; reach; and other key indicators.
The Three Reports
The three recommended reports — L&D Summary, L&D Program and L&D Operations — provide managers and decision makers with information to manage and oversee the L&D function. Each high-level report answers the same questions: how are we doing versus plan, and how is the year going to end?
The reports pull key measures from the statements to help manage the L&D function. These are different for each organization. Consequently, they will be highly customized to reflect the measures most important for a particular function. Although the chosen measures will differ by organization, each high-level report should show the same columns or information: last year’s results, this year’s plan, YTD results, YTD results as a percent of plan, and a forecast for the year. These are the same columns as in a statement with the addition of a forecast column.
The reports may have additional columns — for example, monthly results and monthly plan — but these are the minimum. These reports should be produced at least quarterly and, in most cases, monthly. They should be one to two pages long and interpreted for the audience either in a face-to-face presentation or in writing with a focus on summary conclusions and recommendations. Detailed reports may contain higher frequency data, greater granularity and may not include the plan and forecast.
The L&D Summary Report includes key measures from all three statements (Figure 3). Its primary audiences are business executives such as the CEO and CLO, and it will contain higher-level, aggregated information about the most important initiatives and measures. Typically, the L&D Summary Report will include key business outcome measures including the impact of learning, key effectiveness measures such as feedback, learning and application, and key efficiency measures such as number of unique and total participants, number of courses and hours, reach, utilization rates, cycle times, budget and opportunity costs, and cost reduction.
The L&D Program or Initiative Report’s target audience is learning executives who manage programs, people and budget, including the CLO, directors and program managers. The report enables learning executives to manage programs and initiatives on a monthly basis to deliver the planned results. Consequently, this report will be more detailed and focused than the L&D Summary Report. Typically, it will include the organization goal, learning’s impact on that goal, number of participants, number of courses, completion dates for development and delivery, and levels 1 to 3, including the sample size.
The L&D Operations Report’s target audience is also learning executives. Its purpose is to manage L&D operations efficiently. This report will contain the most important measures from the efficiency report, focusing on key efficiency measures that will be actively managed on a monthly basis.
The Talent Development Reporting Principles (TDRp) will bring much-needed direction and standardization to the L&D field and to talent development in general. Many practitioners today do not know what data to gather, how to organize it, how measures are defined or how to use the measures to create reports for executives. TDRp answers all these questions.
Further, creating the Business Outcomes Statement and the Summary Report will require the learning leader to work more closely with senior executives to better understand the organization’s goals and talent development’s impact on achieving those goals. These discussions will benefit both the learning leader and the executives and will position learning and development to have an even greater impact on the organization’s success.
The effort is still in its infancy, and there is a long way to go to achieve widespread implementation, but TDRp is gaining traction and word is spreading. It is already the evolving standard for L&D.
David Vance is the founder of Manage Learning LLC, a strategic learning consulting company, and author of The Business of Learning. He can be reached at editor@CLOmedia.com.
