SHOWING THE VALUE OF LEARNING

From the desk of Marcia Dresner...

Measurement matters.

Learning organizations that do measurement well have the advantage when it comes to being a part of the strategic planning process for the business. When L&D can show the value of its efforts, it is easy to gain support for new program development and to avoid being the “first place companies cut” in a downturn. There is no doubt that a well-designed measurement process is essential for the success of any learning organization. CLOs have to respond to the increased pressure to describe the results of programs and practices in ways that are respected and understood by stakeholders at all levels of the company.

Every other month a new product is launched at Victoria’s Secret, and a big challenge is how to go to market with those launches. The company focuses on conversion and how to convert customers, using that as the metric to success. In an exclusive CorpU TV interview, Marla Zarlenga, Director of Brand Education and Education and Development for Victoria's Secret notes that "if your focus is not on evaluation, you need to understand how to get there."

Marla Zarlenga, Victoria's Secret

One of the most dramatic statistics to come out of the CorpU 8th Annual Benchmarking study was that 58% percent of respondents were feeling increased pressure to show the value of learning. Equally dramatic, however, was data that showed that executives and business leaders didn’t always believe the measurements when they were presented, and that L&D organizations often relied on anecdotal evidence of their effectiveness.

Source: Corporate University Xchange 8th Annual Benchmarking Study

There are, however, some encouraging signs when it comes to how companies are handling this critical aspect of their learning organization. Many L&D groups are engaging business leaders early in their program development process and asking for agreement about “what success will look like”. There are others that, though they continue to collect Level 1 data, are collecting more credible data about how learning has affected critical business metrics. The work of the CorpU award winners described below show some of what is being done.

Measurement in the CorpU Awards Program

The Corporate University Xchange 10th Annual Awards for Excellence and Innovation in Corporate Learning (the CorpU Awards) has a separate measurement category. However, it’s not the only place the awards program asks about measurement. In fact, the CorpU Awards application process requires those companies vying in every category to describe the results of their efforts. It has become clear over the years that the judges want companies to go above and beyond tactical measures like classes taught. They want measurements that reflect how the learning described has affected the business outcome it was designed to achieve. 

For the Measurement category, the corporate learning unit must create tools and techniques that measure the value of an organization’s investment in learning. It has to be able to explain the measurement strategy the corporate learning unit uses to assess its impact on the organization. It must also explain how it uses measurement data and feedback to drive continuous improvement within the organization.

Successful organizations will demonstrate:

• Performance measurement strategies that link learning to the goals of the organization
• Metrics, systems, and tools used to demonstrate learning's impact on the workforce
• Strategies that involve business leaders in determining metrics by which learning success will be determined
• Ways in which the corporate learning unit uses return on learning investment analyses, balanced scorecard, learning-related intellectual/human capital measures, or other innovative approaches
• Evidence of improved performance within the organization based on quantitative and/or qualitative trend data

The Winner’s Stories

The winners of the CorpU Awards in Measurement created metrics that were recognized and respected within their organizations (something that, according to the 8th Annual Benchmarking study data, doesn’t always happen). Here are the tools and results demonstrated by three recent Award Winners, Cerner, Staples, and a healthcare insurer.

Cerner

Cerner, the leading U.S. supplier of healthcare information technology solutions, depends on the ability of their solution delivery consultants (SDCs) to implement their end-to-end solutions in healthcare settings. At the time of their award presentation, the company had 5800 installations of their flagship product serving 160,000 physicians. 

KnowledgeWorks, Cerner’s learning organization took on a challenge which they described this way:

"It takes 12 months for an entry-level associate to be certified as a solution-delivery consultant (SDC). They are billable at six months. Reduce the time-to-competency by 50% and produce billable resources in three months that are as competent as the current output of the certification program."

Their new program, Velocity Takeoff, included a 3-month engagement in an end-to-end, hands-on simulation with just-in-time learning components. This was followed by were 3-months in “finishing schools” that were billable assignments under mentors to develop consulting and client engagement skills. 

And then they measured. Looking at all four Kirkpatrick levels, KnowledgeWorks was able to show that (1) managers felt that new hires were more skilled at 6 months than the previous hires were at a year, and (2) clients thought graduates were system architects because of their breadth of knowledge. The time to competency was halved and the new program contributed twice as much to the revenue stream as it cost.

New Program
Total Revenue	$12,616,800
Total Cost	$6,468,240
Contribution Margin	$6,148,560

Staples

Staples University described three different measurement activities to demonstrate the different measurement methods used throughout their various divisions. The retail division measured the top line sales growth of a featured product after training, and a control group was used to isolate the effects of training. [We have no idea if that’s what they are currently doing and have to use past tense!!]

The North American Delivery Business measured the business advantage that was created through all of their training, from new hires to leaders. The company has a significant advantage over their other competitors and learning is considered to be a major contributor to that advantage.

The third example they gave was the result of process excellence. Their Six Sigma efforts required Staples University to develop and deploy a large rotating pool of Black Belts that aligned with corporate goals of best execution and increasing effectiveness and reducing costs. The projects performed by the various practitioner levels contributed significantly to the company’s bottom line results.

Healthcare Insurer

When one of the largest healthcare insurers in the United States centralized its functional training units into a single learning organization, it was imperative for the learning organization to have the ability to quantify its contributions to business performance.

The  new learning organization translated the organization’s mission to tangible training objectives and measures with a Learning Balanced Scorecard (BSC). The BSC looks at training through four perspectives:

1. Financial measures
2. Internal processes
3. Customer satisfaction
4. Business effectiveness

Teams assigned to each perspective determined formulas for calculating metrics, selected tools for capturing measurement data, and defined deliverables appropriate to measuring performance improvement. 

The financial perspective captured metrics such as training costs per employee, total training costs as a percent of payroll, cost comparison of in-house versus external vendors for course development, and cost savings on distance learning and web-based learning programs. The process perspective captured data on process efficiencies, vendor management and staff competencies and to measure customer satisfaction, the team looked at both participant and manager satisfaction with learning outcomes.

In business effectiveness, the learning team monitored effectiveness of performance interventions and client management skills. In one example,the Balanced Scorecard approach – enabled the learning organization to save the company more than $1.2 million in training development and delivery costs to address new regulatory requirements around patient privacy. Additionally, by moving training in frequently used technology from the classroom to on-line in the actual work environment, the learning organization saves the company another $600,000 annually and delivers training on the job where people need it most.

"What Gets Measured, Gets Done”

In that simple aphorism, and its more negative corollary “what doesn’t get measured, doesn’t get done”, is the key to what L&D organizations must do. They have to be very careful to define what the outcomes are supposed to be of any program or course, and make sure that those measurements are possible and realistic. In all of the examples above, the learning organization demonstrated how their programs made a significant difference in business results. They used the metrics that most mattered to the business, and reported results that were relevant and meaningful to stakeholders. Many of you are doing the same, and we invite you to apply for a CorpU award so we can share your success with the learning community.

 

Marcia Dresner, CorpU Senior Research Analyst

CorpU Members can download these additional measurement-related documents:
Measuring Business Impact
The Success Case Study Method for Measuring ROI