by Kimo Kippen
In the Tour de France, cyclists roll like fastmoving, human-powered trains around corners, up and down hills, and through roadside villages. Cycling is anything but an individual sport. Teams practice precise rotations in and out of the lead to block the wind, before returning into team formation to conserve energy. These are carefully choreographed moves that tap the potential of each member’s strengths to sustain their collective pace through a series of individual sprints. But team members change, as does the course and weather. So each year, teams must synchronize anew, assign roles, and craft a strategy that reflects the potential of each of its nine riders.
It’s a system that stands in stark contrast to the prevailing approach to leadership development in the United States. While organizations and executives often believe in the potential of teams, they continue to invest in leadership development strategies rooted in the skill gaps of individual leaders.
Development programs are rarely organized around leadership teams with shared responsibility for specific business outcomes. So the “riders” never get the chance to practice as a team. And because development often addresses executive succession, not the current business strategy, it’s as if riders never see a map of the course ahead of them.
The challenge is rooted in a data deprived approach to measuring and cultivating leadership skills — a measurement approach in need of improvement. Organizations lack new methods to analyze the success of development strategies and data to help them understand the strengths and weaknesses of their existing leadership.
The limitations of analog leadership development strategies are having an impact. Research has shown that as little as eight percent of executives think that their company is effective in developing leaders. In a survey conducted by the Institute for Corporate Productivity, two-thirds of companies said they were failing at the task. And it’s not just that executives are cursed with a pessimistic view of their leadership teams. Only 15 percent of leaders who receive training go on to create change within a company, according to a study from learning effectiveness expert Dr. Robert Brinkerhoff.
But change is possible. When I oversaw the learning organization at Hilton, I helped transform the learning and development culture of the company. We invested in, and relied upon, data to inform our approach to identifying – and developing – leadership talent. We developed leadership by building the capacity of teams, as well as people. Hilton’s guest scores, brand loyalty, and foothold in the global market all improved as a result.
This paper makes the case for investing in teams, and building the capacity for leadership, through a new brand of analytics-driven development. Because it’s time for companies to give their leaders the tools they need to not just ride — but learn — as a team.
Kimo Kippen is an accomplished thought leader and sought-out international keynote speaker committed to inspiring business leaders to think outside the box. As Chief Learning Officer at Hilton Worldwide, he transformed the learning and development culture and improved the company’s brand loyalty. Prior to 2010 Kimo held various roles at Marriott International Inc., including leadership of the Learning Center of Excellence for all of Marriott International’s properties worldwide.
Acknowledging Our Greatest Fears
Leadership teams are the backbones of every company. Their attitudes and beliefs shape an organization’s soul. Their capacity to learn and take action propels the organization along in its competitive race.
But it is stunning how little executives know and can prove about how well they are developing their leaders’ collective capabilities, shaping their attitudes about the company’s future direction, or ensuring a breadth of diverse perspectives and experiences that can be brought to bear against stubborn problems.
Fundamental, critical questions have defied answers for decades:
- Do our leaders have the right skills?
- Are we getting future leaders ready fast enough?
- Are leaders aligned around a common vision and moving in a synchronized way toward critical objectives?
- How good are we at ensuring all the above?
If similarly fundamental, foundational questions were unknown about a company’s fiscal health, or its new business pipeline, it would be a breach of management’s fiduciary responsibility, and deep down, many know it.
As one Chief People Officer recently put it: “The question I really fear is- how do I know? How do I know how strong or weak my leadership pipeline is? How do I know I’m doing the right things to make it better? Can I prove it? The real answer is ‘no’ and it literally keeps me up at night sometimes.”
Measuring What Counts
The cause of learning executives’ fear and uncertainty stems from two related problems:
Most companies are not good at measuring the strength of their current leadership teams, and most struggle at measuring the effectiveness of programs designed to develop future leaders.
Measuring All Factors of Success
For a leadership team to succeed in implementing a new idea or initiative — to change and grow their organizations — leaders must do a number of things:
- They must engage with the idea—reading about it, analyzing it, reflecting on it, discussing it.
- They must understand it—deeply—and have the knowledge and skills to implement that understanding.
- They must believe in it, changing their mindsets and hearts and becoming advocates for change.
- They must see problems and preemptively identify their solutions as an integrated unit.
- And finally, they must act as an integrated unit implementing the game plan over time as one team.
Most companies do not measure the presence or absence of these conditions with respect to a critical idea or imperative (such as the company’s strategy, a significant business change, or reshaping organization culture). They simply do not.
Instead, they measure a host of other factors available in performance reviews, 360s, OKRs, etc. but, skip measuring the fundamental conditions for developing leadership teams. Performance measures look backwards at past performance and offer little confidence in leaders’ future success.
How do I know how strong or weak my leadership pipeline is? How do I know I’m doing the right things to make it better? Can I prove it? The real answer is ‘no’.”
Improving Proven Methods
Donald Kirkpatrick published the most widely used framework for measuring learning outcomes. Kirkpatrick’s genius was in adapting and applying the scientific method to the business of measuring corporate learning outcomes. His model advocates for measuring reaction, “did they find the training valuable?”, learning, “did they acquire new knowledge or skills?”, behavior “did they apply the new knowledge or skills at work?”, and results “did the new skills help them achieve a favorable result?”
This model has worked well for many decades because it is coherent, concise, and proven. Jack Phillips, another learning measurement expert, added a fifth level, building on Kirkpatrick’s prior 4, which was focused on investment payback and asked, “what financial results can be demonstrated as a result of the training investment?”
The difficulties in implementing all four or five levels of measurement are monumental. They rely heavily on surveying an already over-surveyed population of workers and incur heavy costs to administer, analyze and summarize into reports for broader consumption and decision making. As a result, and as our research indicates, the majority of companies do not measure learning outcomes past Kirkpatrick Level I.
Since measurement is difficult and expensive, organizations just don’t do it very much. And as a result, at least as it applies to leadership development, nobody seems to know what the value of the investments are. Nearly every study shows that CEO’s believe that “lack of leadership bench strength” is one of the top 5 issues that keep them up at night.
With new advances in Artificial Intelligence (AI) come new ways to implement the Kirkpatrick Model of learning evaluation. By using machine learning algorithms, trained to watch each and every interaction made by participants in leadership development programs, we can now generate real-time Level I-IV dashboards that provide a holistic view into progress. Even better, we surface early warning signs that allow expert-guides and coaches to immediately make mid-course corrections. Consider this story:
One early adopter of AI-Driven Leadership Measures found that 1-week into a major transformation, the majority of their workforce was not ready for the change. Workers hadn’t been appropriately educated on the purpose for the change and the key reasons why it was so critical to the organizations future. As a result, the President stopped the online course mid-stream, huddled with his team and recorded new videos that better described the purpose and vision for the transformation, and then relaunched the transformation to 1,000’s of people. This early adopter indicated that the AI approach likely saved them 7 months of rolling out the initiative the old way, via workshops and town hall meetings where they simply have no ability to deeply measure understanding, buy-in, and commitment.
Corp/U Analytics measures all of Kirkpatrick’s 4-Levels by watching what people do, how they engage, and what they have to say, without having to send them a bunch of annoying surveys, many of which would never be filled out anyway. We humbly stand on the shoulders of Donald Kirkpatrick, and offer a new-to-world way to measure the impact of Leadership Development investments.
Reaching for Peace of Mind
CORP/U Analytics represent a step change in capability to measure two things:
- How strong are our leadership teams — defined as the extent to which they are prepared to implement a common vision?
- How effective are we at making our leadership teams better?
CORP/U Analytics presents real-time feedback to drive immediate response and improved effusiveness of learning initiatives
A Global Life Sciences company made quick, mid-course corrections to immediately improve sales skills as business conditions changed.
CORP/U Analytics measures seven precise metrics that assess whether a leadership team is prepared to succeed
A retail company used benchmarks to discover that program participants who achieved higher “Understanding” scores, also achieved higher sales revenue goals. A hightech company illustrated that participants with higher “Commitment to Action” scores stayed with the company longer.
CORP/U Analytics provides actionable insights to strengthen and diversify the leadership bench
A global retailer discovered a new pool of future leaders through CORP/U Analytics’ determination of the company’s “network performers.” Network performers show evidence of making themselves better while also making others better because they ask great questions, provide good answers to questions, and add value by educating and informing peers. Network performers also stand out as future Change Leaders and Initiative Champions.
How CORP/U Analytics Works
CORP/U Analytics is the “AI engine” within CORP/U’s Leadership Development Platform. It uses AI to deliver real-time analytics — an insight-rich view into each step –from business need to business result.
Each metric offers an answer to a question plaguing business executives about the strength of organization leaders, as well as their ability to build more.
The most basic form of learning measurement is completion. Decisions to renew contracts for eLearning often come down to completion rates. However, a completion score doesn’t distinguish those who might have truly engaged with an idea from those who hurriedly click through pages just to get it done.
The CORP/U Analytics’ Engagement Metric goes beyond a completion rate for each single course or course activity (such as completing a lesson, module, or watching a video). It validates true idea engagement by combining activity completion scores with an evaluation of the quantity and quality of leaders’ discussion posts with their colleagues about the ideas at hand.
By measuring and tabulating how participants post and respond in discussion- how they engage with the idea- CORP/U Analytics generates a much-improved measure of how leaders engaged with each content element, and the larger idea at hand.
Traditional Kirkpatrick Level 1 evaluations judge leaders’ opinions about the value of content in a learning program. These assessments come in the form of simple surveys. But as mentioned previously, surveys introduce their own challenges, and sometimes, biases.
To create effective leadership teams, we need to know not whether they ‘liked’ a given course, but whether the intervention changed hearts and minds of the participants. Do they believe in the ideas at hand and will they act on that information?
CORP/U’s Advocacy Metric measures whether leaders become true believers that there is a new and better way do their work. Are they willing to set aside past practices so they can apply and master alternative approaches that they truly believe in? Or not?
Cognitive knowledge—new knowledge, memory, judgment, evaluation and reasoning—comes in varying degrees. Memorization is all a leader needs to meet the standards for knowledge retention assessments. Behavior change and application require something much more.
If leaders are to find novel ways to apply principles within a business context, they need deep understanding. Deep learning occurs only if they think critically and can apply the newly gained insights in different contexts.
CORP/U’s Understanding Metric investigates leaders’ ability to explain and discuss, in a meaningful and relevant way, the ideas, concepts and practices presented in the course material. Check-ins on leaders’ own perception of their Understanding are validated using advanced data science and artificial intelligence techniques that read and interpret what leaders’ write in discussion forums.
CORP/U’s Natural Language Processing (NLP) algorithms compare words and phrases leaders use against the content in each course activity, yielding a reliable and validated estimate of leaders’ comprehension.
The Understanding Metric is a vast improvement over a basic post-course test.
Leadership teams face front line challenges and make thousands of decisions each day. They develop intuitive sense about what might prevent new concepts from taking hold or delivering the intended benefit. They can predict how organization structures, systems, culture, and time and priorities, might trip them up as they move to install new practices in the way they work. It’s important for leaders to identify and acknowledge potential barriers and make their concerns transparent.
When CORP/U Analytics identifies barriers leadership teams raise, it serves three purposes:
- Provides insights to senior executives about the need for structural changes that could smooth the path and accelerate the pace to improved business results
- Highlights where leadership team should target their collective attention to posit solutions
- Shows a Leadership Development team how to adjust course and program activities to intentionally address barriers
The information is invaluable in creating leadership teams that can truly work as a team to implement an idea, preempting and solving problems before they derail critical initiatives.
With hundreds or thousands of leaders taking part in development programs, their solution ideas can quickly become unwieldy, a gigantic blob of unstructured data. But the blob is likely rich with information and diverse perspectives about how to move past barriers.
CORP/U’s Natural Language Processing (NLP) algorithms go to work sifting through the streams of leader generated ideas performing Thematic Analysis to categorize, group and summarize buckets of data, and filter out nuggets of gold.
Solutions extracted through CORP/U Analytics offer leadership teams roadmaps about how they can proceed with efficient action, making it easier to commit to their next steps.
Commitment to Action
Encouraging leaders to make commitments is the most effective tool- and the first one available after a new idea is presented- to promote behavior change. Commitments work because people value consistency between what leaders say and what they do. Research shows that written commitments are more effective than verbal commitments and those made publicly are more effective than commitments kept private.2
CORP/U leadership development programs encourage leadership teams to make public, written statements of their intentions to change through Commitments to Action.
The method encourages leaders to back up their commitments by explaining how they expect to accomplish their plans and how they intend to proceed. The CORP/U AI Engine evaluates the complexity of each leader’s response. The specificity of plans and explanations of ways to accomplish them illustrates the level of credibility in each commitment.
CORP/U’s methodology then uses these captured commitments to measure- and ensure- collective team follow through. It uses automated reminders, refreshers and check-ins to measure leaders ongoing progress on commitments and action plans. This system-driven approach meets the needs of large leadership teams to share successes, manage lapses, and enlist the power of social support systems.
Moving the needle on any business performance metric takes substantial, collaborative and integrated effort from leaders across horizontal functions and hierarchical levels. Substantial effort means that a sufficient number of leaders must adopt new practices and behaviors so that the rate of adoption becomes self-sustaining and drives meaningful progress toward the business objective – a critical mass of action.
CORP/U Analytics represents a breakthrough in measurement for corporate leadership development because it measures the very things leadership teams need to succeed in implementing ideas and critical initiatives, as well as measure the impact of development programs in creating those factors.
It is time to move from hoping leadership teams are in place and prepared to succeed to knowing that the company’s most valuable asset is future-ready, growing and delivering value.
By Robert N. Bernard and Carl Rhodes
 The State of Leadership Development. (July, 2016) Harvard Business Publishing .
 Behavior Change Intervention Tools. Bartram, Ashlea. (January, 2009), Government of South Australia.
Meet The CORP/U Analytics Team Leaders
Carl Rhodes is the Chief Operating Officer of CorpU. Previously, he served as the Chief Executive Officer of HCI and the Practice Manager of the Corporate Leadership Council in CEB, where he led research and product teams working on strategic talent management. He holds a Ph.D. in research methods and political science.
Robert N. Bernard is Vice President of Analytics at CorpU where he develops algorithms to measure the impact of development on corporate leadership teams. Previously, he served as Chief Data Scientist in Boeing’s Autonomous Systems Division. He specializes in combining quantitative and qualitative data into actionable insights regarding human behavior and motivation.
Ryan Taylor is Director of Product Management at CORP/U where he builds and launches new products that help organizations develop dynamic, deep and diverse leadership pipelines. Ryan has over a decade of experience leading high performing product teams that deliver beautifully designed products that customers love.