Machine Learning Enables Employers to Gain More Insights on Skills

This post previously appeared on the SHRM blog.

In 2019, our team at Lighthouse Research & Advisory set out to determine if artificial intelligence (AI) is better than humans at some HR tasks. We know that automation can process some actions faster than a human, but what about finding insights in employee engagement data? Can a computer really do that better than we can?

In the experiment, we gathered 1,000 responses from workers across North America on workplace stressors, manager relationships, and more. We then shared that data with a few consulting firms for expert inputs and recommendations. The insights were broadly helpful and interesting, but they weren’t actionable. For instance, we found that 60% of our “workforce” was stressed at work, but we didn’t get any clarity into who was stressed or why.

We then ran that same data through an algorithm that looked at sentiment within the comments. The output was incredibly dense and rich with insights. For instance, we could see specific stressors—such as manager relationships, working conditions, and family responsibilities—by gender, race, job, or geographic location.

This proved to us that AI is better than humans at analyzing feedback data to help us take action as organizational leaders.

Just like in the experiment above, there are other areas where rich data sets exist within the business that haven't been mined for insights. We’ll explore some of the areas where AI can suggest actions, offer insights, and create value for employers who leverage technology to examine one of their richest, untapped data sources: employee skills.

Skills and the Spatial Relationship

Understanding the spatial representation of skills provides a clear picture of how closely skills are related to one another, as well as to those entities represented with skills (jobs, for example). This enables us to determine a more optimal path toward a target result—in this case matching workers or candidates to jobs, content, learning, mentors, and so on; and vice versa, matching jobs to candidates, learning to workers, and more.

Let’s use a basic example to explore the concept. In a marketing job, these might be relevant skills:

• Writing
• Research
• Analysis

However, in a software engineering job, these skills may apply:

• Coding
• Collaboration
• Analysis

In each of these two jobs, there’s a need for analysis of outputs, but otherwise, the skills don’t overlap heavily. That means the relationship between coding and writing skills (while both of them actually mean sitting and typing at a computer) is relatively far apart spatially. However, within each job, the skills have a much closer relationship. It helps to think about jobs not as discrete and distinct entities, but as clusters of skills.

Now, when you expand this very basic explanation across the thousands of possible skills across the millions of jobs people do, you start to see how the challenge of identifying and relating this universe of skills is something that is well-suited for an algorithm to support.

AI and Machine Learning Insights

As with the experiment we covered above, humans can do some limited analysis of skills, but there is so much more information than we can easily process or use. Machine learning and AI allow us to use this raw skills data in a wide variety of ways to support talent and business decisions. For example, it can uncover:

• Suggested skills. Supports individual skills development paths.

• Skills gaps. Uncovers what skills a company has and what skills they need, and development opportunities or personalized recommendations for learning content.

• Skills verification. Confirms suspected/inferred skills.

• Suggested gigs/opportunities. Suggests personalized recommendations to leverage strengths.

• Job-matching. Provides personalized recommendations for relevant jobs and advancement opportunities.

While not an exhaustive list, this helps to paint the picture of how an intelligent organization with a foundation of data-driven AI can derive and utilize the value from large skills-focused data sets.

Future Vision: The Self-Developing Workforce

The potential for this type of approach is incredible, but it also opens up something that we’re seeing increasingly in the talent market today. Workers are increasingly expecting control over the velocity and direction of the work they do. The future vision of this AI-enabled and skills-focused strategy allows us to have a self-developing workforce. Machine learning understands what skills exist and where gaps may be and helps to target development, hiring activity, and other interventions to solve problems before the company may even realize the problems are happening.

Is this a bit fantastical? Yes. But even a few short years ago, we would have said the same about an algorithm identifying what skills a person has with any degree of clarity. Our research shows that understanding the skills of your organization is a critical business problem, not just a siloed talent issue. Leveraging technology to solve this challenge will create a competitive advantage for those employers willing to adopt and use it. It’s not really a question of “if.” It’s a question of “when.”