Get Started

Crossing the Talent Chasm and the Talent Management Macro Trend

Scaling AI Doug Bryan

Andy Grove was a co-founder and CEO of Intel. The company’s market capitalization grew 50x under his leadership, from $4 billion to $197 billion. During the early days of the PC revolution, an east coast business reporter flew out to Santa Clara to figure out what Intel did and was having a hard time with it. They made machines smaller than your fingernail out of sand? Out of frustration, the reporter resorted to first principles and asked, “What’s the most important input to your product?” Andy thought for a few moments and replied, “Smart 25-year olds.”

A few years later, another east coast reporter visited suburban Seattle to ask Bill Gates, co-founder and CEO of Microsoft, about his company’s book value. They didn’t have any factories, warehouses, trucks, or planes so what was it worth? Bill said that, “The inventory, the value of my company, walks out the door every evening.” Andy and Bill’s insights foreshadowed a trend that’s gaining momentum today. In the digital age, talented employees are almost as important as customers and worth managing like we manage customers.

A few years after Gates’ “walk out the door” comment, theorist and management consultant Geoffrey Moore explained that high-tech products don’t go mainstream unless they get beyond tech enthusiasts and into the pragmatic majority of potential customers. He called it “Crossing the Chasmand it had a big impact on the tech industry. Moore’s insights are now being applied to data science talent management, but what’s changed is that the pragmatic majority aren’t less enthusiastic data scientists, they’re business analysts.  

A chasm has long existed between data scientists and analysts, created by technical complexity and hard-to-use development technology. Data scientists had rare skills that took years to acquire, programmed in R and Python, and wrestled with arcane issues such as overfitting, collinearity, null value imputation, Z-score normalization, normality of distributions, equality of variance, outlier removal, training set balancing, and dataset bias. Those rarified skills kept the number of data scientists low and separate from business analysts.

talent chasm and business skills

McKinsey recently wrote that 87% of executives are not prepared to close gaps like this and that they can’t hire or outsource their way out of it anymore. So how can we close this gap and get more analysts doing data science? Companies have tried to:

  • Provide data science training to business analysts. That fails because most lack the aptitude and interest to learn collinearity and normality so, even though a few pick up the skills, not enough do to create a sustainable learning community. (I once trained 100 business analysts on SQL. Three retained the skill.)
  • Require artisanal data scientists to learn your business. That doesn’t work since many who enjoy collinearity and normality aren’t interested in selling widgets. If you require them to learn about widgets, then they just switch jobs since demand for data science far exceeds its supply.
  • Get rid of collinearity and normality. That works, then far more business analysts can develop data science skills.

Regularized regression, random forests, gradient boosting machines, and best practices for a slew of data preparation tasks made it much easier to learn data science. Graduate programs sprang up around the world and the number of data scientists increased quickly. But people with both skill sets were still rare, so much so that they became known as unicorns.

unicorns with data science and business skills

Over the past few years, cloud computing, open source software, YouTube lectures, Coursera classes, integrated platforms, and automation of more best practices drastically improved usability. Today, the early majority of business analysts are acquiring data science skills and we’ve seen it happening across many companies in many industries. 

business people acquiring data science skills

Going beyond pretty GUIs is what’s increased usability in recent years. Nielsen Norman Group defines usability as five components:

  1. Easy to learn to use
  2. Efficient for the task at hand
  3. Easy to remember how to use 
  4. Prevents serious errors by novice users
  5. Satisfying to use

Here’s how multipersona AI/ML platforms are addressing each of these components:

Easy to Learn to Use

  • Pretty GUI
  • Visual data pipeline development for non-coders
  • Notebooks and IDE integration for coders
  • Dozens of built-in data connectors
  • AutoML

Efficient for the Task at Hand

  • One-click model API deployment
  • One-click model scoring in an elastic compute environment
  • Built-in job scheduling
  • Single pane of glass data pipeline and model catalog
  • Heavy reuse of data preparation tasks and pipelines

Easy to Remember How to Use 

  • Automatic document generation
  • Data and model traceability
  • Model and score transparency

Prevent Serious Errors by Novice Users

  • Automatic checking for common errors  
  • Built-in wikis, task lists, and comments
  • Custom sign-off workflows
  • Always-on monitoring of data and model drift 
  • Always-on monitoring of model risk
  • Model A/B testing
  • Rapid version rollback

Satisfying to Use

  • Easy interdisciplinary team collaboration
  • Big productivity gains
  • Continuously track model catalog, performance and value

AI platforms that address all these usability components are upskilling thousands of analytics and closing the talent gap. McKinsey agrees: The solution to closing the talent gap is upskilling while focusing on analyst happiness and productivity.  We’ve seen it work in many companies in many industries.  Here’s what just a few of our customers are saying: 

Why not turn the business into data professionals?” -John Tudor, GE Aviation Director of Data and Analytics

Dataiku changed our team culture for the better through sharing capabilities.” -Damien Rumble, Aviva Senior Data Scientist

Dataiku is “a central environment that very technical and non-technical people can collaborate in. We don’t throw things over the wall anymore.” -Chris Kakkanatt, Pfizer Senior Director and Team Leader of Data Science

It's all about the wisdom of many people.” -Craig Turrell, Standard Chartered Bank Head of Plan to Perform Data Strategy & Delivery

Amazing. Lightning speed results. Enable play, enable fun.” -Renee Boerefijn, Ph.D. Innovation Director EMEA at Bunge Loders Croklaan


Want to learn more on this topic? Check out some relevant research on the topic below:

You May Also Like

What Is MLOps?

Read More

Solving Foreign Exchange Rate Impacts on Profit & Loss

Read More

How Can AI Help Underrepresented Young People Get Into the U.K.’s Top Universities?

Read More