Solving the Data Science Talent Shortage

Scaling AI Lynn Heidmann

AI and data science talent is notoriously hard to hire. Not only is there a general disparity between available and needed talent; there is also the fact that the fancy data scientists with the doctoral degrees tend to be unavailable (not to mention expensive), because 80 percent of them are taken by Google.

If trends like augmented analytics and automated machine learning (AutoML) are any indication, new tools and technology are constantly narrowing — but notably, not closing — the data science talent shortage. Yet according to Gartner in its report Leading Upskilling Initiatives in Data Science and Machine Learning, “machine learning literacy remains low in most organizations; concerted education and culture change are necessary but difficult.”

So what should organizations looking to get started now do?

By 2020, the number of data and analytics experts in business units will grow at three times the rate of the experts in IT departments.”

- Gartner report Leading Upskilling Initiatives in Data Science and Machine Learning

  1. Educate: Helping staff both understand how AI, data science, and machine learning fit into the larger company’s strategy can be just as important as educating people on the concepts and technology themselves. By clearly communicating the value, existing employees can more easily see how upskilling fits into the picture. In a survey conducted by Dataiku at EGG London 2019, 75% of respondents are already leading some type of education program around the role of data science in the business. So the next step is actually leading education in data science itself.

    pie chart from Dataiku AI Maturity Survey data science enablement
  2. Invest in technology that supports upskilling and collaboration: After education, the next step to addressing the data science talent shortage is actually providing the tools that allow non-data scientists to participate in the data science process. This means features that support AutoML and augmented analytics, but also that allow non-data scientists to do things like connect to data and do data cleaning or exploration — all without code. But it also means being able to work on projects where needed with people like data scientists and data engineers. This is critical because investing in upskilling without the ability to then make these employees part of the overall data science, machine learning, and AI strategy is a misplaced effort.

Gartner, Leading Upskilling Initiatives in Data Science and Machine Learning, 19 July 2019, Peter Krensky

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, and is used herein with permission. All rights reserved.

You May Also Like

From Vision to Value: Visual GenAI in Dataiku

Read More

Understanding the Why and How of the LLM Mesh Architecture

Read More

The Ultimate Test of ChatGPT

Read More