Andy Grove was a co-founder and CEO of Intel. The company’s market capitalization grew 50x under his leadership, from $4 billion to $197 billion. During the early days of the PC revolution, an east coast business reporter flew out to Santa Clara to figure out what Intel did and was having a hard time with it. They made machines smaller than your fingernail out of sand? Out of frustration, the reporter resorted to first principles and asked, “What’s the most important input to your product?” Andy thought for a few moments and replied, “Smart 25-year olds.”
A few years later, another east coast reporter visited suburban Seattle to ask Bill Gates, co-founder and CEO of Microsoft, about his company’s book value. They didn’t have any factories, warehouses, trucks, or planes so what was it worth? Bill said that, “The inventory, the value of my company, walks out the door every evening.” Andy and Bill’s insights foreshadowed a trend that’s gaining momentum today. In the digital age, talented employees are almost as important as customers and worth managing like we manage customers.
A few years after Gates’ “walk out the door” comment, theorist and management consultant Geoffrey Moore explained that high-tech products don’t go mainstream unless they get beyond tech enthusiasts and into the pragmatic majority of potential customers. He called it “Crossing the Chasm” and it had a big impact on the tech industry. Moore’s insights are now being applied to data science talent management, but what’s changed is that the pragmatic majority aren’t less enthusiastic data scientists, they’re business analysts.
A chasm has long existed between data scientists and analysts, created by technical complexity and hard-to-use development technology. Data scientists had rare skills that took years to acquire, programmed in R and Python, and wrestled with arcane issues such as overfitting, collinearity, null value imputation, Z-score normalization, normality of distributions, equality of variance, outlier removal, training set balancing, and dataset bias. Those rarified skills kept the number of data scientists low and separate from business analysts.
McKinsey recently wrote that 87% of executives are not prepared to close gaps like this and that they can’t hire or outsource their way out of it anymore. So how can we close this gap and get more analysts doing data science? Companies have tried to:
- Provide data science training to business analysts. That fails because most lack the aptitude and interest to learn collinearity and normality so, even though a few pick up the skills, not enough do to create a sustainable learning community. (I once trained 100 business analysts on SQL. Three retained the skill.)
- Require artisanal data scientists to learn your business. That doesn’t work since many who enjoy collinearity and normality aren’t interested in selling widgets. If you require them to learn about widgets, then they just switch jobs since demand for data science far exceeds its supply.
- Get rid of collinearity and normality. That works, then far more business analysts can develop data science skills.
Regularized regression, random forests, gradient boosting machines, and best practices for a slew of data preparation tasks made it much easier to learn data science. Graduate programs sprang up around the world and the number of data scientists increased quickly. But people with both skill sets were still rare, so much so that they became known as unicorns.
Over the past few years, cloud computing, open source software, YouTube lectures, Coursera classes, integrated platforms, and automation of more best practices drastically improved usability. Today, the early majority of business analysts are acquiring data science skills and we’ve seen it happening across many companies in many industries.
Going beyond pretty GUIs is what’s increased usability in recent years. Nielsen Norman Group defines usability as five components:
- Easy to learn to use
- Efficient for the task at hand
- Easy to remember how to use
- Prevents serious errors by novice users
- Satisfying to use
Here’s how multipersona AI/ML platforms are addressing each of these components:
Easy to Learn to Use
- Pretty GUI
- Visual data pipeline development for non-coders
- Notebooks and IDE integration for coders
- Dozens of built-in data connectors
- AutoML
Efficient for the Task at Hand
- One-click model API deployment
- One-click model scoring in an elastic compute environment
- Built-in job scheduling
- Single pane of glass data pipeline and model catalog
- Heavy reuse of data preparation tasks and pipelines
Easy to Remember How to Use
- Automatic document generation
- Data and model traceability
- Model and score transparency
Prevent Serious Errors by Novice Users
- Automatic checking for common errors
- Built-in wikis, task lists, and comments
- Custom sign-off workflows
- Always-on monitoring of data and model drift
- Always-on monitoring of model risk
- Model A/B testing
- Rapid version rollback
Satisfying to Use
- Easy interdisciplinary team collaboration
- Big productivity gains
- Continuously track model catalog, performance and value
AI platforms that address all these usability components are upskilling thousands of analytics and closing the talent gap. McKinsey agrees: The solution to closing the talent gap is upskilling while focusing on analyst happiness and productivity. We’ve seen it work in many companies in many industries. Here’s what just a few of our customers are saying:
Why not turn the business into data professionals?” -John Tudor, GE Aviation Director of Data and Analytics
Dataiku changed our team culture for the better through sharing capabilities.” -Damien Rumble, Aviva Senior Data Scientist
Dataiku is “a central environment that very technical and non-technical people can collaborate in. We don’t throw things over the wall anymore.” -Chris Kakkanatt, Pfizer Senior Director and Team Leader of Data Science
It's all about the wisdom of many people.” -Craig Turrell, Standard Chartered Bank Head of Plan to Perform Data Strategy & Delivery
Amazing. Lightning speed results. Enable play, enable fun.” -Renee Boerefijn, Ph.D. Innovation Director EMEA at Bunge Loders Croklaan
Want to learn more on this topic? Check out some relevant research on the topic below:
- “AI on the Front Lines,” Katherine C. Kellogg, Mark Sendak, and Suresh Balu, MIT Sloan Management Review, May 4, 2022
- “Cracking the Culture Code for Successful Digital Transformation,” Carsten Lund Pedersen, MIT Sloan Management Review, April 6, 2022
- “Democratizing Transformation,” Marco Iansiti and Satya Nadella, Harvard Business Review, May–June 2022