At Dataiku, we’re all about empowering everyone (including the business) to participate in data science and AI initiatives, accelerating the time it takes to deliver AI projects from months to days (i.e., through concepts like reuse and automation), and ensuring governed AI projects throughout their lifecycle. In fact, it is some of these market trends in practice that are fueling a shift in the data science and machine learning (DSML) market at large as historically separate markets converge.
As Gartner® notes in the “State of Data Science and Machine Learning” report, “DSML platforms are now addressing two growing and equally important markets. A ‘multipersona’ market with a central focus on time to value, ease of use, and collaboration between multiple technical and nontechnical personas and an ‘engineering’ market focused on primarily technical personas whose primary aim is to engineer (design, develop, deploy, monitor, and maintain) scalable, enterprise-wide AI solutions. The need to cater to an expanding number of personas is bringing new and innovative functionality into these platforms.”*
However, this evolution of the data science and machine learning market undoubtedly makes it challenging for data and analytics leaders when selecting and implementing a data science and machine learning platform, as their internal requirements (both for the business and technical sides of the house) become even greater and more diverse. Gartner summarizes the key trends in the data science and machine learning market under three main themes of access, automation, and acceleration:
1. Access
According to Gartner, “Access relates to the expanding group of user types or personas that require access to the consumption, application, or creation of DSML models.”* As we shared in our annual trends report, 2021 was the year that a significant number of organizations had the realization that they will not scale AI without enlisting diverse teams to build and benefit from the technology. Not only are there additional roles involved in AI projects (think risk managers, subject matter experts, IT operations, project managers, and more), but we observed the roles of three specific groups of business stakeholders evolve:
- Analysts - These roles have existed for a while and have always worked with data (i.e., business analysts). They are becoming increasingly more efficient because of factors like transitioning out of spreadsheets and internal programs to upskill them into citizen data scientists.
- Business users - In the past, these people may have operated with data, but with low data and analytics maturity (i.e., not analysts, but people such as marketers or supply chain managers). They are increasingly empowered to directly build or co-build analytics workflows with experts because they have more access to data, upskilling programs, and proper tooling.
- AI consumers - While they don’t build the solutions or directly work with data, this growing pool is benefiting from data science and AI by being surfaced AI tools and applications built by data teams or one of the other groups above.
2. Automation
According to Gartner, “Automation is about the use of AI and other techniques to automate or augment activities across the entire DSML pipeline from data sourcing and preparation to modeling, deployment, and monitoring.” At Dataiku, we believe that the confluence of analytics and automated BI with data science and machine learning will continue and, instead of just becoming more interconnected, they will actually become one practice for the organization. Automation (including RPA) often involves automating mundane or repetitive tasks while bringing high accuracy, reliability and traceability. Teams often start with tasks prone to human error and tasks that can provide a loss of employee motivation and efficiency, such as payroll processing and claims processing.
3. Acceleration
According to Gartner, “Acceleration enables companies to shorten the time to value for DSML, mostly through more streamlined deployment, integration, and operationalization of models.” At Dataiku, we believe organizations need to take a systemized approach to AI, which involves unifying business, data, and IT teams to work together and collaborate to design, deploy, and manage AI projects, each bringing their own talents to the table. This also involves upskilling business analysts and experts with visual tools that empower them to work with data and build AI models, dashboards, and applications. Then, once that foundation is set, Dataiku helps teams go faster by:
- Building on a visual canvas with reusable components and standard frameworks that make building production-ready projects easy and fast, even for business teams
- Taking advantage of project accelerators and solutions that kickstart their efforts with best practices and industry knowledge
In order to navigate this rapidly evolving data science and machine learning landscape, data and analytics leaders should identify which roles or personas need access to data science and machine learning (both now and in the future), take into account the different needs of advanced, highly technical people like data scientists versus other, less technical ones, and pinpoint which capabilities impact the scope and nature of data science and machine learning usage in their organizations.
*Gartner, The State of Data Science and Machine Learning, Pieter den Hamer, Carlie Idoine, Afraz Jaffri, Farhan Choudhary, Shubhangi Vashisth, Peter Krensky, 10 December 2021