When it comes to calculating the return on investment (ROI) of data science, machine learning, and AI projects, there’s a lot to unpack. Before even jumping into the measurements and calculations, it’s important to understand the scope. Are you measuring ROI gained from a data platform or a series of data initiatives themselves? Are you modifying a previously existing strategy for tackling ROI or creating one from scratch?
On the art of measuring ROI, an ESI Thought Lab survey says, “It is revealing that 79% of companies that report negative or no ROI, and 56% of those showing ROI of just above 0% to 5%, do not have systems in place to measure returns.” So, is it a matter of organizations not driving ROI or simply not knowing how to properly measure it? We’ll question all of this as we dig into the nuances that make calculating ROI for data efforts notoriously difficult.
1. Often, Teams Don’t Know Where to Begin
There’s no industry standard for measuring ROI from AI and, for many organizations, calculating it is a work in progress. Each company’s data, circumstances, scale, data team composition, and so on are customized toward their own objectives, making ROI for AI tricky to be easily applied across organizations.
2. ROI for AI Is More Nebulous vs. Other Tech Investments
According to the aforementioned survey, determining the ROI for AI projects is harder than it is with traditional technology investments “where the costs and impacts are more easily defined and predictable…[AI] is a versatile tool that can generate a rich set of benefits geared to the financial, strategic, and operational outcomes that a company seeks to achieve.” To take the example of the recent global health crisis, organizations likely didn’t expect to utilize resources scrambling to retrain impacted models, something that will play a role when the ROI for that unique initiative is calculated.
3. There’s No One-Size-Fits-All Approach
Just because one organization sees ROI on a data project within six months doesn’t guarantee that it will be a binding number that applies across verticals or even future projects from the same team. Further, chances are that as an organization increases its AI maturity, ROI will continue to grow. Costs associated with data prep, technology adoption, and team development and upskilling, for example, need to be scaled before an organization can break even and start generating true returns. By identifying ways to reduce these costs, such as through capitalization and reuse, teams can avoid having to start from scratch, find previously untapped, high-value use cases, and focus on impact-generating tasks.
4. Do Qualitative Benefits Count?
Another layer of complexity comes into play with benefits of AI such as improved decision-making, a stronger brand reputation, or more agile processes. Further, AI can unearth insights above and beyond the primary objective of the project or even spur the discovery of new use cases that can drive more value than anticipated, which can, in turn, open the door to new pockets of profit or cost savings. For this reason, we believe these elements should be included in the overall project value.
5. Avoid Fragmented Results
Each group involved (business people, data scientists, analysts, and so on) add their own value to the data project and also likely track their own KPIs relevant to their contributions. It is important to have all of these groups involved from the beginning of the project to ensure alignment and transparency to, ultimately, drive cross-team project success.
6. ROI for AI Is Long Term
Just like achieving alignment across people, processes, and technology is a matter of time, so is the process of monitoring the impacts of data science, machine learning, and AI across an organization via data initiatives. Organizations need to identify the right use cases for the business, find relevant datasets and vet data quality, and then develop, fine-tune, and eventually deploy models — all of which are vital to the greater process and take time.
The same survey on ROI with AI mentioned above states, “With frequently high upfront costs in data preparation, technology adoption, and people development, it takes time and scale to generate significant returns. Just to reach break-even takes 17 months on average for the typical firm.” While this information isn’t telling organizations to take their foot off the pedal, it’s important to remain patient during the initial scaling phase before rushing to identify returns that may not be there yet.
Looking Ahead
Don’t let these challenges stifle your team’s ability to identify a unique process for calculating and tracking ROI. The process may not be seamless — it will likely take some time and be quite iterative, building upon itself as new learnings take place. Here are three quick lessons to keep front of mind when crafting a plan for measuring value and ROI:
- It takes a village. To see tangible results with data science and AI, organizational change is required, from senior management down to individual contributors.
- Collaboration is a requirement. There needs to be cohesion between the data teams and the business teams to identify the right use cases and demonstrate AI’s worth to the company.
- Focus on the areas that will drive strong initial results so teams have a foundation to kickstart their ROI calculations with. For example, 74% of organizations are seeing positive ROI by focusing on optimizing customer service and experience with AI.