How are sectors such as retail, financial services, and healthcare managing the complexities of data and AI? We surveyed 200 IT leaders from large, internationally recognized organizations to understand their approaches to using Generative AI (GenAI) and to identify the operational challenges they encounter.
Join Dataiku's Field CDO, Gernot Klein, and Cognizant’s Global Offering Leader, Sandeep Upadhyay, as they delve into four key areas from the survey results and discuss practical strategies that top firms can employ to effectively navigate this evolving landscape.
Are Organizations Seeing Strategic Returns in GenAI Investments?
Organizations are increasingly recognizing GenAI as a transformative force. Eighty-five percent are investing in GenAI, whether through a dedicated budget or other budgets. Within this group, 46% plan to spend over $1 million, and 11% aim to invest more than $10 million in the coming year. Despite these commitments, 39% of those facing organizational or internal policy barriers to using LLMs in the way that they would like will still invest over $1 million, and 74% at least $500,000.
This raises an important question: Are these substantial financial commitments aligning with the expected strategic outcomes?
Sandeep points out that GenAI is still in its early stages. Historically, transformative technologies take decades to fully mature. Currently, businesses are largely in an experimental phase with GenAI, primarily focusing on operational efficiencies. Thus, while investments are significant, aligning these expenditures with long-term strategic objectives is still an evolving process, and the full benefits are yet to be seen.
Combatting New Layers of Operational Risk in GenAI Deployment
As the conversation progresses, Gernot Klein and Sandeep Upadhyay delve into a critical aspect of GenAI deployment: the operational risks organizations must navigate.
The survey reveals several widespread issues: infrastructure challenges, organizational or internal policy barriers to using LLMs as desired, and 88% of respondents who reported that they lack specific tools and processes for managing LLMs. Additionally, 43% do not view their data analytics stack as modern.
Given these challenges, how should organizations approach effective GenAI deployment? Gernot emphasizes the importance of adopting an end-to-end, integrated workflow for AI deployment rather than relying on a disjointed array of tools. He also warns of the significant costs associated with noncompliance and the potential need for expensive system rebuilds if these issues are not managed carefully.
Streamlining Data Management With Fewer Tools
When asked how many tools they use, 60% of respondents reported using more than five tools across the analytics and AI lifecycle. However, this contrasts with their ideal scenario, where 71% expressed a preference for using five or fewer tools.
Sandeep notes that organizations with GenAI in production typically use more tools — an average of nine — compared to those without GenAI, who use around five. This increase in tool usage reflects the added complexity of integrating advanced AI technologies. He explains, “Over a period of time, the number of tools will go down, but at least in the short term, we will see a little bit of proliferation of technologies as clients bring Generative AI into their ecosystems and scale them forward.”
To navigate this complexity, Sandeep highlights Dataiku’s LLM Mesh as an example of a solution that enables end-to-end GenAI management within a single platform, helping organizations streamline their processes and reduce tool dependency.
Optimizing Data Infrastructure to Improve Usability and Quality
As the discussion shifts to data infrastructure challenges, the survey reveals that data quality and access are the primary concerns among respondents.
Gernot and Sandeep discuss the concept of “clean data,” agreeing that while improving data quality is important, it should not hinder GenAI progress. Instead of aiming for perfectly clean data, organizations should concentrate their efforts on cleaning data that is crucial for their most valuable use cases — those that offer the greatest benefit to the organization.
Additionally, Gernot points out that 47% of respondents reported that data quality management is handled by IT or a central team. He argues that this approach often leads to inefficiencies and mistrust because no central team can adequately manage data to meet every need. Instead, Gernot advocates for taking ownership of your own use cases and data to enhance trust and operational effectiveness.
Even if you only do it on a use case by use case basis, you need to have easy access to your data and it needs to be fit for purpose. It doesn't need to be perfect, but it needs to be fit for purpose.
-Gernot Klein, Field CDO, Dataiku