In a just-released survey from Dataiku and Cognizant of 200 senior analytics and IT leaders, only 20% of respondents are currently using Generative AI and LLMs in production. So, what is driving the disconnect among these leaders, such as CIOs, between ambitions and actual capabilities? With Generative AI’s honeymoon stage in the rearview, how can these stakeholders effectively conquer the roadblocks en route from pilot to scale — all while navigating regulatory concerns, data infrastructure issues, and the ever-elusive business value?
Take a Scrupulous Look at Your Generative AI Spend
The Generative AI market has proven its permanence beyond mere hype. Despite this, 39% of respondents* experiencing organizational or policy barriers still plan to spend over $1 million on Generative AI in the next year, and 74% plan to spend over $500,000. This discrepancy highlights a critical issue: spiraling costs that often exceed the value provided.
To address this, IT leaders must scrutinize their Generative AI investments to ensure alignment with organizational capabilities and value generation.
Tame the Explosion of Tools & Maintain Tech Stack Optionality
Generative AI is adding complexity to already intricate IT environments. A staggering 60% of respondents use more than five tools for each step in the analytics and AI lifecycle, which complicates the integration of new technologies. Over 32% believe their organization has too many data tools.
Simplifying the tech stack is crucial to maintain flexibility and adaptability in the rapidly evolving tech landscape. This simplification also enhances the ability to integrate new technologies seamlessly.
Combat the New Layers of Operational Risk
Generative AI introduces new layers of operational risk:
- Infrastructure Challenges: 26% of respondents face infrastructure barriers to effectively using LLMs.
- Policy Barriers: 31% encounter organizational or internal policy barriers.
Without the right tools, managing these complexities becomes risky. Alarmingly, 24% of organizations still rely on spreadsheets for data manipulation, leading to significant errors. Furthermore, 88% lack specific tools or processes for managing LLMs, and 43% do not describe their data analytics stack as modern.
The Dataiku LLM Mesh — a backbone for Generative AI applications that enables analytics and IT teams to securely access Generative AI models and services — is an advantageous option for managing LLMs. With dedicated components for AI service routing, personal identifiable information (PII) screening, LLM response moderation, performance and cost tracking, and auditing of entire application flows, you get maximum control while delivering the performance your business expects.
Keep Working on Data Quality (But Don’t Let It Stop You)
Data quality remains a longstanding challenge. Despite modern tools, organizations still struggle with data quality and usability.
Interestingly, the top areas for tool consolidation all tied back to data: data visualization (58%), data processing (54%), and data ingestion/access (52%). The focus should be on getting the right data for specific use cases rather than perfect data. Leveraging tools like Dataiku’s data quality rules and Cognizant’s Generative AI-integrated Data Scan Tool can help identify and address data quality issues effectively.
Your Next Steps
We are still in the early stages of Generative AI adoption in enterprises. Without proper guardrails and scalability, efforts can quickly become unmanageable. A robust, governance-embedded architecture for analytics and AI is crucial to mitigate risks and reduce IT burdens.
To comply with regulations like the EU AI Act, organizations must embed strict qualification and documentation processes into their Generative AI practices. Cross-organizational collaboration and oversight are essential for compliance and effective risk management.
The findings from this report highlight the significant gap between CIOs' aspirations and their current capabilities. By addressing the four macro challenges outlined — managing Generative AI spend, combating operational risks, simplifying tech stacks, and improving data quality — IT leaders can enhance performance, agility, and insights from their data. Some helpful next steps include:
- Evaluate and streamline your analytics and AI toolsets.
- Focus on actionable data quality improvements.
- Ensure your Generative AI investments align with organizational capabilities.
- Embed governance and compliance into your AI practices from the start.
By taking these steps, CIOs can bridge the gap between their ambitions and reality, paving the way for successful and scalable Generative AI implementations.
*This question was asked to a subset of respondents, in this case n=62.