Aviva, a major British multinational insurance company, has a thriving data science community with AI use cases spanning customer marketing, underwriting, claims assessment, fraud detection, and IT efficiencies.
However, as AI adoption grew, Aviva’s central IT platform faced mounting challenges in scalability, cost, infrastructure complexity, onboarding efficiency, and governance. The existing system could not keep pace with rising demand, causing delays, inefficiencies, and operational bottlenecks that hindered AI-driven innovation.
Breaking Barriers: The Challenges of a Growing Ecosystem
As Aviva expanded its AI operations, several critical challenges emerged:
- Scalability and Performance Issues: The central Aviva platform struggled to accommodate increasing AI workloads, leading to inefficiencies and system slowdowns.
- High Maintenance and Infrastructure Costs: The complexity of the system required significant resources to maintain, driving up operational expenses.
- Slow Onboarding and Deployment Delays: Lengthy integration processes stalled new AI initiatives, limiting agility.
- Governance and Security Gaps: Ensuring data security, compliance, and seamless collaboration across teams remained a challenge.
- Resource Constraints and Inefficiencies: AI workloads could not be optimized effectively, resulting in excessive compute costs and reduced operational efficiency.
Without an optimized AI infrastructure, Aviva risked losing its competitive edge in data-driven decision-making and operational efficiency.
The Turning Point: A Strategic Partnership With Dataiku and TCS
To overcome these obstacles, Aviva partnered with Tata Consultancy Services (TCS) and Dataiku experts to modernize its AI infrastructure. The solution focused on leveraging Dataiku’s AWS EKS integration and MLOps capabilities to build a scalable, efficient, and secure environment for AI operations. With Dataiku, Aviva established a robust foundation for its CI/CD and MLOps framework to streamline the deployment and maintenance of data science applications.
Dataiku sits as a backbone for the entire solution and supports the growth in demand of AI use cases in the organization. Dataiku as a product has helped to consolidate and simplify platform infrastructure, set up a governance and control framework, roll out data science capabilities faster, enabled us to meet internal security and compliance requirements, and more.
-Adam Warburton, Data Science Platform Lead, Aviva
By integrating AWS EKS, Aviva dynamically scaled compute resources, ensuring seamless AI workload management. Containerized execution and Spark profiles enabled GPU utilization, significantly enhancing complex model training. Security and compliance needs were met through an externally managed AWS EKS cluster hosted in a separate Virtual Private Cloud (VPC), ensuring robust data protection and governance. With Dataiku, the team now has flexibility to scale and choose compute resources (including GPUs) on demand.
No matter how we cut it, run times are halved. Both for large scale data processing in Spark (benefiting from containers) and the locally run ML scoring jobs on EC2. This is real-world data and a good story worth sharing!
-Product Owner, Aviva
Transformative Results: How Dataiku Revolutionized AI at Aviva
Modernizing Aviva’s AI infrastructure delivered significant improvements in efficiency, collaboration, and business performance. The transformation enabled:
- Faster AI Deployment: The time to integrate new AI initiatives was reduced by approximately five weeks.
- Increased System Reliability: Offloading compute workloads to AWS EKS eliminated performance bottlenecks and improved platform availability. Aviva also introduced compute allocation usage monitoring by assigning a resource quota for each team and established cost reporting per data science team, user, and project level.
- Significant Cost Savings: Infrastructure consolidation and resource optimization resulted in millions of dollars in savings.
- Stronger Governance and Security: Fine-grained access controls and compliance frameworks ensured secure data sharing and seamless project execution across different profiles and teams.
- Improved Productivity: Data science teams became more efficient, leveraging streamlined workflows and automated deployment processes. Dataiku streamlined and accelerated the onboarding process for new data science teams, reducing time to market by approximately five weeks.
The Dataiku platform has laid a solid foundation to support current growth while also meeting future demands to support AI use cases within the organization. It has helped to democratize AI across the organization, drive implementation of governance and controls, and maintain collaboration among all data science teams and groups.
-Adam Warburton, Data Science Platform Lead, Aviva
A Future-Proof AI Strategy
The transformation of Aviva’s AI ecosystem has established a foundation for continuous innovation. With Dataiku as the backbone of AI operations, Aviva now possesses an agile, scalable, and cost-effective infrastructure capable of supporting future AI advancements. The company’s strategic investment in AI modernization not only enhances operational efficiency but also solidifies its position as a leader in AI-driven insurance services.