Generative AI (GenAI) is shaping up to be one of the greatest technology disruptions impacting business and society in our lifetimes, and enterprises are now firmly engaged in winning the race to materializing concrete outcomes and competitive advantage from these new technologies.
But this new technology brings its own set of unique considerations and risks. How will you:
- Protect sensitive data?
- Control costs?
- Create future-proof solutions and infrastructure that won't need to be re-architected every time a newer generation of models emerges?
- Ensure proper governance and compliance with AI regulations?
Not engaging with GenAI at all is risky for long term market relevance, but engaging without careful strategy around these questions may be even riskier! This article highlights exactly how Dataiku enables operational efficiency and innovation with AI while maintaining security and the right guardrails so that you (yes, we’re talking to you, IT leaders) can sleep at night.
Dataiku: A 3x Gartner® Magic Quadrant™ Leader
Dataiku was recently named a Leader for the third year in the 2024 Gartner Magic Quadrant for Data Science and Machine Learning Platforms. Among all evaluated vendors, Dataiku is placed furthest on the “Completeness of Vision” axis. We believe that, with Dataiku — the Universal AI Platform — IT teams don’t have to choose between enabling transformation and maintaining control. Say goodbye to disparate tools and an overly complicated tech stack and say hello to tech optionality with Dataiku, one single platform for all analytics and AI needs. Dataiku enables teams to:
- Build better and faster: Customers routinely cite a 60%-80% reduction in time spent to go from data and AI project ideation to production.
- Reduce risk: Underpin all processes with the governance required to leverage AI at scale and make informed decisions about risks and resources.
- Control costs: Use the right technology for the job, reuse assets, track spending, and rebill efficiently with one single, centralized platform for data, analytics, and AI.
Dataiku is a real accelerator for a company embarking on the AI journey, aiming to bring IT and business on the same platform.
The Key to Safe GenAI Scaling: The LLM Mesh
The Dataiku LLM Mesh provides a scalable framework for efficiently building safe, secure, GenAI-powered applications while addressing IT concerns related to cost, compliance, and technological dependencies.
With dedicated components for AI service routing, personal identifiable information (PII) screening, LLM response moderation, performance and cost tracking, and auditing of entire application flows, you get maximum control while delivering the performance your business expects.
Decoupling Application From Service Layer
With the constant influx of new LLM providers, models, and vector databases these days, flexibility is key when rolling out Generative AI-powered solutions across an organization. By building in a future-proof manner from the get-go, teams can ensure that solutions can easily adapt to adopt the latest and greatest technologies. This decoupling makes it possible for teams to design the best possible applications and then maintain them easily in production.
The Dataiku LLM Mesh provides an elegant solution to the flexibility challenge, not only reducing operational risk but also making it simpler for application developers to satisfy needs at various project stages, benchmark different options, ensure scalability, and take advantage of future technological innovations. By removing hard-coded routing and technical dependencies between applications and AI services, organizations can reduce operational risk, more easily test and evaluate different models during the design phase, and more nimbly adapt to new innovations and models once in production.
Enforcing a Secure API Gateway
To keep up with the pace of change in LLMs, companies should also consider how they can empower data teams to quickly access approved models for experimentation. But, that agility may seem to come at a cost. By giving users the ability to choose models, admins may feel like they’re losing control over both cost and visibility. With the Dataiku LLM Mesh, now IT teams can empower data teams to explore the latest in LLMs, all while maintaining control.
The LLM Mesh acts as a secure API gateway; an intermediate layer to manage and route requests between applications and their underlying services so as not to create the hard-coded dependencies that lead to downstream technical debt.
With a centralized place for IT administrators to manage and control access to approved LLMs and AI services, enterprises can easily connect to AI services from commercial LLM providers such as Azure OpenAI or OpenAI, or utilize locally-hosted models, whether privately developed or downloaded from Hugging Face. Similar to regular database or data storage connections in Dataiku, the LLM API gateway reduces risk by securing API keys and tokens and enabling administrators to control access to these specialized models. This allows IT teams to easily access and swap out approved models in applications as well as remove the burden of configuring and maintaining connections.
Protecting Sensitive Data
To protect companies that choose to both prevent the input of sensitive data into public services as well as prevent unwanted types of output from public or privately hosted LLMs, admins must have mechanisms to both identify and take action around this type of data. The LLM Mesh provides content moderation techniques that can be implemented both for queries and responses, leveraging dedicated content moderation models to filter input/output. It can help detect PII and take action, identify and filter forbidden terms, prevent toxicity, create custom hooks, and keep tabs on problems with audit trails.
Controlling Costs
With all the different costs associated with LLM usage, IT teams and admins need a way to easily monitor LLM cost and performance so that they can proactively anticipate and manage financial impact. LLM Cost Guard — part of the Dataiku LLM Mesh — provides a means for teams to oversee and control costs (by application, services, users, or projects) and diagnose issues. Dataiku offers comprehensive audit logs that enable admins to easily pinpoint who is doing what, performance monitoring to help admins track usage and costs for LLM services and providers, the option to cache responses to common queries with the LLM Mesh avoids the need to regenerate the response (offering cost savings and a performance boost) and cost and resource usage reporting to proactively identify where teams are spending the most.
The LLM Mesh, like Dataiku overall, is infrastructure agnostic and vendor independent, enabling choice and flexibility among the growing number of models and providers. Unlike other platforms that are aligned to specific cloud infrastructure or model provider's offerings, Dataiku connects to the latest AI services from ALL major commercial LLM providers, as well as model hubs like Hugging Face, if you choose to use open source models and self-host.
Similar to our long-time approach to machine learning, Dataiku's focus for GenAI is on making the technology accessible and usable by customers in their existing enterprise environments, instead of focusing on providing our own models and compute.
In fact, in the Gartner 2024 Magic Quadrant for DSML Platforms report, in the “Market Overview” section Gartner states that, “DSML platforms have evolved to cover full-stack data science, from infrastructure utilization in multi-cloud environments to data engineering and pipeline building, model training and experiment tracking, deployment and model tracking, and front-end applications. Differentiation between platforms is now based on the amount of abstraction that is provided so team members can quickly iterate on solutions without needing to dive deep into each layer.”
We believe that one of Dataiku's key strengths is providing enterprise-wide usage of GenAI models to abstract away the complexity of managing cost and performance, thanks to the LLM Mesh. A robust AI Governance framework and the pre-built EU AI Act readiness solution to support risk management and GenAI oversight are also key differentiators versus other offerings in the market.
IT Success Stories From Dataiku Customers
The fun isn’t over — we’ll leave you with stories from various Dataiku customers, specifically IT leaders, that have seen tangible success with Dataiku. Hear from Aviva (with Wipro), FLOA Bank, ZS, and ENGIE on how their IT teams enabled data science teams to develop more models and improve MLOps, helped business units to scale their impact, and enhanced security, compliance and governance — all with Dataiku, the Universal AI Platform.
GARTNER is a registered trademark and service mark of Gartner and MAGIC QUADRANT and PEER INSIGHTS are a registered trademark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. Gartner Peer Insights content consists of the opinions of individual end users based on their own experiences with the vendors listed on the platform, should not be construed as statements of fact, nor do they represent the views of Gartner or its affiliates. Gartner does not endorse any vendors, product or service depicted in this content nor makes any warranties, expressed or implied, with respect to this content, about its accuracy or completeness, including any warranties of merchantability or fitness for a particular purpose. Gartner Peer Insights reviews constitute the subjective opinions of individual end users based on their own experiences and do not represent the views of Gartner or its affiliates. Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.