As AI architectures continue to evolve, enterprises face a fundamental challenge: how to balance flexibility with security while integrating modern analytics and AI tools. At Dataiku Product Days, Jed Dougherty, VP of platform strategy at Dataiku, joined Lynn Heidmann, VP of content and product marketing at Dataiku, to discuss the shifting landscape of enterprise data infrastructure and how organizations can navigate these changes effectively.
With over a decade in the market, Dataiku has seen AI architectures take many forms — from legacy on-prem setups to fully cloud-native environments and, now, an increasingly hybrid reality. In this session, Jed shared firsthand insights into how organizations are adapting their architectures, the challenges they face, and how Dataiku supports them in this transition.
The Architecture Challenge: A People Problem as Much as a Technical One
For enterprises adopting or evolving their AI infrastructure, technology is only part of the equation. Many organizations have enterprise architecture teams that define how systems should connect and operate, ensuring standardization and security. The challenge isn’t about Dataiku being difficult to integrate — it’s about ensuring it fits smoothly within existing enterprise frameworks, helping teams adopt a more effective approach.
Rather than requiring organizations to build from scratch, Dataiku provides best-practice recommendations that streamline AI deployment. How this takes shape depends largely on where a company sits on the on-prem to cloud-native spectrum — each with its own challenges, constraints, and opportunities.
3 Common Scenarios for Enterprise AI Architecture
1. On-Prem & Legacy Systems: Stability Meets Complexity
Many large organizations, particularly in regulated industries like finance, government, and defense, operate within strict on-prem environments due to security and compliance requirements. While cloud adoption is growing, these enterprises often cannot move sensitive workloads off-prem, making it critical to have efficient, scalable ways to manage their existing infrastructure.
However, on-prem architectures come with challenges:
- Specialized expertise: Hadoop and other on-prem data environments often require skills in Java, MapReduce, Scala, and PySpark, limiting usability.
- High infrastructure costs: Maintaining large-scale, on-prem compute clusters is expensive.
- Scalability constraints: Expanding AI workloads in an on-prem environment is more complex compared to cloud-based alternatives.
Jed shared that Dataiku has long supported on-prem deployments, integrating with Hadoop clusters, Teradata, Greenplum MPP databases, and other enterprise systems. This flexibility allows Dataiku to function as a centralized platform — providing analysts, data scientists, and engineers with a more streamlined way to work, even in complex environments.
With Dataiku, organizations can better manage these complexities, providing a more efficient way to work within their existing infrastructure. By integrating with existing on-prem compute engines, Dataiku allows teams to execute analytics and AI workloads seamlessly, without requiring a full migration to the cloud.
2. The Hybrid Model: A Pragmatic Transition to the Cloud
For many enterprises, fully moving to the cloud isn’t immediate — or even realistic. Instead, they operate in a hybrid environment, where some workloads remain on-prem while others shift to cloud platforms like Snowflake or Databricks over time. While this approach offers flexibility, it also introduces complexity, as managing both on-prem and cloud resources simultaneously can be challenging.Jed and Lynn shared an example of a major financial institution that uses Dataiku across thousands of users while navigating this shift. Their security architecture (SecArch) team is gradually approving cloud components, allowing certain workloads to transition incrementally. Some data still runs on Cloudera, while other workloads are already operating in Snowflake.
As Jed explained, “A dataset is a dataset, whether that's a dataset for a Cloudera Hadoop cluster, or it's a dataset for Snowflake, or it's a dataset for Databricks.” In Dataiku, this means teams can swap underlying connections without disrupting their workflows, making it easier to adapt as infrastructure evolves.
This hybrid state is where Dataiku excels, enabling enterprises to modernize without costly retraining or infrastructure-wide overhauls. Analysts and data scientists continue working as usual, regardless of shifts in compute resources.
Key benefits of hybrid architecture with Dataiku include:
- Seamless access to both cloud and on-prem resources
- Minimal disruption to workflows and user experience
- Incremental modernization without forcing a full overhaul
Rather than waiting for an “all-or-nothing” cloud migration, enterprises can modernize at their own pace — with Dataiku providing continuity across both environments.
3. Cloud-Native: The Freedom & Complexity of Choice
For organizations born in the cloud, the challenge isn’t legacy infrastructure — it’s decision paralysis. Cloud providers like AWS, Azure, and Google Cloud offer multiple competing ways to accomplish the same tasks, from running SQL on big data to managing compute resources. With so many options, organizations can get stuck in an endless loop of architecture planning, delaying implementation while debating which tools, platforms, and configurations are best.
This uncertainty can lead to "choice stasis" — an extended period of iterating on architecture diagrams instead of deploying AI solutions. For example, within AWS alone, an organization looking to run SQL on big data might choose from EMR, Redshift, Athena (on Glue), Snowflake, Databricks, or even running Spark in a Kubernetes cluster. The sheer number of possibilities can slow progress rather than accelerating innovation.
To help navigate this complexity, Dataiku — the Universal AI Platform — provides a flexible way to get started with analytics and AI without getting locked into a single architecture. Instead of requiring a fully built-out cloud stack before work can begin, teams can connect to existing data sources, spin up compute resources as needed, and evolve their architecture over time.
Key benefits of cloud-native AI with Dataiku:
- Start small and scale seamlessly without needing a fully built-out cloud architecture.
- Easily connect to multiple cloud services and treat cloud storage, databases, and compute resources as interchangeable components.
- Maintain flexibility across cloud platforms and transition from S3 to Snowflake or Databricks to Redshift without rebuilding workflows.
By abstracting complexity, Dataiku ensures that cloud-native enterprises can take action now, rather than waiting until they have the “perfect” cloud architecture in place.
The Rise of GenAI & LLMs: New Challenges, Familiar Problems
The conversation wouldn’t be complete without addressing GenAI and large language models (LLMs). These technologies introduce new technical and operational challenges while amplifying long-standing AI concerns. Security, cost, and scalability have always been priorities, but GenAI raises the stakes, requiring organizations to move faster and at a larger scale.
The biggest questions enterprises face:
- Security: Should LLM workloads run on-prem, or can sensitive data be processed externally?
- Cost: How can organizations manage the significantly higher costs of LLM inference?
- Scalability: How do teams stay flexible without creating long-term technical debt?
The sheer pace of innovation adds another layer of complexity. New models emerge constantly, each specializing in SQL generation, document summarization, or multimodal AI. Instead of relying on a single model, most enterprises now use multiple providers, balancing security, cost, and capability. The challenge isn’t just choosing the right model — it’s staying flexible as models continue to evolve.
Beyond security and cost, LLMs introduce new operational challenges:
- Unpredictable Output: LLMs are non-deterministic, meaning the same query can return different responses, making evaluation difficult.
- Lack of Standardized Metrics: Unlike traditional ML models, LLMs lack universal benchmarks, forcing teams to define their own evaluation methods.
- Overwhelming Choice: With constant new releases, some teams get stuck analyzing options instead of deploying solutions.
Rather than waiting for the GenAI and LLM landscape to stabilize, organizations need infrastructure that supports continuous iteration and model flexibility. The most effective AI strategies prioritize adaptability over perfection, ensuring teams can seamlessly integrate new capabilities as LLMs evolve.
Final Thoughts: The Future Is Hybrid, Flexible, and Secure
There is no single “right” way to architect analytics and AI infrastructure. Whether fully on-prem, transitioning to the cloud, or cloud-native, successful AI teams focus on flexibility, security, and efficiency — not locking themselves into a single approach.
With AI evolving fast, organizations need infrastructure that keeps pace. This is where Dataiku's scalable, agnostic platform helps. By integrating multiple AI services and enabling seamless model experimentation, Dataiku ensures teams focus on AI development — not infrastructure constraints.
Bottom line: AI infrastructure will keep evolving. The organizations that succeed will be the ones that build for flexibility from the start.