As 2024 becomes a part of history, the worldwide impact of generative AI (GenAI) has been nothing short of historic. Business has changed forever this year and, at Dataiku, we have been privileged to play a pivotal role in this transformation.
Throughout the year, we introduced new, powerful ways to make GenAI not just accessible, but truly enterprise-ready. Let's explore a few of our top GenAI innovations that have shaped the landscape of AI this year. Before we hop into our highlights, we’ll break each of our favorite features into tiers:
- The Game-Changers: These are the features that have redefined what is possible for enterprises. From generating trusted answers to creating compelling stories from your data, this tier is for the features that lead to immediate impact for all organizations.
- GenAI Your Way: This tier showcases features that put unprecedented control and customization in your hands. Whether it’s fine-tuning LLMs or generating code, these capabilities ensure your AI solutions align directly with your unique business needs.
- Clarity and Trust: These features address the critical needs of transparency and security, ensuring as AI grows more powerful, it remains explainable, secure, and trustworthy for all stakeholders.
Now, on to the list!
Tier 1: The Game-Changers
The features in this tier were truly the game-changers for data teams and business stakeholders. Three standout features have fundamentally reshaped how organizations harness the power of AI, setting new standards for enterprise AI platforms.
Dataiku Answers
Dataiku Answers transformed how organizations leverage their institutional knowledge. Dataiku Answers is a standout for teams for its zero-configuration setup and polished user interface. It also allows stakeholders to easily understand their insights in the same platform where data teams build and operationalize their data projects. Organizations can now instantly transform their trusted knowledge bases into interactive AI insights in an instant.
The impact has been remarkable — just ask Akamai, who saved over 6,000 hours by streamlining their data discovery process with their chatbot. Using Dataiku, they built a locally hosted LLM-powered chatbot that serves as a comprehensive tool for data exploration, making complex information accessible to users across all skill levels. This combination of efficiency and accessibility, while maintaining strict governance through citations and source tracking, demonstrates why Dataiku Answers has become essential for enterprises serious about knowledge democratization.
Dive Deeper Into Dataiku Answers → Discover how organizations are going from questions to answers, fast.
LLM Evaluation
The LLM evaluation recipe tackled one of enterprise AI's biggest challenges: production monitoring. The recipe output goes beyond basic metrics, it provides a powerful, visual way to both measure and monitor LLM performance at scale.
For operations teams, this means full visibility into their AI systems' performance and creating LLMOps safeguards using automation. The ability to generate detailed metrics has transformed how organizations approach LLM governance and quality assurance. The result? More reliable AI systems and confident stakeholders.
See More on LLM Evaluation → Confidently monitor, improve, and iterate with GenAI in Dataiku.
Dataiku Stories
Closing out the year, Dataiku Stories reimagined data storytelling by solving a critical challenge in business reporting: keeping insights fresh and accurate. Unlike traditional static presentations that quickly become outdated, Stories dynamically connects to live data sources, automatically refreshing visualizations and metrics. While GenAI accelerates the creation process, the real power lies in maintaining a single source of truth that updates in real time, eliminating the risk of outdated screenshots and ensuring stakeholders always see current data.
This isn't just about saving time – it's about democratizing the way teams communicate insights while ensuring every bit of information is verified and trustworthy. Rescue your presentations by housing them in the same platform where the data work lives.
Read the Blog → More on how Dataiku Stories transforms how we share insights.
Tier 2: GenAI Your Way
These features put unprecedented control in the hands of our users, enabling AI solutions that adapt to specific enterprise needs.
LLM Fine-Tuning
LLM fine-tuning brought enterprise-grade customization to the Dataiku LLM Mesh. LLM fine-tuning offers two powerful approaches: a visual, managed way for non-coders to fine-tune models through an intuitive interface and a Python-based approach that gives developers full flexibility with access to state-of-the-art techniques.
Whether using HuggingFace instruct-based LLMs or OpenAI models, teams can adapt pre-trained models to their specific domains while maintaining enterprise-grade governance. This dual approach ensures that both citizen data scientists and experienced developers can create AI solutions that truly speak their organization's language.
Read More About Fine-Tuning → Dive into the benefits of LLM fine-tuning in Dataiku!
Generate Recipe AI Assistant
The generate recipe, our December innovation, represents a breakthrough in data transformation automation. By understanding natural language descriptions and considering both dataset schema and sample data, it suggests accurate transformations while providing clear explanations of its recommendations (Go to 1:48 in the video below to see it in action!).
For business users, this means no more wrestling with technical syntax — just describe what you want to achieve, and let the assistant handle the complexity. For data experts, it dramatically accelerates the preparation process, turning time-consuming recipe configurations into quick conversations with AI. The recipe can also learn from user feedback ensuring continuously improving suggestions, making it increasingly valuable for data teams. This new recipe can help directly tie business intent and technical implementation while cutting data preparation time.
Dive Deeper Into Generate Recipe → Understand how Dataiku can take your data prep to the next level!
Tier 3: GenAI for Clarity — Understanding and Trust
Our final category focuses on features that enhance transparency and understanding across the AI lifecycle.
AI Explain
Dataiku gave you AI Explain last year, but 2024 made it even more powerful with the ability to explain code. This update changed how teams understand and document their code assets, whether working with Python scripts, SQL queries, or custom code recipes. Teams can now generate instant, customizable explanations of their code - from high-level overviews for business users to detailed technical documentation for developers - while preserving context.
These explanations can be seamlessly incorporated into project documentation via the project wiki, transforming how organizations capture and share technical knowledge, accelerating developer onboarding, and ensuring project continuity. What used to take hours of careful documentation now happens swiftly, letting developers focus on what matters most: building powerful solutions.
RAG and Knowledge Bank Updates
What a year it was for RAG in Dataiku! Users can now improve their RAG pipelines with sophisticated text chunking controls and intelligent knowledge bank updates, eliminating redundant processing. Vector store flexibility expanded dramatically, with new support for Azure AI Search and ElasticSearch, while the introduction of parent-child retrieval has enhanced context preservation in AI responses.
These innovations are complemented by shareable knowledge banks, enabling teams to build unified sources of truth across their organizations. This blog just scratches the surface. Dive deeper into Christina Hsiao's recent Dataiku blog, which provides a comprehensive look at these game-changing features here:
Read More on RAG in Dataiku → Find out how to streamline your RAG pipelines!
Prompt Injection Detection
Prompt injection detection was a big feature for admins and IT leaders. Users can now select any generic completion LLM as a security judge, with the flexibility to customize detection prompts and add specific examples for improved accuracy.
→ Read Now: Other powerful ways Dataiku mitigates risk!
Injection detection strengthens security by enabling admins to customize the guardrail's system message, providing detailed instructions for handling detected threats and attacks. Integrated seamlessly with advanced guardrails, it delivers robust protection while maintaining system usability. For enterprises concerned about AI safety, this represents a significant step forward in securing their LLM implementations.
Looking Ahead
As we reflect on these features, it's clear that 2024 has been a transformative year for GenAI and enterprise AI. The features mentioned represent more than just technical capabilities – they embody our vision of making AI more accessible, trustworthy, and valuable for organizations worldwide. By combining cutting-edge GenAI capabilities with enterprise-grade controls and transparency, we're helping organizations transform their data operations while maintaining the highest standards of governance and trust.