If you didn’t already catch it, last month, Ahmad Khan, Head of AI/ML Strategy at Snowflake, and Catalina Herrera, Field CDO at Dataiku, presented a fantastic webinar. In the session, they added their insights and technical expertise on the hot takes featured in our 2024 predictions e-magazine on the future of advanced AI.
If you missed it, grab your popcorn and hit play, or if you’re trying to hit your 2024 reading goals, keep going for a recap of the topics they covered in the session.
A New Kid on the Block: Generative AI & LLMs
2023 witnessed a transformative shift, marking the rise of Generative AI as the new player on the scene. Reflecting on the past three decades of AI development, there has been a notable transition from exploratory phases to practical applications that genuinely generate value. The discourse around AI, once dominated by marketing buzz, has now materialized into tangible use cases, prompting interest from C-level executives eager to integrate this technology into their enterprise strategies.
With the advent of Large Language Models (LLMs) and the public release of ChatGPT by OpenAI, there's a palpable shift in the types of questions arising from different business units. The focus has expanded beyond structured data and classic machine learning (ML) techniques to embrace the uncharted realm of unstructured data, opening up a universe of possibilities for enterprise use cases.
Generative AI, often represented by LLMs, emerged as the "new kid on the block” with novel capabilities but also orchestration, optimization, and active human involvement requirements for effective integration into decision-making processes. As we navigate this evolving technological landscape, the umbrella term of AI continues to encompass the diverse ways in which we strive to replicate and enhance human capabilities through digitalization and innovative AI models.
Amid the initial experimentation, the enterprise landscape showcased a unique disregard for budget constraints, highlighting the urgency to investigate and experiment with the transformative potential of LLMs. Security, particularly in sensitive sectors like healthcare and finance, emerged as a paramount concern too. The intricate balance of data and model governance became a hot topic as organizations grappled with the implications of open access to public APIs. This pivotal year laid the groundwork for continued evolution, emphasizing the significance of security considerations and strategic planning in integrating LLMs into the enterprise.
What Can’t Be Ignored: The Physical Necessities
With Generative AI application, the important role of hardware infrastructure is clear. Emphasizing the inseparable link between hardware and software, it's clear that the realization of advancements such as deep learning, computer vision, NLP, Generative AI, and LLMs relies on the physical foundation.
This includes a transition from traditional CPUs to the rising prominence of GPUs, adept at handling the simultaneous computations essential for AI, ML, and deep learning. The surge in interest from business users keen on leveraging these technologies prompts a crucial shift in hardware architecture. Beyond simply expanding use cases for Generative AI and LLMs in enterprises, there's a pertinent need to consider the physical aspects of where these computations occur. This prompts reflection on the interconnected considerations of hardware choices, compute costs, and the necessary agility to navigate these changes, highlighting the tangible implications of technology implementation.
Key Generative AI Use Case Considerations
Starting with straightforward applications utilizing pre-built LLMs like ChatGPT for seamless Q&A interactions, the spectrum extends to more intricate scenarios requiring further experimentation and a more in-depth cost analysis between different models. The flexibility to choose and evolve within these use cases becomes critical, particularly when considering the associated cost variability and architectural decisions.
The technology landscape, with the advent of LLMs, introduces complexities that demand a nuanced approach, acknowledging that not every enterprise pursues identical objectives. This necessitates a level of adaptability, emphasizing the empowerment of experts who navigate structured or unstructured data to derive value. Using the example of handling unstructured data, such as call transcripts, the need to normalize sensitive information while maximizing infrastructure efficiency is highlighted.
Dataiku, as a facilitator, empowers teams to experiment and choose the right LLM, considering public options, open source models, or third-party services, within defined parameters. The emphasis lies in understanding the cost variability, maintaining control over experiments, and operationalizing insights.
Dataiku’s LLM Mesh empowers organizations to build efficient, enterprise-grade applications while mitigating concerns related to cost management, compliance, and technological dependencies. Additionally, it provides the flexibility to choose among a growing array of models and providers, including those made available by LLM Mesh launch partner, Snowflake. Our LLM Mesh integration with Snowflake allows customers to connect to privately hosted models inside their Snowflake account.
From Chatbots to Logical Reasoning Engines
One thing is certain — while analytics has traditionally focused intensely on structured data, the evolving landscape has prompted a reevaluation of the value proposition, recognizing the wealth embedded in unstructured data for a comprehensive data strategy.
In the pursuit of a next-generation knowledge base, the integration of structured and unstructured data presents a unique challenge and opportunity. A common objective emerges — crafting a chatbot-like interface capable of responding to diverse queries, particularly addressing HR-related questions or facilitating code generation. While code generation currently stands out as the most performance-driven use case, the exploration of long-tail, industry-specific applications underscores the expansive potential of Generative AI. This diversity leads to a categorization of use cases into distinct buckets: predictive insights from unstructured data, knowledge-based chatbots, and the intricate realm of multi-step logical reasoning engines.
Perhaps the most prominent use case involves leveraging LLMs to engage in conversations with data. Many enterprises, including those using Snowflake as their data platform, are eager to explore this avenue, recognizing its potential to enable self-service data interactions for business users.
However, as excitement mounts, the challenge lies in moving beyond the allure of a demo to actual production, where considerations of performance and model accuracy take center stage. The ultimate aspiration is to achieve a stage where businesses can seamlessly enable self-service data inquiries, despite the non-deterministic nature of AI systems. This journey involves meticulous efforts to ensure that the right answers are delivered consistently.
Addressing Cost Variability and Operationalization Challenges
As enterprises embrace Generative AI and LLMs, addressing cost variability and operationalization challenges is absolutely imperative for ideal ROI. The focus shifts beyond mere use case deployment to the operationalization phase, ensuring the human-in-the-loop validation of outcomes.
With diverse teams approaching use cases differently, Dataiku's role in unifying insights, optimizing infrastructure usage, and facilitating scalable learning across teams is more important than ever!
Balancing Security, Performance, and Cost Considerations
The significance of data security has also become increasingly evident. Many enterprises prefer to implement tailored solutions from platforms like Snowflake that bring Generative AI technology to the secure, governed dataparticularly as enterprise focus transitioned from experimentation to production use cases as the year progressed. The shift to production also resulted in heightened attention on model performance.
The emergence of LLMs as multi-step reasoning engines prompted an exploration of various model sizes tailored to specific use cases. This shift underscored the importance of performance optimization, highlighting challenges such as hallucination and inaccuracies that necessitated ongoing refinement. Additionally, the journey toward LLM deployment necessitated a careful consideration of infrastructure and cost implications. The demand for GPUs raised questions about optimizing costs, choosing between large and small models, and navigating the intricate decisions associated with GPU types, CUDA drivers, and hosting preferences, especially for leading open-source models like LLama and Mistral.
Key Insights and Trends for the Future
The trajectory of AI, particularly LLMs, unveils several noteworthy insights. The focus has shifted from experimental phases to the production realm, with a spotlight on use cases dealing with unstructured data, showcasing promising opportunities for practical applications.
Anticipated performance improvements in LLMs take center stage, with expectations of newer versions, including the possibility of GPT-4 and other commercial models. Experimentation with diverse architectures, such as retrieval-augmented generation, highlights the dynamic nature of evolving technologies. Striking a balance between open-source and commercial LLMs becomes crucial, necessitating a nuanced approach that considers factors like cost and performance at the forefront.
Plus, the evolving legal landscape around AI introduces challenges related to copyright and attribution for content creators, and this underscores the need for collaborative industry dialogues to navigate these intricacies effectively. Moreover, the importance of explainability in AI models gains prominence, with discussions hinting at API enhancements for increased transparency.
Looking ahead, we will likely witness advancements in LLMs, narrowing performance gaps, and continued experimentation with evolving architectures this year. The forecast encourages businesses to maintain flexibility, adopting modular approaches and the right platforms to seamlessly integrate new models that align with their evolving business use cases.
As teams navigate the diverse components of Generative AI and LLMs, Dataiku and Snowflake are here to provide a unified framework to maximize opportunities and efficiently deploy use cases into production.