Trust Builders
From transparency and consistency to ethical considerations, let's explore how Dataiku's LLM Mesh can help you deploy GenAI that users can confidently depend on.
Choose the Best Model for the Task
Large language models (LLMs) have become the cornerstone for GenAI, each boasting unique strengths tailored to a use case. For instance, the Snowflake Arctic and Databricks DBRX models excel at tasks like SQL generation and coding, or if you have a use case that requires mathematical reasoning, Llama 3 could be an excellent choice. Due to different strengths and limitations, evaluating various models and selecting the best fit for your situation is necessary. LLM Mesh includes connectivity to many LLMs, both as APIs or locally hosted. The list of LLMs available continues to expand with each release.
Maintain Flexibility, Avoid Model Lock-In
LLMs continue to improve rapidly, making it essential to stay adaptable and open to updating the model used — you want to avoid model lock-in. Tying yourself to one model or infrastructure dependency risks creating technical debt that will be difficult to manage. Therefore, staying flexible and receptive to evolving models is crucial for staying at the forefront of innovative applications in various fields.
Dataiku's LLM Mesh acts as a secure API gateway for your LLMs. It's an intermediate layer to manage and route requests between your GenAI projects and their underlying LLM services, avoiding hard-coded dependencies that lead to downstream technical debt. If you want to use a new LLM in your project, it's as easy as selecting the new connection in your LLM recipes.
Incorporate Internal Knowledge
LLMs available from an API provider or open-source models often need help comprehending and interpreting proprietary data, industry jargon, and internal knowledge unique to your company. Frequently, they are trained on publicly available data, so they do not have the context necessary to generate accurate and tailored insights aligned to an organization's specific use case.
Two techniques can address these limitations: Retrieval-Augmented Generation (RAG) and fine-tuning. The first step in using RAG requires creating a knowledge bank that contains vector embeddings of the information you want to include. Dataiku includes an Embed Recipe that makes this process point-and-click easy. When you query an LLM using RAG, the most relevant embeddings from your knowledge bank are automatically selected and added to your query. This additional information augments the model's foundational knowledge and enables it to synthesize the combined information, creating a new, contextually rich response.
A more advanced technique, fine-tuning, involves adjusting the model's parameters and hyperparameters to customize and optimize a model's performance for specific tasks or domains, resulting in more tailored and precise outputs. Fine-tuning requires further training of an LLM on a smaller, domain-specific dataset to tailor it to your use case. Dataiku also includes a Fine Tune Recipe that makes this process easier. Using either RAG or fine-tuning, you can deliver more nuanced, contextually relevant, and accurate responses for your use case.
Validate Your Outputs
Validating the output of your LLMs via source citations is paramount in ensuring the results' accuracy, credibility, and trustworthiness. Source citations provide you with an additional way to verify the accuracy and authenticity of the information produced. Using Dataiku's Prompt Studio, you can include sources in your outputs. Having the sources cited makes it easy to cross-reference and confirm that the generated output accurately reflects the source material.
Use Moderation
Sanitizing GenAI results is imperative to ensure that generated outputs align with your organization's ethical standards and regulatory requirements. Implementing robust moderation methods can help you safeguard against disseminating potentially harmful or inaccurate information and improve the integrity and trustworthiness of the generated outputs. LLM Mesh includes many features that can assist with moderation:
- Forbidden terms — Automatically remove user-defined terms from the query input or output.
- Toxicity detection — Easily enable toxicity detection in query inputs and outputs.
- Personally Identifiable Information (PII) detection — Detect PII information in your queries and allow you to either reject the query or remove, replace, or redact the PII portion of a query.