In the realm of modern analytics, transforming data into actionable insights hinges on providing business teams with seamless access to data to build advanced analytics pipelines. The key to realizing this vision lies in the partnership of Dataiku and Databricks. Dataiku and Databricks make AI capabilities accessible to business teams, helping deliver scalable analytics through quick and secure access to data and ML models in Databricks using the easy-to-use, visual interface of Dataiku. By leveraging the power of AI and lakehouse capabilities across data management and performance, business users within Dataiku can make informed decisions.
In the spirit of working smarter not harder, Dataiku has recently launched in Databricks Partner Connect Portal. This portal is a one-stop shop for existing Databricks customers to quickly discover and integrate AI tools to expand the capabilities of their Lakehouse platform. In this blog post, we will guide you through the process of connecting Dataiku to Databricks using Partner Connect. In just a few clicks, you’ll empower AI and business teams alike to accelerate analytics projects.
Step 1: Connect to Your Databricks Environment
The first step is to sign into your Databricks workspace, navigate to the left panel and click on Partner Connect. Once the Partner Connect screen appears you can search and select Dataiku which appears under the Machine Learning section. Step-by-step instructions are provided here.
After clicking on the Dataiku tile, you will be presented with a few steps to configure the Dataiku connection. Then, your Dataiku Cloud environment will be launched with a pre-configured connection to your Databricks Lakehouse. This means after a few clicks you are ready to start working in Dataiku with your Databricks lakehouse data.
Step 2: Explore an Example Project on Dataiku
After setting up Partner Connect in your Databricks environment, you will be moved to the Dataiku Cloud Launchpad. You can navigate to the Connections tab and see that through Partner Connect, Dataiku has automatically created the connection to your Databricks Lakehouse. You will now be able to read and write data directly from Databricks in your Dataiku workspace.
Once you have verified the connection has been established, navigate to the Overview tab and open the Dataiku instance. Once you launch the instance, you can see an example tutorial has been preloaded on this environment to show you how Dataiku and Databricks work together.
The tutorial goes through each feature integration with Databricks showing how you can leverage the best-in-class Dataiku visual interface with the storage and compute power of Databricks. Be sure to hover over each recipe to see a description of the technical integration and explore the Project Wiki to learn more on how Dataiku and Databricks work together.
Step 3: Deepen Your Skill Set
Now that you have an example under your belt, it’s time to get started on your first use case using Dataiku and Databricks! Create a new project, add a dataset from your Lakehouse and explore how Dataiku’s visual features can accelerate your analytics pipeline development. Invite up to five users to your Dataiku Cloud environment to illustrate how expert data scientists, business teams, and IT can work together to create data and analytics projects that everyone can see and understand.
As Simple as 1, 2, 3
That’s it! With just a few clicks, you've seamlessly integrated the collaborative visual capabilities of Dataiku with the robust compute power of Databricks. Through following this guide, you’ve witnessed the simplicity and power that this integration brings. As you dive into different use cases, envision the impact on your pipeline development using Dataiku and Databricks, delivering more advanced analytics and accelerating data and AI adoption within your organization.