AI is growing exponentially, companies are leveraging this technology to automate internal processes, boost productivity, and increase staff and customer engagement. According to the recent ESI ThoughtLab report on Driving ROI through AI, 31% of AI leaders report increased revenue, 22% greater market share, 21% faster time-to-market, 19% creation of new business models, and 14% higher shareholder value. Most tellingly, the AI-enabled functions showing the highest returns for companies are all fundamental to rethinking business strategies for a digital-first world: strategic planning, supply chain management, product development, and distribution and logistics.
But with great power comes great responsibility. In Dataiku 8, we focused a large portion of our development efforts on helping organizations feel confident with transparent and explainable AI. Here are three key Dataiku features that will move the needle on transparency and explainability for data team leaders.
Model Document Generator
So a data scientist built a model and knows everything about it, from how the data was prepared to the features to the details of deployment. But what about the stakeholders? Providing model documentation is quickly becoming a requirement, but it can be a huge pain to manually update a document after every minor change to keep it accurate.
Dataiku makes model documentation easy, standardizing with one template and automatically generating the necessary documentation. Kiss the hours of manual copy and paste goodbye!
The Dataiku Model Document Generator outputs a .docx file for easy collaboration with the broader organization — all you need to do is click a button once you have trained a model and inspected the results, grab a cup of joe, and voilà! The document is ready to go in a few minutes. The output document also follows a consistent, logical sequence, with clearly indexed sections, delivering a standardized framework for future reference and comparisons.
Individual Prediction Explanations
In Dataiku 7, we introduced individual prediction explanations (i.e., row-level explanations for why a model is producing a given prediction) as part of the Dataiku interface and as an output of the scoring recipe. With Dataiku 8, you can now programmatically obtain these row-level explanations through APIs. It’s not just for AutoML either — individual prediction explanations can be generated for models built from scratch.
Row-level interpretability helps data teams understand the decision logic or "why" a model makes specific predictions. With the global movement toward more stringent data protection laws (requiring data teams to adhere to frameworks like the European Union General Data Protection Regulation, GDPR, and the California Consumer Privacy Act, CCPA), individual prediction explanations ensure compliance and explainability of complex machine learning predictions for hassle-free governance.
Audit Centralization & Dispatch
Audit trails are important, and having a central view is imperative. Dataiku 8 allows you to centralize audit logs across multiple Dataiku DSS nodes through a routing dispatch mechanism. This includes your API node query logs to complete the feedback loop (and your MLOps strategy).
Automated model documentation, row-level prediction explanations, and centralized audit logs in Dataiku 8 help organizations create a traceable and transparent path to Enterprise AI. Without robust transparency and explainability systems, organizations could risk building black-box AI applications that can output prediction at scale without understanding it's more profound implications. With Dataiku, organizations can quickly create sustainable and responsible AI practices without investing in additional governance tools.