In late 2021, Dataiku released Govern which was designed to support organizations to secure their MLOps practices. Our goal was to provide Dataiku users with the foundations for analytics and AI Governance that could be leveraged by relevant teams and individuals, supporting them to safely scale their analytics and AI projects while prioritizing efficiency, enforcing trust, and providing oversight.
Dataiku’s standard Govern provides users with AI Governance fundamentals and the means to secure their MLOps practices.
Since first releasing Govern, the world of AI Governance has advanced significantly in a number of ways. Globally, governments have demonstrated advances in regulatory and non-regulatory interventions to shape how organizations build, buy, and deploy AI. The European Union has progressed its proposed AI Act through Parliament, the U.S. has seen the first iteration of the NIST AI Risk Management Framework and the Office for Science and Technology Policy’s Blueprint for an AI Bill of Rights, Singapore’s IMDA released AI Verify, the U.K. published a policy paper on AI regulation, and sector regulation has begun to evolve to address the wider use of AI.
While governments gave a sense of direction on what new rules and expectations for AI development and use would likely look like, firms the world over began to invest greater resources into their AI Governance frameworks. Altogether, in the past 24 months, AI Governance has begun to be taken more seriously by all manner of global actors and in that seriousness, topics like risk, responsible and trustworthy AI, management of data assets, and impact and value have become increasingly salient and challenges worthy of being tackled.
As the world of AI Governance advanced, so too did new techniques and use cases. Foremost amongst these is the increasing interest, investment, and usage of Generative AI. The promise of Generative AI to facilitate greater efficiencies has been met with calls for responsible development and use from the same cast of actors mentioned above. And so the boom of Generative AI is yet another catalyst for establishing AI Governance practices. As Dataiku launched our Generative AI offering, we simultaneously provided Responsible AI training and a framework for assessing use cases.
Altogether, the developments in AI and AI regulation ought to be heard as a clarion call for more sophisticated, business-line and use-case-specific approaches to AI Governance. Our response to that call is Advanced Govern which takes our standard offer to a new level.
Advanced Govern
Releasing Govern was only the first step in our analytics and AI Governance offering. We know that many organizations will require more than what the fundamentals can provide. And to accommodate this, we have released Advanced Govern. Advanced Govern empowers users to build highly customizable governance processes into their Dataiku environment.
What drives organizations to require such customizations? In general, we’ve identified a core set of motivators that resonate globally.
Without going through each motivator in turn, it’s important to highlight that there is no single right set of motivators. And, in fact, we see various permutations depending on a number of factors.
For instance, a global financial services firm scaling their analytics and AI teams might be concerned about risk & compliance, documenting the impacts of their analytics and AI investments, and enforcing standardized MLOps practices across the growing teams. To meet these requirements their governance approach will be informed by regulation, internal decisions around reviews, sign-offs, and deployment infrastructure, as well as key KPIs that document model performance and business impact — all of which inform a bespoke customization to meet their needs.
With Advanced Govern, such an organization is able to:
- Build bespoke project and model-level workflows that incorporate key obligations set out in regulation, including model performance monitoring
- Ensure the consistent documentation of inputs into such workflows for the purpose of audit as needed
- Enforce required steps aligned to their MLOps practices
- Develop ‘business initiative’ or ‘program initiative’ views for managers that aggregate a set of projects’ expected and existing impacts across time
Here is an example of KPI tracking across projects which sit in a custom page only available through Advanced Govern.
In another situation, a company not facing any specific regulation might decide to develop a governance approach aligned to the National Institute of Standards and Technology (NIST) AI Risk Management Framework (RMF) in an effort to establish strong risk management practices that align with US domestic standards developments. Further, they’re also convinced that all relevant models they produce should be fair and free from bias.
While NIST’s AI RMF could be variously interpreted, an organization such as this might:
- Build bespoke project and model-level workflows that specify intent, give space to discuss datasets used and their qualification, specific metrics during the model build and pre-deployment, as well as space to articulate foreseeable and unforeseeable risks and explanations around risk tolerance for a specific model or project;
- Systematically implement critical requirements for post-deployment monitoring;
- Enforce one or multiple review and sign-off stages across the project pipeline.
Here is a model-level example of how an Advanced Govern workflow can be customized to meet NIST’s AI RMF.
These are only two small examples of the potential customization that is achievable with Advanced Govern. Whatever the set of motivators for analytics and AI Governance, diving into the details reveals the need to make important decisions about what kind of information should be documented and how decisions should be made and documented across the analytics and AI pipeline.
The Key Takeaway
Dataiku is solidifying its analytics and AI Governance offering, working towards refinement through customization.
We think customization is critical to effective governance because of the diversity of organizations’ motivators and approaches. And so our goal is to empower Advanced Govern users to build bespoke, which means integrating and executing analytics and AI Governance in the best way that works for them whether independently, with us, or a partner.
Governance Solutions are also available to Advanced Govern users. Solutions are designed to support maximizing time-to-value in specific thematic areas and can be used off-the-shelf or further customized. Our first solutions address the EU AI Act Readiness and GxP in the pharmaceutical industry.
If you have governance needs that are detailed, complex, informed by leadership or regulation, or simply conform to unique ways or working or standard operating procedures, our Advanced Govern offering could serve your organization for the better.