You already know AI Governance is essential to a mature analytics and AI production pipeline, but are you unsure where to start? You may have seen great demonstrations of what solid analytics and AI Governance could look like, but are you having a difficult time projecting how this would look in your own infrastructure? Are you simply interested in understanding and mitigating risks across your production pipeline and making it more efficient? We’ll get you sorted!
Since the release of Dataiku Govern, Dataiku’s newest environment dedicated to managing analytics and AI projects at scale, we have been supporting dozens of clients across industries, geographies, and governance maturities. Over the course of this work, we’ve identified some common challenges that organizations have to understand and address to efficiently deliver on analytics and AI Governance. One of the biggest pains organizations face is clarifying and gathering the governance requirements needed to use a governance tool, like Dataiku Govern. Where they are not well defined, these requirements embody the chasm between organizations knowing a governance solution is needed and organizations implementing an effective and relevant solution. This two-part blog post aims at helping organizations close this chasm and get to value faster.
How? We believe that stating governance requirements drives clear business objectives and reduces time to business value. For example, formalizing your internal auditing process with Dataiku Govern can accelerate your adoption time significantly by decreasing a week-long process to half a day. In order to prove this argument further, in the second part of the blog, we’ll explore defining good governance requirements before reviewing easy ways to cut down your time to value with Dataiku Govern.
The Why: Compliance and Risk Management Requirements
If your organization is interested in governance, mitigating operational or reputational and compliance risks, your first step is to reflect on why this is important.
Many organizations we talk to are unsure how to qualify their requirements because there’s not a concrete understanding of what’s driving the need. Without a clear needs statement, it’s nearly impossible to generate an appropriate framework that reflects business objectives. It’s like putting a puzzle together without the picture on the box and all of the pieces flipped over. You start without a clear goal of your end product, no final imagery to help you put the puzzle together, so you’re left without a process to select the pieces and make sure that the generic square at the end represents something accurately and to a particular standard. Your why is the picture of the finished puzzle on the box. It informs what your process should be, how to implement a framework, and helps establish a baseline value to compute the lift of implementing a governance solution.
Generally speaking, we’ve been able to break down organizational why’s into the five categories in the image below. These five categories are driven by what we observe, not by what we think should be the case. As such, we don’t think there is a right answer and we find organizations see anywhere from one to all five of the categories as important to them. A secondary question that ought to be asked and answered by whoever is pushing forward with a governance initiative is the weight of that importance. This in turn helps to hone discussions about the why as well as the what and how.
1. Scaling Strategy
This is the widest category and can mean different things to different organizations. An example of this particular why is when leadership has set the direction for AI scaling, seeking innovation and new use cases across business areas that had been, to date, light on data science capabilities. This strategy is responsible for calculating and ensuring big lifts for business units, to deepen and widen usage, estimate the value, and drive AI practices into new business units. Here, analytics and AI Governance can play a role in establishing common expectations and standardized practices that align to the strategic course laid out.
2. Ethics
This is principally about how to embed notions of what’s right and mitigate notions of what’s wrong in data science pipelines by leveraging an analytics and AI Governance framework. Clarifying values/ethics is a challenge in and of itself and approaches like Responsible AI can become a significant part of governance, especially around concepts like reliability, accountability, fairness, and transparency. Dataiku has built-in visuals to help you along this journey, but some items in this category might include requiring sign-offs on models or ensuring visibility and control over your model lifecycle.
3. Risk/Compliance
Risks relating to AI are a common theme across geographies and sectors. Organizations like NIST in their AI Risk Management Framework and the EU through the AI Act emphasize that risks related to AI are important and exist across pipelines. Besides, where unmitigated, these risks can impact individuals, society, and businesses alike. Internal policy or external regulatory compliance are therefore important considerations here insofar as when they are integrated into a governance practice, your AI and analytics pipeline can be fortified with the right processes to mitigate reputational, financial, or consumer-related risks as well as with content for audits.
4. Operational Efficiency
Imagine you have teams across different business units who all have their own systems to organize their approach to governance or there’s a patchwork of teams with their own approaches and others operating without strategy. This is all too common, described as a decentralized or a federated data science structure. And this approach can make sense, especially if data science teams are dedicated to specific business functions: a data science team that supports HR will not demand the same requirements as one that works on customer-facing products.
Establishing a baseline or foundational governance approach with flexibility for various business lines helps to create some consistency. In turn, this allows for greater efficiencies at the enterprise level, making your programs more scalable and easier to read cross-functionally. This could be your data scientists who need to have a standardized way of monitoring model drift, required information about projects or models, or approaches to beginning new challenger models.
5. Value Growth
This governance category centers around AI usage. It estimates the risk involved and determines business outcomes of AI usage across the business. Assets, resources, outcome, operationalization, and business value are all of interest.
Where do you sit on your AI Governance motivations? (Source: Dataiku)
Once your whys are gathered, it’s important to prioritize them between must-have and nice-to-have in order to focus on the most important requirements for your organization. The why you created might be different across business units because of different risk requirements. As an example, marketing data is less sensitive than patient data which might fall under data protection requirements. Just note that the rules can be adapted and are flexible to meet your needs. Now we are ready to move from the why to the what in the next part of this blog post!