Logistically speaking, data science and AI operationalization is often more difficult for enterprises to execute on (especially compared to self-service analytics) because it requires coordination, collaboration, and change not just at the organizational level, but often at the system architecture and deployment/IT levels as well.
But it’s truly the final and most important step of the process, as data projects are rendered incomplete without being operationalized — that is, incorporated into the fabric of the business to see monetary results that have real impact. So what are the steps to smooth execution? We recently shared four main reasons that operationalization efforts may fall flat so you can prevent them from happening in your own organization and, now, we’re rounding up tips for smooth execution.
1. Alignment With the Business From the Beginning
The first, and perhaps most overlooked, step is simply learning from, listening to, and working with business teams to ensure that the solution that will be operationalized is a viable one. While this sounds like a no-brainer, it’s often overlooked, resulting in lots of lost time and effort from data teams in building a solution that doesn’t actually provide any business value in the end. For more on turning business data into business value, your business counterparts can be sure to check out “7 Concrete Ways to Create More Value From Analytics & AI.”
Data teams work hard to ensure that their projects meet business needs, but the reality is that they often lack the context or business knowledge necessary to build the most optimal solution to a specific problem.
For example, say that the data science team for a website offering personalized loans is tasked with creating a more sophisticated fraud detection model. If they set out to create this on their own without speaking to the team currently handling fraudulent claims, they might create a model that is perfectly reasonable from a technical perspective, but doesn’t answer the original business problem. For example, the fact that too many people are being caught as potential fraud and going to manual review, overwhelming the operations teams. With this contextual knowledge, the data team can tune the model to better balance it to the reality on the ground.
2. Consistent Packaging and Release
From a methodology standpoint, operationalization requires a consistent, efficient process for integration, testing, deployment, and then measuring impact and monitoring performance (followed by, of course, making necessary modifications and integrating, testing, deploying, etc., those modifications). Inconsistent packaging and release can lead to a subtle degradation of a model’s performance between development and production.
Traditionally, it’s the data engineering or IT team that is responsible for refactoring of the data product to match target IT ecosystems requirements (including performance and security). However, this handoff between data team and IT or data engineering teams can be significantly eased when the two are working with the same tools and are aligned on project goals — so again, communication (even between technical teams) is key. Check out this blog for more on how IT can help prevent AI project failures.
3. Efficient Model Retraining
Following release, it is critical to implement an efficient strategy for the retraining and updating of models. Implementing a retrain-in-production methodology is a key to operationalization success; without it, retraining a model becomes an actual deploy-to-production task, with the result requiring significant manpower and a loss of agility.
4. Functional Monitoring and Performance Communication
Additionally, a successful operationalization strategy involves functional monitoring, which is used to convey the model’s performance to the business sponsors, owners, or stakeholders. This provides an opportunity to demonstrate the end-results of the model in production for evaluation. The kind of functional information that can be conveyed is variable and depends largely on the industry and use case. Examples of the kind of data displayed can include the number of contacts in a case, the number of broken flows in a system, and measurements of performance drifts. Functional monitoring goes hand-in-hand with having a viable rollback strategy in case something goes wrong.
This step is critical because knowledge transparency must be constantly shared and evangelized throughout an organization at every opportunity. A lapse in communication can compromise the importance and the value of using ML technology within an organization.
Tying It All Together
Hopefully it’s clear now that operationalization is not a one-time project, but an investment that involves the devotion of plenty of resources (both technical and personnel). Operationalization is rooted in the fact that data science needs are intertwined with business needs and can arise from any line of business or department. That is, data initiatives cannot happen in a central team that exists in a vacuum without a deep understanding of the business (or connections to those who have that knowledge). For example, it would be impossible for a data team to execute successfully on a customer churn prediction and prevention project without input from teams like marketing and/or sales.
Instead, operationalization can thrive in an organization that establishes a central data team as a center of excellence, a sort of internal consultant that can be deployed to activate data efforts across the company, through a combination of self-service analytics and operationalization. In practice, this kind of organizational model means establishing:
- A platform for data access, discovery/exploration, visualization/dashboarding, as well as for machine learning modeling and deployment into production that can be the basis of both a thriving self-service analytics and operationalization environment. This platform (like Dataiku) should ensure that everyone across the organization (regardless of his or her technical skill set) is working with data that can be trusted and that they can produce desired outcomes — whether that means dashboards, a predictive model, etc.
- A centralized — and, importantly, not siloed — data team or organization that maintains said platform, ensuring that all data is accessible, accurate, and generally usable in a self-service context. This team would also be responsible for larger deployment and operationalization efforts based on self-service projects or other data projects executed along with business units.
- A means of collaboration and communication between business units and data team(s) so that any questions arising from data projects can be easily addressed in context — especially questions surrounding where data comes from, what it means, and how it can be accurately used in projects. This means of collaboration should also ensure that any larger data project produced with self-service analytics is validated by the data team to ensure.
- Feedback loops connecting operationalized data projects to business objectives and ensure they continue to meet those objectives (and can be easily adjusted if not).