In the earlier articles of this series, we’ve discussed the need for AI builders to be aware of the benefits and risks associated with it, as well as our first deep dive into risks associated with the source data. Now, it’s time to discuss the risks and impacts associated with models and service implementation. This implementation of AI-driven services can take many forms. As for many digital technologies, it depends on both software and hardware components. Such a technical stack has specific characteristics that imply a potential negative impact on the use and the maintenance of the relative service.
Evaluating How Explainable the Service Is
We know that the lack of understanding behind AI models can sometimes slow down AI adoption. In this direction, the ability to transcribe the model behavior as a decision tree or into a rule (and make it explainable) not only reassures users and regulators on the process of adopting the service but also drastically speeds up the identification of incoherence or potentially dangerous approximations made during the training phase.
Another side effect is that a service for which we cannot easily explain the behavior would possibly:
- Suffer the competition of pre-existing customs or new circumventive habits
- Be regarded as shady and see its adoption reduced accordingly
The essential aspect of many digital services is their adoption, as they are generally born from transformation processes and mean to replace pre-existing ones. No matter its resilience and efficiency, a service with no adoption requires additional investment to either correct its behavior or invest in communication campaigns with users.
Confidence in Algorithms and Methods
ML algorithms are based on statistical methods that vary drastically according to the use case. Once researchers and the community validate the efficiency of an algorithm for a given type of problem, you would pick an ML library that supports this method to implement in a project. Their implementations are also quite different. The libraries implement these techniques under constraints like the compatibility with frameworks used for data manipulation and preparation, the computation resources leveraged, or the ability to scale or distribute parts of the process.
In fairness, implementing these services could not always go by the book. Some situational conditions would suggest using a different type of algorithm generally used for another problem or a custom implementation of a known algorithm. The danger lies in custom implementations or alterations of the original algorithm, beyond a potential individual implementation mistake.
Long-term maintenance of such a model will inevitably pose additional constraints, especially given the fact that the majority of ML libraries have:
- An intense release lifecycle and that their previous versions are quickly deprecated and no longer maintained
- Standard tools and methods to verify and potentially constantly monitor the effectiveness of generic algorithms
The weight of maintenance must be the principal factor influencing the choice of your methodology and processes. It’s more difficult to operate changes on an existing service, and a successful project generally paves the road for others. MLOps modern stacks are getting better at curbing the increase of maintenance effort, but most business efficiency relies on your algorithmic approach.
A safe and recognized library and algorithm should always be your baby steps. So, even if you have a better and more innovative approach, starting with implementing the conventional one would at least give a chance to compare the actual gain in every business and technical aspect.
Continuity and Production Readiness
Once you complete the implementation of a model or service, as long as you can assert its robustness and comprehensiveness, its adoption is generally guaranteed. Such a service could impact operational decision-making, a business-critical service, or simply be integrated as mandatory in other applications’ workloads in the organization.
We also acknowledge the possibility of service disruptions caused by a software or hardware resource outage. Data-intensive applications rely on multiple layers of software provided by many vendors or open source communities installed on several data centers. There’s an obvious need to put in place or review various measures to recover the service from an IT standpoint, but these are not the core aspect of the problem we want to address here.
Still, AI is never a means to an end, so its disruption signifies a service or function is wholly or partially unavailable. Therefore, there’s a need for a detailed answer to each scenario surrounding its unavailability.
In the early days of ML in the industry, we observed many experimentations where AI became a weak point in the service architecture to the point that:
- Attackers use it to disrupt a critical service.
- It exposed the competitive advantages of the organization. For example, competitors could easily instrument tools to collect information and determine a company’s strategy for a given market segment through a recommender system.
The side effects of these unsuccessful experiments were sometimes severe enough to have some industries dismiss ML for years, as it occurred for some cybersecurity services. Nevertheless, many among us tend to believe that releasing a service technically safe and successful from a business standpoint is the end of the road.
It isn’t. The nature and scale of changes AI brings are beyond that, so we will end this series of blogs by addressing the risks associated with the impact of AI adoption. Stay tuned!