When it comes to AI maturity and the role of data in organizations’ digital transformations, many — if not most — companies struggle to go beyond the media hype. Very few projects ever get to the critical stage of entering production, and even fewer reach scale (or if they do, it takes too long).
The bottom line is that discussions around data are happening at most companies, but they fall flat when it comes to implementation and industrialization, which is problematic because:
- It considerably slows down innovation and transformation within organizations when they need it most.
- Companies (and their employees) are spending huge amounts of energy on projects that, in the end, add hardly any overall value and even less ROI. The more time passes, the more the people involved in data and AI projects become bored with the situation, are discouraged, and are no longer engaged in making these initiatives a success.
This creates stagnancy internally and feeds “shadow IT,” along with all the difficulties that this can entail. How can IT teams get past implementation issues and start industrializing AI?
Neither Forcing — Nor Avoiding — Industrialization Works
When AI projects arrive in production, they are often no longer aligned with what was decided with business stakeholders in the design phase. By forcing projects into production, they often get transformed at a technical level, with recoding, data changes, etc., all which directly impact models’ usefulness and the ability to keep them updated (hello, data drift).
The opposite might be the ridiculous situation where data innovators say “Let’s avoid going into production, because it will be problematic.” Hopefully it’s obvious why this isn’t an option in today’s data landscape. In fact, today, more advanced organizations aren’t debating whether production is important or not, but rather who should be responsible for deploying to production.
It is not all bleak, however. At Dataiku, we are seeing companies that have decided to concentrate on the production phase in intelligent ways.
Learn From Success Stories
There are various examples, such as giants like GE Aviation and SLB, but also includes the likes of Euronext. The common denominator is not the sector, country, or company size. The notable difference comes from initial commitment to the idea of AI industrialization.
These companies aren’t saying: “Let’s do data science.” They’re saying: “Let’s lead a company transformation project,” which goes a lot further than the question of data on its own. From there, the support is different, as well as team strategy and the willingness to facilitate organizational, cultural, or technical changes.
When data science, machine learning, and AI aren’t alone in their corner — when they turn into Everyday AI — it can become a joint effort across IT and business lines. This approach is very different from the data labs that popped up in many organizations several years ago, which end up working on the fringes of the company but aren’t truly embedded into the core of the business.
If we take the example of Euronext, the company has implemented small, mixed IT-business teams that are truly considered transformational teams and are embedded into the heart of the company. In other words, they are not “removed” from the company to innovate on their own.
Data labs can be thought of as a starting point to test concepts, but longer term, we’ve seen success at Dataiku with a hub and spoke approach. That means creating a central “hub” in the company to manage governance and the availability of tools, whether for data science or for other domains.
The users of these resources are the spokes, the boots-on-the-ground carriers of data and AI projects The hub is not an active operator but rather the overseer that facilitates the production phase. This setup can be split into sub-sectors in the case of large companies that have operations or business lines that differ greatly.
The Role of IT
The reality is that it’s natural in the hub and spoke model for IT to be the central overseeing agent. The key is not to fall into excessive centralization, which is a big temptation in IT — the hub must help, but it can’t substitute itself in for the spokes. IT teams must avoid the trap of providing technological choices to the business that then require reworking AI projects once they have to go to production.
That means from the outset, investing in a platform (like Dataiku) that avoids this desynchronization between business and IT. As with most business applications, the AI platform must be seen as a bastion of stability, centrality, and a meeting point that allows quality, obsolescence, and documentation to be managed dynamically. Without this management, transformation is useless: who has done what, when, on what version, and for what exact reason?
Providing this central platform has become even more pertinent as regulation is tightening around the processing of data and AI — it’s becoming critical to be very meticulous regarding traceability. The hub minimizes risks, but it is useless unless it integrates the business lines in a way that’s agile and flexible. At the same time, IT teams can’t put their heads in the sand by falling back on practices that hinder innovation.
Keys (and Barriers) to Success in Industrializing AI
Unfortunately, companies or data labs are often made up of data scientists who think like craftspeople. That is, they prefer to work alone than to rely on a shared platform which would automate part of their work while also providing a level of project supervision (even if this means losing sight of their primary objective, which is to create value for the company).
In some sense, this thinking makes sense in the context of the cultural wave we’ve been in for several years that talks about data science and similar roles as “the sexiest jobs of the 21st century.” What must change is not only this mindset among data scientists, but also the organization, including the vision of directors. The Chief Data Officer (CDO), IT leaders, and more play a role in shifting this status quo. But how?
The biggest win will be in creating a common language across all those involved — business, IT, data scientists, and more, by:
- Investing in a platform that minimizes code and design changes between project development and production.
- Having documentation that is clear to everyone, including a shared understanding of what is being done (data transformations), the way data is processed, etc.
- Centralizing knowledge, which, bonus: also does as good a job of facilitating the adoption of new arrivals as teams experience staff turnover.