It’s not who has the best algorithm that wins, it’s who has the most data.” - researcher Jean-Claude Heudin
It’s no secret that training AI models is an energy-intensive and large dataset-dependent endeavor and, to corroborate this, researchers at the University of Massachusetts Amherst performed a lifecycle assessment on large, widely accepted AI models trained on the vast datasets needed to achieve accuracy. The research confirmed that, even on common AI models, the process can emit more than 626,000 pounds of carbon dioxide equivalent — nearly five times the lifetime emissions of the average American car (and that includes the manufacturing of the car itself).
At first glance, this could be interpreted to mean that companies with large datasets and the accompanying resources to train and operationalize large mainstream AI models are the inevitable victors in the race to adopt, scale, and gain meaningful insight. However, as focus on the triple bottom line (people, planet, profit) and sustainability increases, companies are now conducting reinforced scrutiny on all processes for increased economic and environmental efficiency. This opens the intriguing possibility that having the largest datasets and the most computing resources are not the outright advantages that were once assumed.
Enter Frugal AI, a technique that promises the use of less data and less compute power while guaranteeing robustness within the intended field of use for a given AI model.It comes as no surprise that business leaders are increasingly assessing if Frugal AI will have a net positive impact on their AI scaling objectives or if it’s a fool’s errand. In this blog series, we will explore Frugal AI from the perspectives of data, design, trust, and sustainability.
It's All in a Name
AI systems are built using logic to implement an algorithm and data used to train the algorithmic system. Simply put, Frugal AI is training AI systems with little resources. For AI, the need for frugality is usually triggered when there is little data available to begin with, or to align to internal sustainability (and cost efficiency) objectives. To move the dial on AI frugality, input frugality and/or learning frugality can be adopted.
With input frugality, which can be motivated by resource constraints and by privacy constraints, focus is placed on the data and using fewer training datasets or fewer features than needed to achieve the prediction quality achievable in a non-frugal setting. With learning frugality, emphasis is placed on costs associated with the learning process, specifically computational and memory resources. Learning process frugality is primarily motivated by resource constraints, including limited computational power and limited battery capacity
For companies with small datasets and a mandate to move beyond experimentation, Frugal AI promises to be a way to overcome this challenge. Conversely, companies with large datasets may leverage Frugal AI to reign in AI scaling costs, lower their carbon footprint, and align to internal mandates. Regardless of the motivation, AI models trained with smaller datasets have the potential to reduce:
- Compute resource requirements and needs
- Storage infrastructure and data collection/processing costs
- Energy costs associated with training and operationalizing AI systems
Frugal by Design: Why Focus on the Data and Not the Code?
For many years, the conventional approach in AI was to acquire a dataset and expend the majority of efforts perfecting the code. Improvements in AI and ML code have been ongoing for many decades, with a breakthrough achieved in 1989 when Carnegie Mellon University pioneered the use of neural networks to steer autonomous vehicles.
Today, in 2022, the code (i.e., neural network architecture) needed to process and gain insight from the millions of data points that is often available to organizations is mature and — whilst achieving Frugal AI by design involves more than just achieving efficiency in the data — focusing on the data is a good starting point.
To test this hypothesis (whether the most the most efficiency can be gained from engineering the code and not the data), Andrew Ng (a computer scientist and entrepreneur focusing on ML and AI, renowned for spearheading the formation of Google Brain and co-founding Coursera) conducted multiple experiments to measure where the most efficiency was gained.
Results indicated that focusing on the data and engineering a small, near perfect dataset for defect detection yielded a 16.9% improvement in accuracy versus 0% improvement in accuracy when focusing on the code. While Frugal AI doesn’t hinge on data alone, focusing on the data first is a concrete step towards achieving this outcome.
Now one could then wonder: Is this obsession of limited data not in contradiction with Responsible AI which has encouraged us to be very wary of small — and possibly biased — datasets? There is of course no wrong and right answers, but it does reinforce a conviction: Data is only as good as the business acumen supporting its reading, and the conviction that large quantities of data is no guarantee for bias avoidance. Going for a Frugal AI approach, which some would also like to name pragmatic, will however require further reinforcement of the integration of business understanding and cautious data analysis at all stages of the AI development process.
Looking Forward
Like a beacon of hope, Frugal AI has been heralded as a way to alleviate the perceived need for large datasets to scale AI but also to decrease the carbon exhaust that is generated from training models. Training AI systems on small, well-engineered datasets is counterintuitive to existing AI and ML norms, and robustness within the intended field of use for the model is a core component to building trust in AI systems.
In this article, we took a back-to-basics look at one aspect of Frugal AI. In the next article, we will focus on Frugal AI by design and what the implications are for AI systems, trust, and sustainability. This blog post should already be an encouragement for all practitioners to ask themselves this simple question before pressing the train button: Why am I using such a large dataset? Is this level of computation and engineering of my model needed? And even if modern elastic stacks have significantly lowered the barrier to technology, is this aligned with the business objective I am trying to reach?
As Frugal AI principles continue to be adopted and become more mainstream, and the tipping point when Frugal AI and Frugal AI by design is no longer a practice for innovators and visionaries, perhaps the reconciliation of corporate ambitions for sustainability and advanced innovation through AI will be accomplished.