“From Generation to Supply: How AI is Transforming the Energy System” is a six-part series on the many ways in which AI is helping to transform the energy sector at every stage of the generation, transmission and distribution, system operation, supply, and regulation cycle. This is Part 3. The next part will appear in May, 2023.
No electricity for an hour — the horror! Have you ever wondered how many essential services can be disrupted if there’s a blackout with no backup available? Electric train journeys disrupted, urgent surgeries halted midway through, no access to data centers or airports, communication completely shut down: that’s just the beginning.
For a whole host of daily activities, a continuous supply of electricity is a must-have to keep our modern communities operational, productive, and communicative. And it falls to electricity system operators (ESOs) to guarantee that whenever and wherever we need electricity, we’ll have it.
Among the ESO’s many functions, it works to match the electricity supply with the total system demand on a second-by-second basis. They have a diverse range of financial and physical tools to make the adjustments required to keep the electricity system running and also to react in case of an unexpected failure.
Keeping the system operating 24/7 is not the only challenge ESOs have to deal with. ESOs have to maintain a low electricity infrastructure cost for consumers (affordability), support a continuous and efficient system operation (reliability), and run everything with the lowest carbon footprint possible (sustainability). The need to use large amounts of data and automation across their daily operations is not a new concern for ESOs, but recent changes in the electricity system have pushed them to rethink their operations to accommodate a new, decentralized, and sustainable system.
What Are Electricity System Operators?
Before we dive into the use cases, let's first clarify what electricity system operators are and how they differ from transmission network operators (TNOs). While TNOs are responsible for the physical high-voltage infrastructure of the power grid, such as the transmission lines and substations, ESOs focus on the real-time management of the electricity market.
ESOs use a variety of technologies, such as SCADA (Supervisory Control and Data Acquisition) systems, EMS (Energy Management Systems), OMS (Outage Management System), and DMS (Distribution Management Systems), to monitor and control the flow of electricity in real-time. They also use market mechanisms, such as energy pricing and demand response programs, to balance supply and demand and ensure grid stability.
In today's energy landscape, ESOs face many challenges that make their operations more complex and uncertain. For example, the increasing penetration of renewable energy sources, such as wind and solar, makes it harder to predict and manage energy supply. The rise of distributed energy resources, such as rooftop solar and battery storage, creates new sources of demand and supply that are harder to monitor and control. And the fragmentation of energy markets and regulatory frameworks across countries and regions makes it harder to coordinate and optimize the use of resources.
AI in Electricity System Operators
To overcome these challenges, ESOs are modernizing their operations. In this section, we’ll focus on three main ESO activities: forecasting supply and demand for electricity, coordinating system-wide balancing mechanisms, and designing new market products to increase efficiency.
Forecasting Generation, Load, and Outages
When it comes to predictions, accuracy is the main goal for ESOs. The closer ESOs are to the total system’s electricity demand in a forward period, the cheaper they can fulfill any generation gaps required to satisfy the demand. And in a dynamic system, continuous predictions are necessary to adjust the system operation at the most granular level. ESOs need to deal with changes all the time, some of them prompted by uncontrollable variables like the weather.
On the generation side, the weather has a direct influence on how much electricity the system will generate from natural sources like wind and solar. More advanced sensors and continuous monitoring of existing assets with cameras are giving new and more granular data inputs for modelers to evolve their prediction capabilities. This can be seen, for example, in the transition from statistical analysis to more robust ML approaches to automate cloud identification and movement, and to better predict solar power output.
On the demand side, temperature is also an important factor in calculating how much electricity consumers will require, either to keep them warm in cold periods or to cool down if it is too hot. And though the fast expansion of new electric-powered technologies like heat pumps or electric vehicles sounds like a low-risk solution for the grid at a granular level, when aggregated in large quantities, it increases the chance of equipment failure if not accounted for properly in power scheduling. Different ML techniques can support ESOs in identifying the existence of these pieces of equipment based on customer smart meter data, whether they are consuming or generating electricity, and can consequently help them improve load forecasts and grid operation plans.
Even predicting the impact of natural disaster events like hurricanes or heavy snowfall can provide an important boost to ESOs. Simulating potential disruptions to the network supports decision-making and the development of rapid response plans to maintain the electricity supply of essential services during extreme periods. Overall, AI can bring significant benefits to ESOs by improving load and generation forecasting, which is essential for effective system planning, operation and maintenance.
System-Wide Balancing With AI
ESOs’ responsibilities go beyond the medium and long-term forecasting of supply and demand. In addition to the electricity flow, the grid requires many other services to keep its reliability and maintain smooth operation, normally described as “ancillary services.” One example is controlling grid voltage and frequency levels to transport power over long distances without damaging any equipment connected to the grid, all while ensuring that the energy is safely accessible to low-voltage consumers.
But as the task of keeping the grid stable on a second-by-second basis grows more complex, ESOs are being challenged to push the digitization of their operations further. Part of this modernization is leveraging decentralized assets to support the grid in maintaining stable levels of frequency and voltage. ESOs have developed, for example, a machine learning-based strategy that successfully uses multiple heating-controlled loads across the distribution network to decrease voltage violations without requiring active communication.
And it’s not only demand-side equipment that can help provide ancillary services to the grid. Edge computing data from inverters in solar plants with neural network algorithms can be a cheaper alternative for providing reactive power to the grid and supporting voltage control, even overnight, with successful trials happening in several countries.
Optimal Market Design With AI
The energy transition has also pushed ESOs to create new markets that, until recently, had little room for alternative or non-traditional service providers. In addition to balancing services, ESOs need to design and operate the electricity market, promoting a competitive environment for agents that also incentivizes the efficient and sustainable use of energy resources. And a smart operation with a high level of automation is essential for integrating non-traditional agents like households into these new markets, and for benefiting from their flexibility without making it too complex for them to participate.
One progressive solution modeling the impact of these new agents in the electricity system is to build digital twins of the grid. With the grid replica, ESOs can run simulations combined with other digital twins, like electric vehicle models, to better identify the interactions of these new technologies with the grid and harness the opportunities provided by them. Such experiments can also be useful for operating the system with the lowest carbon footprint possible. ESOs have the possibility to simulate different pricing formats and optimal dispatch models that take into account carbon emission factors when designing new markets. They can even use ML models to forecast the grid carbon intensity and support industries in planning ahead for their production.
Moving ESOs to Everyday AI
For players like ESOs that have data embedded into their daily operations, adapting to this new decentralized, dynamic, and sustainable electricity system should be an easy task, right? The reality is that moving from an offline spreadsheet-based working culture to a collaborative online and agile one challenges even those who are more used to dealing with data.
The growing system complexity will require data literacy to go beyond the walls of centralized data teams. ESOs have engineers and market experts at the core of their workforce — enabling them in advanced analytics and data management does not necessarily need drastic changes. Empowering these domain experts to become citizen data scientists without depending only on hard coding to solve problems, all while boosting centres of excellence (CoEs) to collaborate with domain experts, is a surefire way for ESOs to accelerate their transition into the new electricity system.
A platform like Dataiku can support ESOs to streamline their operations — not only for workforce efficiency, but in other ways as well. Whether firms want to build digital twins to simulate new markets or enhance existing prediction models, integrating AI into everyday operations requires a platform that allows for data collection, processing, and real-time decision-making. What’s more, having a centralized place for connecting to multiple data sources enables ESOs to respond more quickly and effectively to changing conditions, allowing them to better manage their operations’ risks.
A Central Piece of the Puzzle
Within the European Union alone, the strategy for moving towards a digitalized energy system includes solar panels in all commercial and public buildings by 2027, 10 million heat pumps in the next five years, and 30 million EVs by 2030. All of this major transformation will need to connect to a grid that, today, relies on aging infrastructure (33% of it was built over 40 years ago). And this is a common situation across many other regions across the globe.
ESOs are central to enabling energy system decarbonization. Accelerating their digital transformation is one of the best routes to make grid operations and new markets more agile and inclusive of demand-side technologies. These will allow for a more cost-effective transition and support a faster uptake of greener technologies like renewable generation.
In our next installment of From Generation to Supply, we’ll explore the role of energy suppliers in facilitating customers’ active role in energy system transformation.