Financial forecasting is one of the primary and also most challenging tasks of finance teams. Using existing historical data, like balance sheets, income statements, or business data sources, financial analysts attempt to predict future outcomes.
These predictions are then used as drivers for key business decisions and business planning. In other words, they ultimately can have huge impacts on the overall company performance. Yet while financial forecasting models greatly influence strategy and policy, they’re not always as accurate as they could be. In fact, nearly 40% of CFOs feel their forecasts are not accurate and that the process takes too much time.
What is more, significant time is spent circulating results for approval and feedback, often involving significant summary efforts that go beyond what is achievable via dashboards. Machine learning (ML) and Generative AI offer powerful opportunities to improve.
Financial Forecasting Methods: Challenges & Pitfalls
Some of the most pervasive challenges in efficiently delivering accurate financial forecasts include friction with finding data and process bottlenecks. These can be seemingly small roadblocks like getting access to the right data and ensuring it's in the proper format. But it also includes larger, more systemic problems. A big one is understanding whether the data is even accurate and fit for purpose (i.e., data quality). Getting buy-in and sign-off on results is another.
Given all the potential points of resistance along the way, unfortunately, financial forecasting is sometimes more art than science. These day-to-day challenges take up precious time and resources and don't leave much room for leveraging more advanced techniques. ML holds immense potential for improved accuracy, while Generative AI offers powerful opportunities to streamline result circulation and management review. But teams need the time and tools to implement it.
Many FP&A teams aspire to improve, but they’re caught between today's issues and asks and the necessity of future improvement. Nine times out of 10, the urgency of today's issues and asks wins out over long-term advancement.
What Does Better Forecasting Look Like?
Even when teams do have time to focus on improvement, they might not have a clear path forward. CFOs feel their forecasts are not particularly accurate, as we saw above. But in that same McKinsey study, companies also report being "generally satisfied" with their forecasting processes. Where does that leave finance teams regarding what "good" looks like and the ideal processes to get there?
Financial forecasting software and technology in general, as always, is never a magic bullet. But the right tools can provide useful guardrails for introducing more efficiency and consistency into financial modeling processes, and make leveraging traditional techniques seamless alongside the newer capabilities of ML and large language models (LLMs).
A Financial Forecasting Template From Dataiku
Today's most advanced finance teams are transforming their forecasting processes by streamlining the data process, improving accuracy through ML, and leveraging Generative AI to make the results interactive at scale. With Dataiku, more precise, less costly, and more easily explained forecasts are within reach.
The financial forecasting solution from Dataiku is a plug-and-play blueprint for more efficient and more accurate reports, with an optional Generative AI module that can be incorporated from the start or added-on later. The ready-to-use template means that finance teams can get started in days and see results in weeks, not months. The optional Generative AI module makes sharing those financial forecasts easy for FP&A teams with LLM-powered report generation. Highlights of the Dataiku Solution include:
- The ability to enrich your forecasting approach by blending ML and enhancing existing techniques, improving results while reducing effort.
- Flexible, driver-based evaluation to quickly test and select potential financial drivers from internal (e.g., headcount) or external (e.g., inflation) sources.
- ML predictions that are easy for finance teams and people on the business side to understand.
- Powerful visual analytics that clearly reveal historical and future forecasting accuracy.
- Using the existing structures of the Solution, such as region and values, an LLM creates a structured report in text form with commentary on forecasts and key drivers.
- The financial analyst is able to make direct manual edits to the text, or leverage the LLM via conversational queries to make adjustments and enhancements.
- Once finalized, the resulting text can be sent from within Dataiku to the intended recipients.
How Does the Financial Forecasting Solution Work?
We interviewed the creators of the financial forecasting solution from Dataiku. John McCambridge, Business Solution Lead for FSI and Finance, and Lea Senequier, Data Scientist, dive into the details below.
Beyond the financial forecasting solution specifically, what are Dataiku Solutions in general?
[JM] Dataiku Solutions are pre-packaged projects that speed up the delivery of data, analytics, and AI use cases. They are industry- and/or business function-specific and cover a range of topics.
Dataiku Solutions can also be called a foundation, a base, or a starting point to meet a business objective. They consist of outlines that will guide the user to the completion of a data, analytics, or AI use case.
Currently, we have 34 Solutions in our catalog, both industry-specific and transversal. The most recent release is the Financial Forecasting Solution, which can be used transversely across industries in finance teams.
Of all the data and analytics use cases out there, what made the team decide to focus on financial forecasting?
[JM] It is not a secret that financial forecasting processes play a central role at every company. They support organizations worldwide in making appropriate cost management and investment decisions. The financial forecasting solution is a natural response to the general transition toward more automated and accurate analytics.
Even though Generative AI and LLMs are all the rage, most organizations still struggle with basic data processes. Dataiku Solutions, including the financial forecasting solution, can help accelerate progress on these baseline use cases to pave the way for more innovative applications down the road. Then, when they’re ready to advance, they have the Generative AI add-on.
You have extensive experience in the finance space. What are some of the biggest challenges you've personally seen play out with financial forecasting?
[JM] You already saw that 40% of CFOs reported that they feel their forecasts are not accurate and that the process takes too much time. To be more specific, we can highlight three main challenges that current methods of financial forecasting exhibit:
- They are either time- and effort-intensive, or fast but lack business logic and insight.
- Scattered and unstructured data sources — this problem also contributes to the time needed to perform financial analysis.
- The time needed to invest in building new or improved forecasting approaches is in very short supply. And spending the time required is impossible if there is risk of under-delivery.
What did you learn from a data science perspective while developing the solution?
[LS] To address the challenge of limited input data, typically comprising fewer than 100 data points, we did some research and adopted a methodology inspired by a paper authored by Microsoft employees.
Originally, this methodology was implemented to forecast revenues across different geographic regions or product types. It involved utilizing simple time series models to generate forecasts, which were subsequently incorporated into a regression model.
What are some of the coolest features of the solution?
[LS] One of the coolest features of the solution is the ability to incorporate an unlimited number of drivers. Drivers are variables that can have a significant impact on the target variable being forecasted. These drivers can be specific to each time series or apply to all categories. They can be either company-specific or macroeconomic data chosen by the user in a numerical format.
[JM] Additionally, it is built so that the initial investment in running the core solution allows you to easily add-on the Generative AI module as desired and immediately start seeing additional efficiency and explainability gains.
How can someone get started with using the solution?
[LS] To get started with the solution, users simply need to download and install the required code environment. They should then follow the data model schema provided in the wiki to prepare their input data. Once that is done, they can easily create an application instance and generate results with just a few clicks.
[JM] Once the core solution is up and running, the team can grab the Generative AI module from the same source and integrate it using the guide provided.
Key Benefits of the Financial Forecasting Solution
Finance teams can use this customizable template to quickly improve and enhance financial forecasts. By blending machine learning and enhancing existing techniques, teams can improve results while reducing effort, all without requiring the involvement of highly technical data profiles.
