End-user computing applications (think Excel spreadsheets) have long been a vital element of operations in heavily regulated industries, such as financial services, enabling analysts and other users to manipulate data quickly for financial calculations, reporting, operational checks, and more. However, what doesn’t get as much coverage is the significant amount of risk and opacity associated with EUC, such as:
- Manual errors (i.e., leading to misstated financial statements and, potentially, operational impacts)
- Single points of failure (i.e., spreadsheets aren’t productionized, so if a laptop crashes or someone leaves the company, that work is missing)
- Data redundancy (due to data residing in several desktops and in several formats)
- Limitations associated with a lot of data (when a spreadsheet hits a billion rows, it will likely stop working)
- Lack of recovery and audit capabilities
- Overall regulatory and compliance challenges, with increased scrutiny of regulators on EUC in critical processes
This article will break down the value of moving away from EUC and replacing it with an end-to-end platform like Dataiku, so IT executives (for whom reducing shadow IT to better manage IT landscapes over time and lower EUC risks is a key priority) can ensure that analysts and other users can operate in a governed environment for their data, analytics, and — eventually — machine learning and AI.
Let's Back Up: Reasons to Avoid Hot Water
In the U.S., the Office of the Comptroller of the Currency (OCC) charters, regulates, and supervises all national banks. When banks have persistent and pervasive risk management and data governance issues, the OCC will indeed step in to course correct (and, in some instances, issue fines for millions of dollars). If that’s not enough motivation, CIOs and IT executives are keen to get assets and information off of personal devices (EUCs) and ensure they are put somewhere that can be managed, viewed, and audited by the company so it can become a “protected asset” — or otherwise risk having sensitive information on unprotected devices.
From Theory to Practice: Making the Switch Happen
So, if your company’s bespoke production calculations are being done on a spreadsheet on someone’s laptop or there aren’t any governance guardrails applied to processes that touch your customer data, it might be time for EUC reduction in favor of an end-to-end platform like Dataiku. Here’s how:
1. Easily Move Workflows Into Dataiku
While spreadsheet-based tools are mostly cloud based, there are no pushdown capabilities to leverage other compute options when dealing with operations on large volumes of data pulled from various sources. It’s also difficult to connect to different types of sources, such as API services.
In Dataiku, analysts can easily:
- Connect to permissioned data sources (self service)
- Explore, prepare, and enrich large datasets
- Analyze data using intuitive visual tools (and, over time, scale up to machine learning)
- Communicate results in a dashboard
Dataiku is structured as a visual pipeline for performing calculations and other operations on data, enabling users to view the input data, operation, and results by clicking on each step in the flow. Dataiku also provides data profiling features to identify and automate data quality checks to spot human errors, which are a primary cause of spreadsheet problems. When it comes time for teams to move their workflows, Dataiku offers several step-by-step guides, such as the Excel-to-Dataiku Playbook and the Excel-to-Dataiku Quick Start.
2. A Governance Layer, Productionized
Want to avoid shadow IT? Given that EUC applications are generally used outside of core IT systems and across the enterprise, an end-to-end platform like Dataiku enables teams to ensure a governance layer by:
- Producing intentionally auditable trails on all changes (including where the data originally came from, what happened to it, and where it’s being used)
- Keeping documentation of data sources with sensitive or proprietary information
- Viewing what data is used for a given project and who “owns” what data
- Easily explaining decisions with historical workflow and period data
3. Keep Flexibility Where It Matters Most
To avoid any drastic pitfalls associated with using EUC applications like spreadsheets, teams should do the majority of their work in Dataiku. However, spreadsheets can be used on last-mile analysis (in unison with Dataiku) for tweaks and modifications in order to maintain flexibility.
Let’s take a practical example: A financial analyst needs to compute (every month) the month-on-month reconciliation and 12-month rolling metrics for the perimeter they supervise. All data checks, data pipeline computations, and KPIs computation can be automated with full ownership and control by the financial analyst. But, in parallel, the need to retain the capacity to do quick, ad-hoc specific metrics computation — a percentage of the evolution over the last three months versus the three previous ones — should remain. These calculations can both be tackled leveraging the agile analytics environment offered by Dataiku or doing quick last mile computations in connected spreadsheets.
Overall, the goal for IT executives will be ensuring governance associated data and analytics (and eventually AI) scaling and Dataiku offers the one-stop-shop to help enable that in practice — governance built into every project, security control measures, and fine-grained access rights — all to help drive EUC remediation.