Data architects have a tendency to feel like unicorns: Somehow they can manipulate data storage and computation structures like putty and also keep business objectives in mind.
Data architects need to be able to do it all: Manipulate data storage and computation structures, and prepare backups for system failures. Data architects can be said to act as data physicians, diagnosing both the physical maladies in the data pipeline and the organizational habits that are detrimental to the health of the system. Yet their role varies drastically depending on the organization and industry they’re embedded into. We wanted to explore what the role of a data architect might look like and how their work drives data initiatives.
Business & Technical Specialties
Architects are often embedded in the IT team, but are multidisciplinary agents. They balance on the fine line between engineering acumen and business savvy, ensuring that communication, expectations, and KPIs between these teams are as unified as possible.
Grant Case, Senior Analytics Architect (and one of the authors of our latest architecture guidebook), explains that data architects are required to wear many hats: "Data architects can't just focus on optimizing the technology to the exclusion of all else. If they're not basing plans on business concerns in addition to the scale and cost constraints, then they're not creating a truly robust data architecture." Some of the different processes that are coordinated and executed by data architects include:
- ROI Calculations — Evaluating which architectural decisions will generate the most business value in the short and long term.
- Translate business needs to technological solutions — Communication between technical and business teams is critical to the design of successful and empowering data architecture.
- Support change management — Adjusting the architecture will inevitably create short-term pain points and operational inefficiencies as users adjust to new processes. Ensuring that educational paths are clear and that stakeholders understand and support the the shifts is a necessary part of successful data architecture. Even the best architecture in the world is ineffective if users don’t take advantage of it.
- Evaluate competing suppliers — Once an architectural decision has been made, competing suppliers and even open-source offerings must be evaluated in order to select the best fit.
- Adjust operations as the organization scales — Keeping pace with the evolution of the organization and evolving the architecture as needs and scale shift.
- Monitor backups and fault tolerance — Attempting to prevent single points of failure and imbue a data architecture with resilience is critical as organizations scale and more end-users depend on them.
- Design and build the best architecture possible — This may seem like a summary step, but so much of a data architect’s role also involves the organizational changes that come with new architectures. Executing well-laid designs is important, but so is clearly documenting the work and processes required for maintenance.
Data Architecture Is Always Changing
While we often compare data architecture to physical architecture, in this way building a data architecture is not like building a house. Instead of creating a masterpiece and letting it mature, data architectures require near-constant adjustments and revisions to keep pace with the organizations they support.
Joel Belafa, a Dataiku Data Architect, says that “you should always be prepared to rethink your data architecture at any time.” In this way, data architects must always think iteratively; there is no finished product when it comes to architecture, so killing your darlings is part of the job description. Just as the same way that education and a changing competitive landscape evolve a business, data architects are never at a standstill; they’re always growing and evolving their architecture (and skillset) to improve organization performance.