The Role of the Software Architect

An architect's role is to provide technical vision, coordinate direction across teams/components, and counterbalance short term thinking.

— Karl Isenberg (@KarlKFI) July 7, 2017

What's in a Name?

Collaboration improves when the roles of individual team members are clearly defined and well understood.

— Tammy Erickson April 05, 2012

I recently switched roles, from distributed systems engineer to distributed systems architect. I've been doing professional software development for about ten years. I remember telling my boss five years ago that my five year plan was to be a software architect. Guess that worked out! (Now what!?)

Looking back, it's obvious that I'm much better equipped for the position than I was five years ago. But the role itself also seems to have changed over time, as the tech world has evolved to integrate agile tendencies. So as I approached the new position, I found myself needing to repeatedly answer the same questions. Here are some of my notes.

What is a "Software Architect"?

An architect is an engineer who empowers other engineers as a force multiplier.

An architect focuses on the big picture of the system as a holistic product, rather than owning individual components.

An architect drives consensus between tech leads and product managers, rather than making or dictating critical decisions.

An architect promotes flexibility by identifying potential interface points and presenting engineers with options to reduce the cost of change, balancing short-term tactics and long-term strategy.

An architect is a specialist with strengths in diagramming, documenting, communicating, information architecture, and system design.

What is a "Distributed Systems Architect"?

The title itself doesn't seem to have a commonly accepted definition. So let's build one from similar titles.

A systems architect defines the architecture of computerized systems to fulfill specified requirements. Such definitions include the design and specification of components, component interactions, interfaces, technologies, resources, and dependencies.

— Wikipedia - Systems architect

A distributed system is a model in which components located on networked computers communicate and coordinate their actions by passing messages to achieve a common goal. Significant characteristics of distributed systems include: concurrency of components, lack of a global clock, and independent failure of components.

— Wikipedia - Distributed computing

Therefore, a distributed systems architect is a systems architect who defines the architecture of distributed systems, with special focus on communication and coordination between components.

Increasing Complexity

Whether you call yourself a software architect, a systems architect, or a distributed systems architect, the computing landscape continues to evolve with or without you. A few things specifically have changed in the last decade or two that make these roles harder than they used to be.

1) The demand for high availability and intolerance of down time has caused components that used to be simple to be replaced by distributed systems. Likewise, previously distributed systems became systems of systems. With multi-layer systems it's no longer appropriate to rely on legacy architectural design patterns, like a single API layer or a single communication bus.

2) The demand for agility and larger development teams has mutated service-oriented architecture into microservice architecture. This mode of development pushes even more glue code out of the individual components and into the platform or infrastructure layers, increasing their complexity. Deploying and managing a sufficiently large system of microservices increasingly requires a container or application platform above and beyond the scope of modern infrastructure platforms.

3) The demand for hyperscale (the ability to quickly scale up and down without re-architecting) often calls for the architect to design a dynamic system that considers dramatic change over time normal to system operation. The ability to survive a flash crowd of users (aka the slashdot effect) caused by social media is increasingly required by even the most mundane of business domains. Preparation for this and similar phenomenon happens largely at the system level.

4) The demand for open source software creates proprietary systems that aren't completely controlled by their developers. Unlike proprietary components, where one company is in charge of development, open source components can be both easier to change (with a fork or local modification) and harder to persist change (merging changes upstream often requires politics, persuasion, and modification). Partial control of the architecture of a system is nothing new, but partial control over individual components (specifically their APIs and communication patterns) increases demand for intermediary translation, proxy, and caching services. These ancillary components are often tightly coupled with one component to allow loose coupling with another, but tight coupling in a microservice environment often requires new deployment and management patterns, like sidecars and pods.

Musings

Not all software engineers will or should aspire to be software architects. Architecture is a specialization of the engineering track for people who enjoy abstract thinking, are good at and enjoy documentation and diagramming, have the ability to be political and drive consensus, and have demonstrated a solid grasp on prior art and the competitive landscape.

An architect may or may not also be an engineering manager or tech lead, depending on how much focus on architecture is required or desired. Since being hands off of code tends to erode peer confidence and personal skill set, it is desirable that an architect continue to be partly involved in programming efforts, either through tool, component, or ecosystem development.

The bigger an architecture team gets, the more they tend to lose touch with the every day life of the engineers they're supposed to be supporting. However, the more engineers an architect has to support, the less time they will have to stay hands on with code. So a balance must be struck such that architects retain respect and skill yet also spend enough time on architecture to actually provide the benefit of a higher level perspective. It's a fine balance.

Ontological Evolution

The idea of a distinct architect role has in recent past become somewhat contentious among software engineering agile practitioners. The more senior a thought worker, the less they want to be told what to do. But at the same time, modern development patterns are pushing complexity out of individual components with well defined teams into the between space. Plus each component may have a life of its own: independent release cycles, independent versioning, multiple interface/protocol paradigms, independent motivations, and external contributors. Someone has to align these components and the teams that develop them, to unify them into a single cohesive system. So just as engineering practice has evolved, so must architecture practice evolve from controlling and managing technical vision to leading the direction of its evolution. And since collaboration is easier with clearly defined roles, hopefully this new definition can help point the way.


(Revised on 2017-07-13 to expound on the increasing complexity of systems and the increasing demand for architectural design.)