How many times have you heard something like this? “We’re an Agile organization. We have two-week sprints!”
I’ve heard it so many times, I don’t even react to the claim. It is usually a case where the proof of the pudding is in the eating.
Truthfully, I really like the intentions behind the Agile movement. Whenever I am faced with projects that are in real trouble, I implement Agile-esque tactics without thinking about it within the context of any particular method or framework: I remove all unnecessary overhead, streamline all technical communication and closely involve stakeholders, co-locate teams into a “war” room, focus on delivery of proven incremental value, and so forth.
Arguably the most significant point of the Agile manifesto was, in my opinion, the idea to build software in small, functional, incremental pieces. When compared to old-school waterfall-ish approaches where one is presented with an enormous scope captured as requirements that nobody fully understands, or a massively engineered architecture that is thrust upon a development team, it seems clear that the incremental approach is a more realistic way to solve problems.
With that said, I’ve worked in enough so-called Agile shops to have noticed a common challenge that frequently seems to present itself.
This challenge has to do with understanding what ‘iterative’ really means, and what functionality actually makes sense to deliver in an iterative manner.
Vertical Iterations
Let’s imagine a simple example. Say a business wants to build a stereotypical software product that processes orders, tracks inventory, and handles shipping (I will refer to these as “functional areas”.) When a product owner and a business analyst get together to lay out the project dimensions, they might decide to build inventory management first, followed by order processing, then shipping. This seems to make sense, simply because there is a relatively clear conceptual boundary that even a layperson could draw around each functional area. Literally, it could be thought of in terms of the diagram below:
These conceptual boundaries, in turn, directly promote a rightfully attractive idea about how to plan the actual work. It would seem clear that three parallel teams would work in this case. So, our product owner and business analyst decide to staff some software teams, sketch out a backlog, and get started. I think of this method as a “vertical” approach, because the functional areas are implemented to their highest form in independent and vertical “slices”. The iterations cover “vertical” functionality in chunks, which I mentally imagine like so:
This is a very common approach, and it is frequently viewed as optimal by software management and those in the planning domain. After all, look at the parallelization being achieved! It’s like a free lunch! Fred Brooks, eat your heart out! As an added measure, it is increasingly common that the teams are located in a different geographical areas, purely for cost-saving purposes.
On the surface, this all sounds well and good.
However, reality on the ground can be very different. There is often no holistic architecture from which to work. The implementation teams often don’t communicate effectively. Concepts that should be common across functional areas are often poorly understood at an overall level. Contracts between functionality that should be carefully reviewed and managed are instead produced at the last minute and left to evolve with little oversight and little to no change management. All this results in preventable bugs that ripple across functional bounds, thus impacting the stability of the entire system.
Certainly these three functional areas could be built as something like compositions of microservices, avoiding challenges related to shared domain models and other monolithic infrastructure. If each team implements sensible oversight policies, and are self-aware enough to coordinate with each other early and often, such a development model can certainly be successful.
In any case, the involved development teams will usually end up specializing in each of the functional areas. This is what I commonly refer to as “development silos,” and it is a common enough situation to have its own informal law. A silo-like ecosystem can be extremely risky to a business. A technical person could not move from working on the inventory system to working on the shipping system without a steep learning curve. Suppose a competitor hires away most of the engineers and analysts on the order processing team. Now the company has nobody left who understands either the original design, or the problem space in general. Ouch.
There is another way to approach the same scenario.
Horizontal Iterations
Given the same set of functional areas, a technical person might point out that there may be a common domain model that underlies all three functional areas: the order processing domain would have to either integrate with, or at least share part of a model with the inventory domain, which in turn must have a common (or derivable) definition of what makes a ship-able entity for use by the shipping system.
This point of view is more horizontal, rather than vertical: we imagine building the system horizontally – meaning, across the domains – rather than building each domain in relative isolation. Within a single team, build the absolute minimum functionality to process an order, then build the minimum necessary to calculate the order processing system’s impact on inventory, followed by a rudimentary shipping system. I think of such an approach in terms of cross-cutting iterations, successively moving to higher levels of abstraction as system functionality becomes more elaborate and functionally mature. I picture it like this:
From the diagram, each iteration is essentially augmenting the bare-MVP functionality of the previous iteration. Perhaps it takes a few initial iterations to really get the functionality moving; perhaps the system does not demo very well in early iterations. Yet as the functionality evolves, something very interesting is happening: the implementation team is learning holistically! The team learns that an order processed means a possible change in inventory; they understand that out-of-stock should possibly prevent ordering. The team is also learning the key concepts and limitations of the business domain. In other words, the team is building a broad understanding of the problem space, and the dependencies of the system functions. It also means the team members have more portable skills and are better prepared to understand system-wide ramifications of their own decisions.
I believe this natural evolution of teams deepening their understanding of their solution domain while building the solution, is a very important benefit of an iterative model. This aspect seems to be often overlooked.
In fairness, the horizontal approach is not a silver bullet and is certainly not without its challenges. I sometimes find that the horizontal approach seems difficult for deadline-driven or Type A personalities, possibly because they feel more confident in their projected outcome when a particular function is complete to a “sale-able” degree. I’ve done plenty of damage control on projects where the leadership was uncomfortable (to the point of passionate objection) with a horizontal approach, simply because they tend to dangerously equate the ability to demonstrate the software with the actual software product itself. Hybrid models can be explored, for example, involving horizontal iterations within slices, and rotations of development expertise between the domains.
Planning the actual content of iterations in a sensible way can be a deceptively difficult problem. Whether a given approach is “right” or “wrong” is sometimes only clear in hindsight. While every organization differs in their ability to choose and execute such models, I usually find the common denominator in the selection of a model that the business alignment and development culture can both support.