Tuesday, December 13, 2011

Discipline Envy: Building software worlds with blueprints?!


In many engineering disciplines, civil engineering is a perfect example,  implementation is preceded by a distinct design phase which produces a defined architecture; a blueprint for implementation. For all practical purposes it can be said that design is an activity which spawns the architecture and the corresponding implementation. We know that this paradigm works extremely well when building bridges but does it work when constructing software.

I don’t think that in the software world design can be compartmentalized into a phase. The interactivity inherent in software systems creates severe dynamism in requirements. Moreover the swiftness with which tools and technologies evolve do not allow for a Big Design Up Front (BDUF). The fundamental assumption that BDUF makes is that architects and designers can visualize pitfalls without prototyping and some implementation. This unfortunately does not apply to the software development process and to me it looks like a classic case of Discipline Envy.

Ward Cunningham (of Wiki fame) wrote that the programming discipline has always had an identity crisis. It never seems happy to be itself but instead pretends to be the discipline it serves at the moment, or the discipline from which it has drawn the most members. First our discipline wanted to be like mathematics and then we were a science, off on a voyage of discovery with the scientific method at our side. Now we have matured to engineers, liberated by certified processes and writing our handbook.

I think Ward is absolutely right; what we seem to be doing now is borrowing practices from other engineering disciplines to write our process handbooks. Engineering Envy! This may work sometimes but comparing practices in software engineering with those in civil engineering is going to fail most of the times. Instead of looking at established practices, we need to invest into looking at our own patterns and create paradigms which will help us better optimize the age old iron triangle of speed, cost and correctness.

There needs to be a balance struck between the time spent on planning and the time needed to refactor to fix a defect. In most software projects there is always a dearth of requirements which will make any analysis or design phase stretch out and lead to an anti-pattern called "analysis paralysis", which I have already spoken about in an earlier post. With the inherent dynamism, the cost of planning is almost always higher than the cost of fixing, effectively laying waste to the time that was put into planning.  Continuous Deployment, Automatic Updates, Fault Tolerance and related ideas substantially reduce the cost of defects in production so that they become cheaper to fix at run-time than to plan out at the beginning. These concepts are far more suited to the software development environment.

Another undeniable feature of any software development life cycle is change. Customers normally don't really know what exactly they want. Most of the time, given a prototype, they will be able to say definitely what they don’t want. Given this situation if you are making a BDUF, a lot of assumptions are inevitable that later will prove to be false but these are already designed and possibly coded. This brings me to the concept of YAGNI or You Ain't Going to Need It. The idea is to design, code and test only when a feature is absolutely needed and not when it is foreseen. A kind of Just-In-Time approach which eventually reduces a lot of wasted work and time.

Design has to be an iterative activity with no real start and end points stretching from inception all the way up to maintenance. Software design or architecture cannot be completed in a "phase"; design and architecture need to be emergent across the span of the product lifecycle which will in turn create a sustainable development environment.

I am not advocating "cowboy coding" or saying that there is absolutely nothing needed upfront. There definitely has to be an approach and a vision for the development process but it need not be big and grand. Nothing is going to evolve unless it is seeded. All I am questioning is the depth of the design made upfront and saying that it looks that we will be much better off at a point three quarters down the line below:


I sometimes wonder though is this too darwinian for a creationistic world!

No comments: