Discipline Envy: Building software worlds with blueprints?!

In many engineering disciplines, civil engineering is a perfect example,  implementation is preceded by a distinct design phase which produces a defined architecture; a blueprint for implementation. For all practical purposes it can be said that design is an activity which spawns the architecture and the corresponding implementation. We know that this paradigm works extremely well when building bridges but does it work when constructing software.

I don’t think that in the software world design can be compartmentalized into a phase. The interactivity inherent in software systems creates severe dynamism in requirements. Moreover the swiftness with which tools and technologies evolve do not allow for a Big Design Up Front (BDUF). The fundamental assumption that BDUF makes is that architects and designers can visualize pitfalls without prototyping and some implementation. This unfortunately does not apply to the software development process and to me it looks like a classic case of Discipline Envy.

Ward Cunningham (of Wiki fame) wrote that the programming discipline has always had an identity crisis. It never seems happy to be itself but instead pretends to be the discipline it serves at the moment, or the discipline from which it has drawn the most members. First our discipline wanted to be like mathematics and then we were a science, off on a voyage of discovery with the scientific method at our side. Now we have matured to engineers, liberated by certified processes and writing our handbook.

I think Ward is absolutely right; what we seem to be doing now is borrowing practices from other engineering disciplines to write our process handbooks. Engineering Envy! This may work sometimes but comparing practices in software engineering with those in civil engineering is going to fail most of the times. Instead of looking at established practices, we need to invest into looking at our own patterns and create paradigms which will help us better optimize the age old iron triangle of speed, cost and correctness.

There needs to be a balance struck between the time spent on planning and the time needed to refactor to fix a defect. In most software projects there is always a dearth of requirements which will make any analysis or design phase stretch out and lead to an anti-pattern called "analysis paralysis", which I have already spoken about in an earlier post. With the inherent dynamism, the cost of planning is almost always higher than the cost of fixing, effectively laying waste to the time that was put into planning.  Continuous Deployment, Automatic Updates, Fault Tolerance and related ideas substantially reduce the cost of defects in production so that they become cheaper to fix at run-time than to plan out at the beginning. These concepts are far more suited to the software development environment.

Another undeniable feature of any software development life cycle is change. Customers normally don't really know what exactly they want. Most of the time, given a prototype, they will be able to say definitely what they don’t want. Given this situation if you are making a BDUF, a lot of assumptions are inevitable that later will prove to be false but these are already designed and possibly coded. This brings me to the concept of YAGNI or You Ain't Going to Need It. The idea is to design, code and test only when a feature is absolutely needed and not when it is foreseen. A kind of Just-In-Time approach which eventually reduces a lot of wasted work and time.

Design has to be an iterative activity with no real start and end points stretching from inception all the way up to maintenance. Software design or architecture cannot be completed in a "phase"; design and architecture need to be emergent across the span of the product lifecycle which will in turn create a sustainable development environment.

I am not advocating "cowboy coding" or saying that there is absolutely nothing needed upfront. There definitely has to be an approach and a vision for the development process but it need not be big and grand. Nothing is going to evolve unless it is seeded. All I am questioning is the depth of the design made upfront and saying that it looks that we will be much better off at a point three quarters down the line below:

I sometimes wonder though is this too darwinian for a creationistic world!

Mushrooming Clouds!

Mushroom cloud from a hydrogen bomb test

All hierarchical and process-bound organisations are guilty of "mushroom management" at some point or another. Mushroom management is a satirical metaphor for a situation when access to information and decisions are restricted to employees and furthermore when the information is conveyed, it is inaccurate, incomplete and full of hyperbole. It is like being kept in the dark and having the dirty stuff thrown at you. Even if you are mildly ambitious this style of leadership will curtail your creativity and hold you back.

Business today is about change and adapting to it. Nowhere else is this more evident than in the software industry. Free flowing and ubiquitous communication are the corner stones to innovation, good customer service, high retention and change. You would think that with the Internet, email and mobile phones communications would have improved by leaps and bounds. But it turns out this is not really true. If you ask most people what they would like their supervisors or managers to do to make the work environment better, you will hear "communicate better" most of the times. In my opinion this is the case because, email and mobile phones have substantially increased the quantity of communication but they have had a damning effect on the quality. The tools are improving but the content is deteriorating creating a rude deception!

Keeping a tight rein on information has never proved to be a good management strategy. The most famous example of this was the Teamster's strike at UPS in 1997. It is estimated that UPS lost nearly $700 million in revenue apart from the loss of credibility with its customers and employees. Many commentators argue that as part of its negotiations the management held a lot of information close to it's chest. It turned out that this flawed strategy caused many loyal employees to feel betrayed because they simply did not know what was going on. As a consequence they simply decided to leave. The lack of information caused a backlash with anger, resentment and  loss of revenue. 

Functional groups in organizations need to be "high-access" with communication at the center. Good communication will invariably lead to high retention. On the other hand "low-access" groups are characterized by constricted communication flows where people find themselves kept in the dark like mushrooms, stuffed in silos of job descriptions, rankings and the organizational chart. It is but natural that these groups have trouble responding to change and wavering customer needs which is a reality of the modern software producing work environment. The mantra has to be "when in doubt, over communicate!".    

All in all mushroom management in the software product development world is an anti-pattern that will only end up producing mushroom clouds. 

  

  

The Dangerous Chicken

Angry Birds

Chicken is a fairly well modelled conflict pattern in game theory circles. It is a simple game really where each player tries not to yield to the others. The worst possible outcome of the game though occurs when none of the players yields. You would have seen this situation depicted in many movies. Two gang brats with their snappy cars set out to prove something to their cronies or settle a dispute. They get into their cars and a skimpily clad hottie flags them off at warp speed towards each other. The brat who blinks first and veers off the path is the "chicken" while the other one gets the bragging rights and probably the flag bearing hottie as a bonus. If neither of the hooligans swerves of the path, the result is a crash which will probably be deadly producing the worst outcome of the game.

In the software industry especially in project management the game of chicken manifests itself in scheduling and is often referred to as "Schedule Chicken". The condition occurs when teams start committing to unrealistic schedules assuming that other dependent teams are crunching their schedules even more.

Project delivery dates, especially so for new product development, are ridiculously difficult to estimate primarily because:

• Uncertainty is inherent and inevitable in software development processes and products - Ziv’s Uncertainty Principle 
• For a new software system the requirements will not be completely known until after the users have used it - Humphrey’s Requirements Uncertainty Principle 
• It is not possible to  completely specify an interactive system – Wegner’s Lemma
• Ambiguous and changing requirements, combined with evolving tools and technologies make implementation strategies unpredictable.

It might seem like I am stating the obvious but these tenants do not seem to be very well accepted. "The management" is always looking for a "date" for obvious reasons, irrespective of the uncertainty and the ambiguity. They are in most cases pulling in an untenable date that can be sold to the bosses which in turn is sold to their bosses and so on. To arrive at a date there is bound to be a lot of analysis with:

1. Requirements being gathered and formally written 
2. Big designs being made upfront
3. Project estimation being made and
4. Complex interrelated plans being drawn up. 

With all the requirements being far from clear since customers themselves only have a vague idea of the final product, the analysis phase in most cases will turn into an "analysis paralysis". There is absolutely no tangible functional value created with the steps mentioned above. There may be some organizational value but that can almost be equated to bureaucracy! It is not surprising then that analysis paralysis is commonly referred to as an anti-pattern. All the analysis though takes a lot of time and since it is not producing tangible results is, in effect, creating a "value" debt for the project which will always be looming large.

Even if you were to be brutally honest with every dependency analyzed (which is pragmatically impossible), the date you come up with is never going to be accepted because somebody up the hill won't be able to fit it into his presentation. The inevitable advice will be to slash the estimations which will end up making the already shaky estimates absolutely and firmly unattainable. That the dates don't work is now a open secret in every sense of the word. That said, we live in a dysfunctional corporate culture where you are discouraged from being forthright with the messenger always being the first guy to be shot. The value debt and the culture together setup this cult-like delusion where the dates are committed to in unanimity.

Now the question is how long is this charade going to last? Most likely till the the first significant milestone in the project. The next obvious question is who is the guy who is going to blink; who is the chicken? Most likely it is going to be a "not so experienced" project manager or a service vendor. The chicken in most cases is crucified and accused of every incompetence conceivable under the sun. Once the chicken has been labeled an outlier, shot and killed; everyone else readjusts the dates and the game starts over to find the next chicken.

The real problem with Schedule Chicken is that it delays the inevitable, hurting the project schedule and driving up the cost. It prevents the stakeholders from looking at the root cause of the problems and taking corrective action early on.

The bottom line is that the game of chicken will not only disembowel the chicken but will also nuke the project. Schedule Chicken is a dangerous game to play!    

   

Back to the past with policies!

It has been a month since i started consulting for the big IT firm. Day one was just a precursor to month one. A lot of frustration, a lot of complaints but invaluable and important lessons learnt. I am still trying to analyze all the information I have and trying to make sense of some of the absolutely ridiculous policies that seem to be in place.

The first thing that struck me was the restrictions that were in place especially on the IT infrastructure and data movement. All computers are centrally managed to the extent of wallpapers and screen savers. Computers are alloted only with XP, some ultra basic version of Office, the Lotus Notes client, Winzip and Acrobat Reader. There is a blanket ban on all binary downloads from the internet and I am told that acquiring a license for any software from the IT department is nothing short of a nightmare. This essentially means that you have to be content with the list of tools I listed above. The policy seems to be that all activity either managerial or development has to be carried out with the list of tools I mentioned above.

I was already cribbing at this point. This set of policies meant that I had to be content with IE6 which comes bundled with XP. No downloads meant no Firefox which in turn meant I would be deprived of my cherished feature of modern day browsing, tabs! How could they do such a thing to anyone? This amounts to cruelty in by book. It is just absurd. I would have settled for IE7 but even that is not available.

It is obviously not impossible to work only with these tools. It calls for some level of ingenuity and some patience. What I mean is that it is not impossible to make a GANTT chart or a block diagram in Excel but imagine coloring cells to show a time line, it is just tedious and error prone. Now, if it is a 12 Lakhs a year manager / engineer coloring the cells it becomes even more absurd. Wouldn't it be cheaper to give him a copy of Microsoft Project or a similar tool is something I kept asking myself. It would make life simpler when the chart is being made and when the chart is being modified and when the project needs to be tracked. Why put in a lengthy process to obtain the necessary tools which can substantially help improve efficiency especially at the managerial levels is beyond me.

Maybe it is my outlook which causes me to think this way. For example for me my computer is a natural extension of myself, my being. I know her (Sunshine) inside out. She is always has all the tools that i will possibly need. She is always updated and ready to go. Over the years I have learnt the hard way that you are only as efficient as you know your tools. I customize Sunshine to my whims and fancies and this definitely helps me work faster and produce results more effectively. Now at this firm there doesn't even seem to be a concept of one machine per employee. People are alloted and re-alloted machines randomly at a drop a hat. Forget tools and efficiency how does one manage data in such a transient environment. This is another one of those polices that makes no sense to me at all. Maybe there would be some order in the chaos if there were shares on some central server so that data remains at a common place while machines are being shifted but there seem to be no shares as well. Without shares the only legitimate way to move around data is mail and yes I am sure you guessed it by now, mailboxes have a 100MB quota. To complete the viscous circle you need to archive mail on your local hard disk! I have to wonder if these guys really know what they are doing!

The restrictions on the Internet are also stretched to a point which amounts to nothing less than paranoia. Apart from a blanket ban on binary downloads and popular email servers there is also a ban on searching pictures and maps because this activity is thought of as criminal or so says the firewall message. I wonder how the people working at this firm will use wikimapia to do something criminal considering the fact that at induction they have been through extensive background checks including a police verification.

That aside, not having the Internet for simple day to day development can have some seriously adverse effects. Just yesterday, I saw a case when this became a serious issue. These guys have a hardware lab which just got some fancy boards from TI running Linux and the plan was to get some basic tests done like booting up the board and mounting an NFS file system and so on. First some ground rules. The lab is isolated from all networks that means there is no Internet there. There is a single Linux box there with some archaic distribution of Suse and best of all USB and CDROMs are disabled on all machines. The first problem we hit upon was that there was no minicom. That was a easy problem to solve, we just got another laptop running Windows and used Hypeterminal. But then we discovered that the NFS server on the Linux box may be corrupt. Now how do we replace the defunct server. There seems to be no way of doing this and the whole activity is just stuck up for the last two days. This seems like a such a waste. Standing there I felt like I had time traveled to 2000 or something when I was in college and such lack of infrastructure was more common place.

There were more reasons why college memories came up. One more rule that i forgot to mention before is that the fancy board is not allowed outside the lab because it costs about 2000 dollars. In college this rule made sense. Undergrads playing with 2000 dollar boards is a little scary but 10 years on when I am paid twice as much a month to build something of similar complexity, I couldn't help but be amazed at the sheer oversight of the person who came up with such a policy.
The levels of ignorance among the less experienced engineers could be one of the reasons for this policy. The guys in the lab sat on the no minicom issue for two days before we were brought in to solve the problem. The manual said minicom and there was no minicon on the Suse box so the whole thing just stalled. Nobody even remotely thought of using Hyperterminal. I wouldn't trust a 2000 dollar board with these guys even inside the lab with supervision let alone the outside.

But my point is that blanket bans are counter productive in almost all cases. I could have solved these problems out in a couple of hours if I had the right resources with me. Some level of restriction is fine but when it reaches a level when work gets impeded so severely that deadlines are not met then the restrictions and the policies that enforce them need to be reviewed and modified. Scraped if necessary! One solution that comes to me is seniority based access to resources such as the internet and fancy boards instead of a blanket ban. An assumption that senior people will be more responsible and will also be more liable is not all that bad. Some training to everyone concerned wouldn't hurt either.

Again I guess the basic rule remains. It comes down to implementation of the policies on the floor. Any and every policy should be periodically reviewed to check if it is really working as it was designed. The only way is for the policy makers to taste their own medicine.

Talent acquisition and deployment!

I volunteered my services as a consultant to one of the bigger IT firms in Pune beginning about a week ago. My main objective was to look at and learn how these companies operate on a day to day basis. The last week has been very interesting and taught me more of what I should not do and very little of what I should.

Going through the selection process of the company was a breeze. I spoke to an engineer on the phone and then had a one-on-one with a couple of more engineers and the group head of the team that I was to consult for. Nothing out of the ordinary there and nothing I haven't been doing for recruitment at ECM (that is Embedded Computing Machines, the company I co-founded). Once this was done, I was referred to a group called "Talent acquisition". When I walked through the door and read the name of the group, I was rather amused at the title, probably because I was expecting something more common like "Human Resources". I had a fairly productive talk with one of the members of the group and I was warming up to the idea of working here. But little did I know that things were all set to move south from here on.

I was expecting to hear from the Talent Acquisition group within a couple of days with an offer. The couple of days turned out to be a couple weeks with repeated pestering. In such a competitive market where the demand for engineers far exceeds supply and companies are beating each other silly in hiring people, this delay made very little or actually no sense at all. I am still wondering what made them take so long when you would expect that a company of this size and standing to have preset and tested processes which typically should have extremely small turn around times. Finally it looked like the engineering junta hammered their way through and I was invited to start work without an offer letter.

I drove down to their office at the appointed time and started by announcing my presence there. An hour went by and I was still waiting for someone to guide me through the initial formalities. I was predictably upset at this point. A young lady then came out to meet me and told me that my offer letter was still in pipeline and it would take another hour before she could start with me. What got to me was the tone in her voice and her demeanor, there was not even the slightest hint of regret or apology. On the other hand she gave me an impression of complete and absolute power, that somehow I was a slave to her whims and she owned me and my time.

A while later, a member from the engineering team came by and took me across what I later learnt was the "Deployment HR". The sheer apathy in that room overwhelmed me. I met a few more of the young lady's colleagues. Nobody had a clue to why i was there and what they had to do with me. It looked to me like there was no team, just a bunch of disjoint angry people in a room. Angry at what, I have no clue, their boss or simply the nature of their job. Surprisingly though the men were more courteous than the women. A few agonizingly curt comments later, I left my phone number with one of the men who promised he would find out what is to be done with me while I was introduced to the engineers. I did get a call in a couple of minutes and was told I should meet my "handler" (the same lady who thought she owned me).

On the way down, I stopped by and met my handler. She handed me a form which seemed to need a couple of personal details like my address and date of birth, which I filled up. Then to my utter amazement I was told that I need to get this form ratified by a whole bunch of people who were seated across the height of the building, right from reception to administration to the library. I did not know what to say, I was dumbstruck! It reminded me of the last day in college when I had to go to every department and get a NOC signed so that they could relieve me from college. I could not fathom the need for such a bizarre process, were they trying to prove some kind of point, were they trying to show me how big they were and that I was just a drop in the ocean. Whatever it was, it did not go down well with me, I think that the idea is downright ridiculous!

Fuming but tired, I did not resist, all I could think of was how do I get this over with and get the hell out of here. I took the form and went to the reception and was told that I needed to get other signatures first. Any amount of persuasion would not sway them. So giving up again, I went to the next department, to be told the same thing. Believe it or not every department I went to told me that the people were either busy or out to lunch or just unavailable. Twenty minutes later, I had given up completely. I have seen my share of bureaucracy and red tape over the years. There been times when I have got bills paid out of CDAC, but this one takes the cake. Completely frustrated and just plain angry having gotten into this mess in the first place, I left contemplating whether to return or not after wasting a good two and three quarter hours.

I am still appalled at the attitude with which employees are inducted. The "Deployment HR" as they call it is the first point of contact for a new employee with the company. I don't think it is all that wrong to expect some basic courtesy and understanding from these guys. I would go one step further and say that this junta should be trained to make day one of people joining an organization hassle free and pleasant. At least theoretically it looks a situation where both employee and organization would benefit.

Another important lesson was that, all policy makers should monitor periodically how the policies they make are being implemented on the floor. Couple of years ago there was a series on BBC where CEOs were made to spend a day out on the floor to see the pitfalls of working in their organizations. I remember one episode where Nandan Nilaikani was asked to join Infosys as a new employee. By the end of the program he was as hassled as I was. Most policies, I am sure are made with good intentions but what really matters is the way in which these policies are implemented. Probably the "get the paper signed" policy was brought in to acquaint new employees to the location of the various departments and the people that man it, which would be a good thing for the employee in future interactions. But the way it is implemented has deteriorated to such an extent, that it has now become a chore and any good that the policy was meant to do is completely lost.

I did go back the next day and when I look beyond the initial disappointments and the anger, I see that this experience was actually fruitful. I have learnt valuable lessons which will stay with me for some time to come. Of course this is not the end of the journey. There is much more to see and much more to learn and find myself looking forward to it.