Friday, September 27, 2013

the Recipe (Software) & the Quantities (Estimations)

Software Development & Software Engineering are a bit like Cooking & Recipes.

 
For the Chef(s) (any category of experts ranging from street food joints to house hold cooks), their experience helps think & act in a Blink.
Further in most cases, its an individual cook who is at the helm of affairs. 
When its a bigger group to be catered, the Cooking Team members assumes different responsibilities to deliver the taste on-time, on-quality and sufficient quantity.

Software Development by a group of specialists is a different game versus by a group of individuals (Diverse/Cross-Functional Teams).
To ensure the combined knowledge is uniformly adopted and put to the best use, Software Engineering (aka Recipe for Software Development) creeps in.
 
 
From the seemingly "Unwanted List" of elements of Software Development (from what is otherwise a sort of a creative endeavor), lets discuss Estimations (the Quantities in the Recipe).
 
Like the recipes get perfected over generations, the process of estimation takes a couple of cycles/years to mature.

The process evolves gradually starting with:
 
Estimation Level & Unit(s)

Based on the phase, the level of estimation is different. With a basic understanding of the overall concept to be built, the level of estimation is of relatively higher degree (possible variance in the range of +/- 50%).
As the concept gets broken down into Requirements, the level of estimation starts to move closer to the possible effort needed (variance comes down to probably +/- 25% range).

At the time of detailing of the individual Requirement(s) into work break-down (Tasks), the level of estimation is closest to the possible effort needed in realizing the requirement (variance around +/- 5%).
If the 5% variance appears as a surprises, lets not forget it is an estimation. 

At each phase of detailing i.e. Concept -> Requirement -> Task, there is an increase in understanding, uncertainty is lesser hence the approximation process is better.  
To be able to map the estimations across the various phases, the unit of estimation has to be agreed.
The unit (Day/Hours) has to remain same across all phases, the approximation improves.
The actual planning and execution cycles have to based on the this unit as well.

Estimation Approach
  • amount of detail needed across the various phases of the software development life-cycle is different
  • estimation at various levels is done based on the understanding and expertise of the concept, skills and other parameters involved in the building and delivery process
  • effort needed (Estimate) and duration to get the software built (Duration) are two different aspects
  • unit of estimation (Days/Hours) must not be mis-interpreted
  • sum total of All Tasks across All Requirements has to add up to the estimate of the Total Concept
Accordingly different teams approach estimation process differently based on how much detail they are working with

Normalizing Estimation(s)
 
The estimations can be different per individual team member given the relative skill match. However for the combined endeavor, the overall estimation has to be based on the average skill of the team involved in building the software. This calls for Normalizing the estimates to ensure, they stay applicable for the whole team. 
 
The process of Normalizing is often based on the build-up of the functional/domain knowledge and the involved technical skill-set. The case of a team member with [High Technical Skill + Moderate Domain Skill] is different than [Moderate Technical Skill + High Domain Skill] or any other applicable combination of these factors.
 
Based on the team composition, the applicable Normalizing has to be applied to keep the estimations reference-able. (The impact due to the newness to domain or the technical skill in terms of applicable learning curve is a matter of planning.)

the Quantities (Estimations) thus have a crucial role in the Recipe (Software) to lead to a tasty delight.

bon appetit !!!

Wednesday, August 28, 2013

Experiencing the Manifesto as you grow Agile


In some (if not most) of the transitions to an agile way of building software, the awareness of the manifesto happens after having started on the journey. The trigger to adopt agile probably varies in any given situation.

While some take a structured approach to adopt one of the agile software development frameworks, there are many who are looking to find a solution for a specific situation bothering them for a (maybe relatively) long time already.

In cases where the start is a specific situation either "taming the backlog of work" or "incremental software development" or any similar situations, the idea about existence of the manifesto and/or the values it promotes is more a matter of discovery than probably a master plan.  

Aah, there you go.


In the field of Software development there is always some parameter which limits you from re-using an existing piece of software and being able to make a predictable plan of the re-use scenario. 

Barring few instances, most cases involve researching, adapting to a certain degree and dealing with certain unknowns. Making a comprehensive plan in such situation(s) asks for (some) room to maneuver. 

To relate, let me know share two cases to illustrate the point.

Case 1: A software development organization having experienced success with a set of components that worked in tandem wanted to weave them into an integrated system. The idea was to deliver enhanced business value in a seamless manner. This grand ambition off-course lead to meticulous planning and drained all the available capacity to execute and deliver on this multi-year plan. Apart from the delays and impact due to the re-work of the design and re-planning, as more and more resources got absorbed into the plan, work started piling-up, concerning the support of in-production versions of the components and a huge technical debt. Without an end in sight, something needed to change to get things back in control.

Case 2: A software development organization set off to build a completely new product as a stand-in replacement for the existing product. The idea was to use the latest and emerging technologies of the day to stay current with the times. This new product was en-visioned to be a very flexible, and a highly configurable product. In dealing with a product of such complexity, years went by, the team size grew from a select few engineers to almost everyone (barring few entrusted to maintain the existing product). At the end of this period, there was nothing tangible to show to the customer. A different approach was needed to get the development to move further on building value for the customers apart from making a robust software.

As evident in both the above cases, these ambitious endeavors shared space with the other regular bits of work related to maintaining the existing product(s) and component(s). Building these new software in silos and attempting to build them in a start-to-finish manner was apparently not working in the expected way and probably caused frustration across the organization.

These situations lead to looking for a different approach that can help piece the half-baked portions together in a time bound manner. This was off-course "Responding to change over following a plan" in action.

The choice was to adopt Scrum. 

As a first step, instead of "start-to-finish" approach, step-by-step approach was adopted to incrementally get the half-baked portions to shape up into a working software that delivers value to the customer. 

Rather than focusing on detailing all the steps into a plan or finalizing the complete software architecture, it was a brush with working software over detailing.


The new approach did not promise a quick turnaround for all the problems. It helped prioritize the issues and deal with them in an iterative manner. To ensure we stay on course, it was decided to work with the potential users of the software. 

This lead to earlier feedback and to keep the software fit for business use rather than having to defend the quality of the software using the contractual clauses.


kicked off and "Responding to change over following a plan" was happening.

A start-to-finish plan and a completely detailed architecture enforced rigidity into the approach. 

All the internal learning while building the software could not get the required attention due to the rigidity in the plan and not enough scope to re-work and improve.

With the move to Scrum, there was more room to process the internal feedback as much as the external inputs. This helped create an environment of collaboration within the teams. Building the software right took as  much precedence as building the right software. There was increased involvement from all the stakeholders as the software started to shape-up.
became the order.

This was a process of discovery of better ways of developing software. This was a multi-year period of learning the nuances of Scrum and improving upon the way software could be built better. 


The tenets of the manifesto, kind of naturally comes as you start to thing outside of the fixed paths to solve some of the issues.
The paradigm from Manifesto as a goal to Manifesto as a result becomes interesting as it actually matches the whole notion of Agile and the Manifesto itself i.e. change over plan and learn as you go along.

Accepting the change is more important than just sticking to a plan. The world of tomorrow will be different than the world today and hence the plan most likely less relevant. Plan sets a direction and is not set in stone. It needs to be adopted as the execution proceeds further.

When the complexity levels are high, tackling change becomes difficult as all the consequences may not be seen through. Responding to change in such situations might even blur the sense of direction. As you make further steps in this journey, you will come to realize the benefits of working software over detailing, DO over THINK, SHOW over WRITE.

While there was a promise of a certain value in the existing practices, there was more value in the tenets of the manifesto and the Principles behind it.

Monday, February 11, 2013

Author Review - Malcolm Gladwell


How much of advertisement of a product is sufficient to generate enough sales (to meet the revenue expectations)? Can there be a single answer to this question? (which product, what is the target market?). Anyway, let’s look at this story. A group of youngsters wanted to be different than the crowd. For the weekly partying at the local pub, they wanted to make a style statement. They chanced upon an old generation brand trying to keep up with its new generation competitors but unable to match. The choice of their shoes gave a much needed lease to the brand to re-establish itself in the new market. The trend of youngsters using that brand spread up from the local pub to the city and eventually caught up across the country. Hush Puppies was back. Apparently Hush Puppies achieved its tipping point with the new mode of publicity created due the youngsters choice of their products. The phenomenon of tipping point happens in many spheres from product marketing and sales to epidemics. Tipping point represents the point where the activity catches up the momentum and drives itself bigger and bigger. In the case of a disease this represents the spread where it becomes an epidemic. In the case of a product marketing it represents a state of branding where people seek the products without any additional need to advertise.

Some people seem to have a very advanced understanding of certain activities, events, trends or specific situations. They seem to be able to make snap-judgment. They are connoisseurs in specific areas (especially in Arts) who get so good at identifying, analyzing specific situations or trends. A reputed museum housed a piece of a famous statue made in marble. They had in-house experts who helped immensely in the selection and procurement process to guarantee the genuinenity of the piece. The statue was scientifically analyzed for its age, material, origin and other critical parameters qualifying it be showcased to offer the visitors a truly rewarding experience of Italian art in the US. One of the visitors happened to have seen a similar statue elsewhere and was surprised at the possibility of this museum being able to obtain another original copy. This led to some background investigation by the museum curator. The special team from the museum paid a visit to the city of its origin in Italy. There they met one of the artists who expertise’s in the carving of this form of statues. His examination of the copy in the museum revealed some differences compared to the original form. One striking feature that stood out was the nose of the museum piece as compared to the original form of carving. Was the museum piece a well-made fake? This thought spend a chill down the spines of the curator and the museum staff. The statue was procured for a hefty cost and a detailed check to make it worth a certain US$ per visit. What explains the age of the statue? What explains the color degradation of the marble? The Italian carver offered the details on how fake’s market achieve this. The Italian carver was able to think without thinking in a Blink. This ability of rapid cognition happens in many spheres of life by people who have gained such experience to be able to deliver snap-judgments.

In a selected group of youngsters for a sport, some seem to have an advantage due to their date of birth helping them qualify for the group but gives them an advantage of almost 11 months compared to the others in the group. This advantage of almost 11 months (close to an year) in age and probably physical growth can affect their performance considerably. Like in this case with certain persons lie outside of the regular group, this situation occurs in other spheres as well. The term Outliers represents certain things, events  or phenomena that lie outside the normal experience. The situations and/or events leading to the Outliers phenomenon analyze the individuals circumstances combined with the personal drive of in the involved individuals that have either lead to success or trouble.

The above stories have a certain mystery being unveiled. Malcolm Gladwell likes and specializes in unearthing the possible reasons behind these happenings. He has written a series of book focused on topics which seem to re-occur in the daily life around most of us but don’t seem to have a clear explanation.  His books Tipping Point – How little Things can make a Big Difference, Blink – The  Power of Thinking without Thinking, Outliers – The Story of Success bring these stories with their hidden facets either of the individuals involved or the circumstances.

 You can follow more of his work, books and his blog on www.gladwell.com. Happy Reading!!!