Posts Tagged ‘Product Development’

Assembly Line Software Product Development

May 24, 2009

Compared to many of its engineering counter-parts like Manufacturing or Civil/Construction Engineering, Software Engineering, IMHO, probably still has not come to the level where one can see the maturity of standardization across. There are lots of arguments for this and many of them quite valid. Many consider that in Manufacturing (and similar engineering fields), it is all about creating the same thing again and again. Whereas Software is about creating something unique every time. Also, purists do differentiate between Development and Production. Few authors have said that Development is like creating a recipe, whereas Production is like following that recipe. So the variability in both Development and Production should be expected.

I am not going to disagree much with many of the arguments listed above. With more than 20 products I have been actively involved in my career, I know that each one of them had its unique DNA. Each had a big element of variability and trial-and-error model of software development. So I will agree that the expectation of standardization across them would have been far-fetched. However, my talking point in this post is that while at an Application or Product level – things can be difficult to standardize, however at a software delivery level things can certainly be standardized. For me in this post, software delivery signifies what Assembly Line signifies in a Manufacturing scenario – an arrangement of workers, fixtures, processes, and equipment in which the product being assembled passes consecutively from operation to operation until completed.

As part of Version 1.0 program, my team and me took many of the concepts of the manufacturing assembly line at an abstract level and configured them to apply that in our Software Delivery to a significant success. While we were building multiple products each with a complete different and distinct DNAs – we were also keen in making sure that these products go through certain arrangements which were common across all the products. (The possibility of developing multiple products in a Services organization certainly gave us this flexibility to experiment with such concepts; however I think many of these concepts can even be applied in product organizations where multiple product lines/releases do take place)

Here are some of the concepts which we believe resemble the practices of assembly-line software engineering -

  • Standardized Process for a Release – We felt that regardless of the type of product under development – we could standardize the process of how we go ahead in terms of making a product release across all the products. Here are some of the concepts which tried religiously following all across -
    • Use of Iterative way of Product Development Methodology. We even went to the extent of standardizing each of the iteration length to one month across all products.
    • Atleast one dedicated Design iteration would always prelude the Engineering Iterations.
    • Each iteration had fixed set of activities associated with design, development, and tests
    • Typical Releases across all the products were equivalent to 3-4 iterations.
  • Standardized Milestones across the Release – Every product went through similar milestones – Requirements Prioritization and Iteration Plan Development; Retrospection; Deployments, etc.
  • Standardized Product Engineering Fixtures and Tools – While the core product development platform varied from product to product, we made sure that the core underlying engineering platform which facilitated product engineering processes like Requirements Management, Project Management, Continuous Integration, etc.  were as similar as it could get.
  • Standardized Team Structures and Roles – While team size varied based the product complexity – we made sure that the team was organized into two groups – the functional group and the engineering group. Each group had some roles (such as Product Manager, Architects, Program Manager, Engineering Leads, etc.) which were common across all the products. The roles were well-defined so that each individual knew what was expected from them at each stage of the product life cycle.
  • Standardized Measurements and Continuous Improvement Mechanisms -  Each of the products across their life cycle were getting measured by common metrics on the dimensions of quality, schedule, and customer delight. We also made sure that the process across all the products had Continuous Improvement loops built in through methods like Retrospection, etc.

I think Software Engineering with the knowledge of millions of software products delivered to the market across decades can be standardized to significant extent similar to its other counter parts. Once again, I am not saying that recipes can be standardized. However, my argument here is that the process of creating that recipe and delivering that to its end users can certainly be standardized to a great extent.

Thoughts and comments are most welcome.

Applying Theory of Triple Constraints to Early Stage Products

September 14, 2008

Two things I need to expand upon before I delve into this post. Describe what I mean by ‘Early Stage Products‘ and then expand on ‘Theory of Triple Constraints‘.

While there might be many perceived perceptions of what an ‘Early Stage Product‘ means – for the discussion in this blog I am implying to a product which is typically at a pre-market validation stage. The product is still being conceived and built.  The hope is that after this stage (within our group we refer to it as Version 1.0), the product would be rolled out to its potential users which in-turn would facilitate the potential revenues. The primary goal of products at this stage is to get a real-world market validation – beyond just surveys, interviews, and reports. The organization which is building this product may be a start-up or even an established organization ready to launch a new product offering. Products in this stage may or may not be financially bootstrapped.

Now regarding “Theory of Triple Constraints“! This is nothing new. For anyone who have gone through Project Management – 101, would recall the picture of triangle with Scope, Time, and Money at its node.

Theory of Triple Constraints

Theory of Triple Constraints

Alistair Cockburn refers to it as “Iron Triangle for Software Development“. What the Theory of Constraints is telling us is that there are three variables of a project which one needs to manage for its successful completion – Scope (what needs to be built), Time (time required to get it built) and Money (money required to build to what is there in the Scope in the given Time). Change in any one of the variable can affect another or both of the other variables.

[Few folks have asked me about Quality as the variable. While I have seen this Triangle being morphed to include Quality too, personally I would like to think that in modern-day products, Quality is part of the Scope. Also, with the rising use of Agile ways of Product Development, a lot of assumptions associated with this Triangle are getting questioned. This is a topic for discussion on another day though.]

Now coming to the main topic of this blog – applicability of Theory of Constraints in the development of Early Stage Product considering the real-world constraints under which it is typically developed.  In all my experience of working with products trying to reach Version 1.0, I have seen the following to be always true.

Theory of Triple Constraints - Early Stage Products

Theory of Triple Constraints - Early Stage Products

  • Time to Market: How soon can one take the product to the market or Time-to-Market is the most important driving aspect for 99.99% of the early stage products today. I have left the 0.01% for products which are hopefully going to solve the cancer or hunger problems of the earth. The theory behind Time-to-Market is that because of the ever-changing needs of the consumers or increasing competition, there is a limited window which is available for a product to be launched. So with this, it is essential that the Time variable cannot be (or should not be) changed for such products – at least to the higher side.
  • Limited Cash: As I had stated in one of my previous blog, almost all product entrepreneurs today operate with a limited inflow of money or a fixed budget. This is also constrained because most of the products in this stage do not have the backing of revenues too. Under such scenarios, it becomes extremely important to lock-in the Cost variable too.

So the only variable which now remains is Scope. For anyone who has gone through an experience of the product development, this is one variable you want to keep as a Variable! And this is especially true in early stage products.  Many a times, the scope which is true in the morning may not be true in the evening. Also, any events which affects both Time and Cost, would need to be potentially adjusted by varying the Scope.

During my discussions, many have asked me as to what happens when some Scope has to be fixed from the “Go-To-Market” perspective. However, IMO, if you agree with me in the above para that scope is a variable, this is probably a mute point. Second, if things come to such a stage that all three variables have to be fixed, one of the first thing to assume would be that the initial planning was not upto the mark.

I would welcome your thoughts and comments.

Add to Technorati Favorites

Product Development – From Idea to Reality…

August 24, 2008

It has been some time since I have blogged. As usual, hectic work schedule over the last month took the toll on my blogging habits. Not that I am complaining about the same. The last month went into some wonderful work. On a personal front, it was an enriching experience for me to work in a high profile environment and to look at things from a completely different perspective. It also was a big validation of the GlobalLogic Version 1.0 Offering which we have put together recently and now have taken it to a completely different level. The ability of the entire team to come together to understand, comb through the issues, and deliver was very fulfilling to see.

While I will not go through the details of the project, the key challenge for us as a team was to help a high profile client of ours to go through a series of planning activities to convert an idea to a product and then take the product to form the base of a successful new organization. Envisioning a concept getting converted into a real enterprise product is not a simple task. IMO, almost most of the times how well one executes the tasks associated with product development supersedes the brilliance of the idea itself. Product Development today is an ardent task in daily-changing business scenarios and tough competition. In addition to that one has to hash through the combinatorics of available go-to-market options, product visualization, technologies, competition, risks, and partnerships and come up with a clear strategy. This requires broad and experienced heads to come together and work around evolving the best plan forward. This is where my team comes in. Our work starts with understanding what is driving our client to his/her vision; go through the multiple dimensions of issues; and come up with a prospective plan. There is typically no one single perfect plan to build a product. However, simply putting things in proper context and perspective can be of great value. Almost all our clients vouch for that. The main questions which we help them answer are -

  • What are we trying to build?
  • How will we build it?
  • What is it going to take to build it?

It is not easy to answer these three questions to the highest degree of correctness. One has to put the past experience to work a great deal here. However, the biggest criterion here is how to make the plan Agile and Adaptive. That is the key.

Add to Technorati Favorites

The Engineering Aspects of Software Estimation.

July 11, 2008

Before I jump into this post, let me describe what I mean by Engineering Activity. An activity where knowledge of relevant science is applied to achieve the required result is an Engineering activity.

I was reading this paper titled “Large Limits to Software Estimates” (PDF link) by J. P. Lewis (published in ACM Software Engineering Notes,Vol 26, No. 4, July 2001). I am not much of a mathematical theorems person, so I skipped the initial few sections and came to the conclusions part. In the conclusion part, the author raises the question – “Should Software Estimation be called as Engineering?“. He argues that if we were to consider Software Estimation to be an engineering process, then litigation/conflicts should be a reasonable or acceptable outcome for incorrect estimations. If not, the author raises the questions associated with ethics in the software community – considering that so many estimates are ill-founded or bloated. The author also points to another article titled – “Are We Developers just liars or fools?” by Robert N. Charette. (If you are not ready to pay to ACM for this, here is a link to a similar article written by the same author.)

Before I jump into writing my thoughts on this one,  I wanted to point out to a quote from Hannah Arendt, a self-claimed German-Jewish political theorist, which I had read sometime back.

Promises are the uniquely human way of ordering the future, trying to make it predictable and reliable to the extent that this is humanly possible.

To come to think about it, software estimation is also about trying to make the future predictable and reliable. However, except probably the psychics, I am not sure how many of us would make a claim on predicting the future to the reliable extent. This brings to my next point.

Anyone who has been in this business, doing software estimation is one of the toughest job. If given a choice, most of us would probably prefer the “will call you when it is done” approach for communicating the estimates.  While I agree with J. P. Lewis that estimation should be based on sound engineering principles; however in reality that is rarely humanly possible in software business.

Most of the estimates today are based on past relative experience and gut feel of the individuals even if the functionality is broken down to the nearest function points. I am not saying this in any negative way. Almost all who have had some good relative experience in this business would realize the fact that software development is a process of gradual refinement. This is especially true in today’s world of internet-enabled mass-consumer products as compared to a decade back. We have to keep in mind that both the consumers and the producers of the software products are in the end human beings. IMO, it is hard to find human beings who know what exactly they want consistently. Because of this the software estimation process is going to remain fuzzy in the initial stages and typically gets better refined with the time. So Robert Charette’s argument that software industry should stop reporting incorrect estimates in order for it to be held up for respect and emulation is slightly naive in my humble opinion.

Software estimation should be treated for it really actually means. Dictionary meaning of estimation is “approximate calculation“. And the dictionary meaning of approximate is “nearly exact; not perfectly accurate or correct“. Estimates like in any other context should be treated as a guideline. All the trouble starts when it starts getting treated as a quote or as an actual quantitative number. I am personally of the opinion that estimates can never be challenged; only thing we can do is to help improve it.

So if I were to plagiarize Hannah Arendt’s quote mentioned above and apply it to software estimates, this is how it would read -

Promises Software Estimates are the uniquely human way of ordering the future, trying to make it predictable and reliable to the extent that this is humanly possible.

I would love to hear your thoughts or comments.

Add to Technorati Favorites

Convincing others for Waterfall or Agile Methodology

June 27, 2008

A colleague of mine sent me this email today. He mentioned about the dilemma which one of his client was facing. I have pasted his email below (I have changed some of the names to respect the confidentiality)

Hi Manish,

I needed some help on projecting Agile as compared to water fall. PQR product is getting implemented at ABC and ABC’s project head Dorothy is facing some challenge in motivating her team to work in Agile way, especially when it comes to UAT. I think it’s becoming hard for her to convince that ABC’s users can continue part testing for features released in UAT. ABC’s user’s team is more comfortable doing QA when complete product is rolled out for UAT. This approach will hamper implementation dead line (before <Month> 08 end). She was looking from us some kind of presentation / stats which clearly explains benefits of Agile over Waterfall and give some stats (if possible) on failed big projects using waterfall and successful agile ones.

Can we get some stats / figures etc around this request?

Having been in similar situations in the past, I can certainly understand Dorothy’s challenge. While I did respond back to my colleague (please see below), I thought about posting this email as a blog post to see if any of the readers out there can help Dorothy with their experiences and metrics. Also, I am sure there would have been many in Dorothy’s shoes and wanted to find out what they did to overcome the challenge. Please feel free to add your experiences in the comments section below this post.

My response to my colleague -

Hello,

Software industry is littered with examples where the software failures (by failure I mean that it was not able to meet end-users needs) have occurred. However, most of the time it is very difficult to say that the failure occurred because of waterfall or agile or any other similar methodology. IMO, failures are always because of certain risks which could not be mitigated at the proper time. This article at the link below shows a survey results which lists why project failures have occurred in the past. When you look at these risks, it is fair to say that Agile Methodology is better equipped to mitigate these risks as compared to Waterfall.

http://www.bcs.org/server.php?show=ConWebDoc.19584

Coming to your point of metrics – I personally do not have the project failure/success metrics comparing Agile vs. Waterfall. I can certainly hypothetically speculate, however I do not think that is what Dorothy wants. If I can understand what she is trying to look for is to look for ways to mitigate the uncertainties associated with the implementation deadline. If she lists these risks down and plans to mitigate that, she will very clearly see and convince people that Agile is probably their best bet.

I am slightly going on a philosophical note here – unfortunately we engineers many a times fall into a debate of one vs. other – Agile vs. Waterfall, Java vs. .NET, and so on as if they were two religions. I think achieving the goals is important as compared to looking at the rewards associated with selecting one vs. the other.

While I do not have a ready presentation on Agile vs. Waterfall (there are plenty on internet though which you can point to though). However, I would be happy to show and explain how we (in GlobalLogic) do it in Agile if Dorothy is interested. Also, please do note that the way there is ‘bad waterfall’ there is ‘bad Agile’ also ;-)

Thanks

Manish….

As I said in my email, the success and failures of a project are typically not because of a particular methodology – but how the methodology helps to manage and mitigate the risks associated with any projects. There are good and bad implementation of all the methodologies out there. In my experience, it is not in the best-health of the project to get into the debate of goodness of one approach vs. the others. We should leave it to the ‘academicial-minded‘ to do that. The key is to take the best of all the approaches which would mitigate all the risks associated with the project.

Add to Technorati Favorites

Design/Development in Waterfall/Agile

June 22, 2008

I came across this post written by Kevin Barnes on Agile Processes. A slightly old post and I am not sure what exactly Kevin was trying to convey in this post. However, there was one picture on the same post which caught my eyes and to which I could relate completely.

Design/Development in Waterfall/Agile

(src = http://www.codecraft.info/images/processSeesaw.jpg)

The picture nicely describes how the two processes are loaded with respect to the Product Design and actual Product Development. Whether it is intended or not, there is typically a lot of upfront design effort in Waterfall. Whereas the Design process is more emergent in Agile. As they say, more important in Agile is the working code.

Add to Technorati Favorites

Developer Productivity vs. Runtime Performance of Programming Languages

June 16, 2008

Although we have been playing around with it for more than year and half now, over the last few quarters my company (GlobalLogic) has invested heavily in and adapted the new generation development platforms like Ruby-on-Rails (RoR), Adobe AIR, etc. One might term it to be slightly slow in comparision with the industry adaption trend. However, our adaption was also driven by the needs of our clients. In the recent few quarters, we have suddenly observed that these platforms have now become main-stream. RoR/PHP is becoming a popular option in building the modern web apps whereas AIR seems to be the current ‘cool kid’ on the desktop side.

Adapting a development platform at the early stages of its evolution is always a challenging decision which an Engineering Head has to take for the product. I have always believed that adaption of any new technology is driven a lot by the herd mentality. Just because of the notion that many others are also using the platform, many a times the selection of a development platform becomes a blind decision. On the flip side, if a choice is about selecting a new platform – the same ‘herd’ mentality also comes in. The onus is more on the Head to prove his/her choice criteria rather than simply using the regular proven and established ones. Also, it is one thing to build up an experimental system using the new platform. However, the decision to adapt the same platform for engineering commercial products can be a hair-pulling one. Many a times in the end one has to finally depend on the two G’s – guts and gamble. Also the decision-maker will have to be continuously ready to answer questions associated with the hypothesis as and when any bad news associated with the platform (e.g. Twitter’s challenges attributed to RoR) or the product (e.g. initial breakdowns of the product) trickles in.

Because of the nature of the products we have been developing in my group, we felt that RoR was a very good platform choice for the same. So far (and I am knocking the wood here) our choice of the platform seems to have been validated based on the results we are seeing. It is amazing that how much dramatic productivity-related improvements these new platforms have made over the years as compared to some of the older generation-ones. I have myself observed this in my teams. Probably one of the big reason for this improvement is that these new platforms have started becoming very specific (e.g. web in the case of RoR) rather than the general purpose language platforms like C/C++, Java, etc. Dion Hinchcliffe in his posting “Creating Open Web APIs: Exploring REST and WOA in Rails 2.0” refers to productivity improvement numbers in the range of 10 to 20 times. Considering that programming time (or the development time) is the biggest cost of product development, the savings because of these improvements can be enormous. The built-in practices associated with Unit Testing, Agile, Reusability, etc. also seems to have aided in the aspects of building a more quality product.

Lately, the news about Twitter performance/uptime related challenges seem to have raised a lot of speculation on the run-time performance capabilities of RoR. While nothing has been proven as yet, such questions associated with newer platforms is enough to keep one awake in the night. Before jumping into RoR, I had in my mind that the run-time capabilities of these new frameworks would be weak in comparision with the compiled software platforms like Java, C++, etc. The above post by Dion has a good explanation about the same -

One is that the more popular, older programming languages tend to be relatively low level and general purpose and were designed for a different, older set of constraints. This has given us baggage that is often not very applicable to the modern Web-based world. Second, we’ve become very good at understanding the idioms and “syntactic sugar” that makes developers more productive for Web development and we’ve put that into the latest generation of programming languages and Web frameworks. Unfortunately, the combined newness of these new Web development platforms and their preference for coding time efficiency in favor of run-time efficiency has conspired to make the results they produce relatively slow and resource inefficient compared to what is potentially possible. Newness in this case is also a kind of performance tax since we just haven’t had enough time learning how to make these new platforms perform well at run-time, similar to early versions of Java before the advent of the Just-In-Time (JIT) compiler.

Based on the above and my experience until this stage – I am more comfortable in recommending these modern languages towards Consumer/B2B products as compared to telephony/banking transactional products. For the preferred products, many a times it seems like the advantages associated with improved developer productivity far over-weighs the slight runtime performance disadvantage these modern languages bring in.

Also my opinion is that it is too early to start comparing the modern development platforms with the older programming environments. Increased usage will drive the initiatives associated with the strategies targeted towards improving their weaknesses.

Thoughts, comments, and suggestions are welcome.

Add to Technorati Favorites


Follow

Get every new post delivered to your Inbox.