How have uncertainty of software during development been overcome ? — The birth of Agile software development, Software product line engineering (SPLE) and eXtreme derivative development process (XDDP)
This story shows important history of software engineering between 1990s and 2000s, which includes the background of the birth of Agile software development, Software product line engineering (SPLE) and eXtreme derivative development process (XDDP).
Importance of estimation of efforts of software development
Most of the actual cost of software development is personnel expenses because it is a human-intensive work. They are roughly proportional to the product of the numbers of the people and months spent on development. A unit called man-month is derived from an assumption of this proportional relationships. Usually, a software developer charges its customer development costs based on estimated or actual man-months. It is an important fact that the charge will be based on estimate man-months, of the cost is determined before development. This implies the estimation is so important because it affects a profit of the development.
Difficulty of estimation of efforts of software development
It is, however, too difficult that any software developer estimates the efforts of software development precisely. According to Software Estimation by Steve McConnell, project costs may be overestimated by a factor of four or underestimated by a quarter when estimating them before launching software development. This uncertainty of estimation will be reduced as progress of the development. This is known as the cone of uncertainty.
This uncertainty tends to make software development like gambling. Too often, it will “burn”, and fail to complete it, in normal. This may produce a huge deficit, and exhaust developers and stakeholders.
One of the reasons of the uncertainty: Uncertainty of user and customer needs
There are several reasons of the uncertainty: One of them is derived from uncertainty of user and customer needs.
A user or customer doesn’t recognize their own needs, precisely. This has been illustrated humorously as a tree swing cartoon (see Wikipedia or search images with keywords “what the customer really needed” by Google), since early 1970s. This may bring software product unsatisfying for the user and/or the customer.
There are two approaches to solve the issue. One is to show a fragment of a real software product that may meet the needs, and to validate it by the user and/or the customer. Another is to have various requirements of it within some scope to meet the needs, and to realize various implementation with reasonable costs.
The former has been grown as Agile software development (see Wikipedia), while the latter has been grown as Software product line engineering (SPLE, see Wikipedia).
In order to reduce the uncertainty: Estimation after development
The fact of the cone of uncertainty leads to a kind of the proposition that progress of the development make such an estimation preciser. This is a main advantage of iterative and incremental development, including Agile software development and derivative development.
The latter is grown as eXtreme derivative development process (XDDP), which is a software development methodology famous in Japan, created by Yoshio Shimizu at least before 2007 (see here, written in Japanese).
