How much should we do software testing?
Verification and Validation
Software testing is an activity as part of verification and validation or software verification and validation.
In other words, verification is to check whether or not software or system meets to its requirements, while validation is to check whether or not requirements of software or system meets to its purposes of users and/or customers.
On the other hand, read the definition of software verification and validation from the article "Software verification and validation" of Wikipedia.
Boehm defines:
Verification: Are we building the product right?
Validation: Are we building the right product?
What is the difference between the definitions of validation and software validation?
Ideally, requirements of the software or system should express what it is. However, practically, they may not always express it exactly. Usually, there is the gap between requirements and what the software or system is.
This causes the difference between the definitions of validation and software validation. The former is to check whether or not requirements of software or system meets to its purposes of users and/or customers. The latter is whether or not we are building the right product. The former implies the requirements matches what the product is, exactly, while the latter may not be always.
So, I adopt the following definition:
Validation is to check whether or not requirements, design and implementation of software or system meets to its purposes of users and/or customers.
How much should we make an effort at verification?
Suppose we’ll confirm whether or not a software or system meets to its requirements, completely. There are only two approaches: prove it mathematically, or enumerate all cases.
The former is known as Hoare logic. Read the article “Hoare logic” of Wikipedia.
One of approaches of the latter is model checking. Read the article “model checking” of Wikipedia. This is also one of applications of a finite state machine to software engineering.
Both of these “complete” approaches aren’t scalable. That is, they can’t apply to a large-scale software or system, due to computational complexity.
Both of them requires the requirements of the software or system to be defined strictly, or the software or system cannot be confirmed to meet its requirements.
Both of them also requires the requirements to follow the software or system that it expresses, or they will prove not the software or system but unreliable requirements.
So, practically, we are reconciled to confirming whether or not a software or system meets to its requirements, incompletely. This is the reason why we should continue endlessly to make an effort at verification, usually with software testing.
How much should we make an effort at validation?
Remind yourself about the definition of validation. Validation is to check whether or not requirements, design and implementation of software or system meets to its purposes of users and/or customers. In other words, validation is to check whether or not we are building the right product.
Notice what ”the right product” implies. This implies it is important to define what is right. Notice what “its purposes of users and/or customers” implies. This implies it will be changed by various reasons, i.e. user and customer needs, social background, laws and regulations, technology and etc. These are the reason why we should continue endlessly to make an effort at validation.
How do we solve the endless effort issue?
The fact that we are forced to continue endlessly to make an effort at verification and validation requires software to be maintained and updated. This denies that a software or system is fixed and frozen. This produces the concepts of Agile software development, DevOps and Continuous Integration(CI).
