Software verification and validation


In software project management, software testing, and software engineering, verification and validation is the process of checking that a software system meets specifications and that it fulfills its intended purpose. It may also be referred to as software quality control. It is normally the responsibility of software testers as part of the software development lifecycle. In simple terms, software verification is: "Assuming we should build X, does our software achieve its goals without any bugs or gaps?" On the other hand, software validation is: "Was X what we should have built? Does X meet the high level requirements?"

Definitions

Verification and validation are not the same things, although they are often confused. Boehm succinctly expressed the difference as
"Building the product right" checks that the specifications are correctly implemented by the system while "building the right product" refers back to the user's needs. In some contexts, it is required to have written requirements for both as well as formal procedures or protocols for determining compliance.
Building the product right implies the use of the Requirements Specification as input for the next phase of the development process, the design process, the output of which is the Design Specification. Then, it also implies the use of the Design Specification to feed the construction process. Every time the output of a process correctly implements its input specification, the software product is one step closer to final verification. If the output of a process is incorrect, the developers are not building the product the stakeholders want correctly. This kind of verification is called "artifact or specification verification".
Building the right product implies creating a Requirements Specification that contains the needs and goals of the stakeholders of the software product. If such artifact is incomplete or wrong, the developers will not be able to build the product the stakeholders want. This is a form of "artifact or specification validation".
Note: Verification begins before Validation and then they run in parallel until the software product is released.

Software verification

It would imply to verify if the specifications are met by running the software but this is not possible. Only by reviewing its associated artifacts, someone can conclude if the specifications are met.

Artifact or specification verification

The output of each software development process stage can also be subject to verification when checked against its input specification.
Examples of artifact verification:
Software validation checks that the software product satisfies or fits the intended use, i.e., the software meets the user requirements, not as specification artifacts or as needs of those who will operate the software only; but, as the needs of all the stakeholders. There are two ways to perform software validation: internal and external. During internal software validation, it is assumed that the goals of the stakeholders were correctly understood and that they were expressed in the requirement artifacts precisely and comprehensively. If the software meets the requirement specification, it has been internally validated. External validation happens when it is performed by asking the stakeholders if the software meets their needs. Different software development methodologies call for different levels of user and stakeholder involvement and feedback; so, external validation can be a discrete or a continuous event. Successful final external validation occurs when all the stakeholders accept the software product and express that it satisfies their needs. Such final external validation requires the use of an acceptance test which is a dynamic test.
However, it is also possible to perform internal static tests to find out if it meets the requirements specification but that falls into the scope of static verification because the software is not running.

Artifact or specification validation

Requirements should be validated before the software product as a whole is ready.
Examples of artifact validation:
According to the Capability Maturity Model,
Validation during the software development process can be seen as a form of User Requirements Specification validation; and, that at the end of the development process is equivalent to Internal and/or External Software validation. Verification, from CMMI's point of view, is evidently of the artifact kind.
In other words, software verification ensures that the output of each phase of the software development process effectively carry out what its corresponding input artifact specifies, while software validation ensures that the software product meets the needs of all the stakeholders. Software verification ensures that "you built it right" and confirms that the product, as provided, fulfills the plans of the developers. Software validation ensures that "you built the right thing" and confirms that the product, as provided, fulfills the intended use and goals of the stakeholders.
This article has used the strict or narrow definition of verification.
From a testing perspective:
Both verification and validation are related to the concepts of quality and of software quality assurance. By themselves, verification and validation do not guarantee software quality; planning, traceability, configuration management and other aspects of software engineering are required.
Within the modeling and simulation community, the definitions of verification, validation and accreditation are similar:
The definition of M&S validation focuses on the accuracy with which the M&S represents the real-world intended use. Determining the degree of M&S accuracy is required because all M&S are approximations of reality, and it is usually critical to determine if the degree of approximation is acceptable for the intended use. This stands in contrast to software validation.

Classification of methods

In mission-critical software systems, where flawless performance is absolutely necessary, formal methods may be used to ensure the correct operation of a system. These formal methods can prove costly, however, representing as much as 80 percent of total software design cost.

Test cases

A test case is a tool used in the process. Test cases may be prepared for software verification and software validation to determine if the product was built according to the requirements of the user. Other methods, such as reviews, may be used early in the life cycle to provide for software validation.

Independent Verification and Validation

ISVV stands for Independent Software Verification and Validation. ISVV is targeted at safety-critical software systems and aims to increase the quality of software products, thereby reducing risks and costs through the operational life of the software. ISVV provides assurance that software performs to the specified level of confidence and within its designed parameters and defined requirements.
ISVV activities are performed by independent engineering teams, not involved in the software development process, to assess the processes and the resulting products. The ISVV team independency is performed at three different levels: financial, managerial and technical.
ISVV goes far beyond “traditional” verification and validation techniques, applied by development teams. While the latter aim to ensure that the software performs well against the nominal requirements, ISVV is focused on non-functional requirements such as robustness and reliability, and on conditions that can lead the software to fail. ISVV results and findings are fed back to the development teams for correction and improvement.

ISVV History

ISVV derives from the application of IV&V to the software. Early ISVV application dates back to the early 1970s when the U.S. Army sponsored the first significant program related to IV&V for the Safeguard Anti-Ballistic Missile System.
By the end of the 1970s IV&V was rapidly becoming popular. The constant increase in complexity, size and importance of the software lead to an increasing demand on IV&V applied to software.
Meanwhile, IV&V gets consolidated and is now widely used by organisations such as the DoD, FAA, NASA and ESA. IV&V is mentioned in , and formalised in .
Initially in 2004-2005, a European consortium led by the European Space Agency, and composed by DNV, Critical Software SA, Terma and CODA Scisys created the first version of a guide devoted to ISVV, called "ESA Guide for Independent Verification and Validation" with support from other organizations, e.g. SoftWcare SL , etc.
In 2008 the European Space Agency released a second version, being SoftWcare SL was the supporting editor having received inputs from many different European Space ISVV stakeholders. This guide covers the methodologies applicable to all the software engineering phases in what concerns ISVV.

ISVV Methodology

ISVV is usually composed by five principal phases, these phases can be executed sequentially or as results of a tailoring process.
ISVV Planning
  • Planning of ISVV Activities
  • System Criticality Analysis: Identification of Critical Components through a set of RAMS activities
  • Selection of the appropriate Methods and Tools
Requirements Verification
  • Verification for: Completeness, Correctness, Testability
Design Verification
  • Design adequacy and conformance to Software Requirements and Interfaces
  • Internal and External Consistency
  • Verification of Feasibility and Maintenance
Code Verification
  • Verification for: Completeness, Correctness, Consistency
  • Code Metrics Analysis
  • Coding Standards Compliance Verification
Validation
  • Identification of unstable components/functionalities
  • Validation focused on Error-Handling: complementary validation regarding the one performed by the Development team
  • Compliance with Software and System Requirements
  • Black box testing and White box testing techniques
  • Experience based techniques

Regulatory environment

Verification and validation must meet the compliance requirements of law regulated industries, which is often guided by government agencies or industrial administrative authorities. For instance, the FDA requires software versions and patches to be validated.