Author: V.Kharytonov
Software measurements and metrics: fundamentals (on the example of eGovernment and eCommerce)
With the recent establishment of new regulatory bodies and eGovernment organizations, the growth of software developers and quality assurance professionals has almost doubled in the past 2-3 years. To ensure the sound and more predictable development of high quality systems, it is important for developers to gather and evaluate measurable data that guide estimation, decision-making and assessment. It is common sense that the ability to measure and analyze will lead to improved control and management.
Product metrics are also referred to as software metrics. They are directly associated with the product itself and attempt to measure product quality or characteristics of the product that can be connected with product quality. Process metrics concentrate on the process of software development and measure process structures with the aim of either distinguishing problems or pushing forward effective practices. Resource metrics are associated with the properties that are essential for the development of software systems and their realization.
Measurement is the process by which numbers and symbols are determined to characteristics of objects in the real world—this allows us to identify such objects according to defined rules. In software development, measurements are performed by using metrics, which are experimental designations of a value to an object aiming to characterize a definite quality of this object.
Software metrics are used to measure both the process and the definitive product characteristics connected with software development. The importance of software measurement and metrics has increased over the past 30 to 40 years; nevertheless, the main attention in forming a supportable software quality measurement program is focused on following some sort of cyclical trend.
History of Software Metrics
The first software metrics were suggested in the mid ’70s and since then, a great number of metrics have been recommended in the following years. In the 1960s and 1970s the attention for IT was on product and tools evaluation, in the 1980s and 1990s it was on process evaluation and quality enterprises. And in the 1990s the attention altered to measurement process integration. The advancement of metrics was followed by more effective suggestions on how to interpret results from metrics and methods merging metrics into measurement practices. Nowadays to make a quality measurement succeed, there must be a positive return on investment with a strong tie to development of the business and not simply to the IT department.
Either private or public objects connected with software measurements development for eGovernment and eCommerce applications can choose from a wide selection. Applied metrics should be chosen according to which criteria is most appropriate to be involved in the development process.
Therefore it is no longer a question of simply choosing metrics for an eGovernment or eCommerce project instead we must find suitable metrics and elaborately prepared engineering teams to operate them. Nowadays there are a large number of metrics (measuring almost everything) and if a metric is selected without basing the assortment on a complete breakdown of the development needs and a general examination of the suggested metric’s applicability – this could result in minor profits from its use or even no profits at all. To benefit from the use of metrics it is important to have a complete understanding of the different existing metrics. We should also be able to clearly define why, what and when to measure.
So the first question is: ‘why use metrics?’ Metrics are required to provide understanding of different components of eGovernment and eCommerce software systems and projects. Since it is not always known what makes a project fail, it is necessary to take measurement of qualities of projects that work well together with the qualities of unsuccessful projects. Metrics provide a number of indicators for the developed software. Indicators are metrics or combinations of metrics that present valuable information of the software development process, the software project or the product itself.
Applied software measurement tools
Measurements aim at the valuation function of the status of the development process and the developed product. Thus, metrics can be used for performance evaluation, cost valuation, effort estimation, improving efficiency, choosing best practices and, in general, for enhancing the quality of eGovernment and eCommerce systems.
This discussion leads to the next question: ‘what to measure?’ As stated before, process and product are what we need to measure. People may say that, since the result of eGovernment and eCommerce projects is software, what we need to measure is only software. But this is not true. If the product you have created is incorrect, do not try to fix just the errors, but also fix the process that permitted the errors into the product. This way you will be fixing the errors in your applied software measurement systems and following productions. So, both process and product metrics and measurements are important in eGovernment and eCommerce software development. Never forget how important it is to define the needed product quality features before choosing the suitable metrics. The selection of quality features supports in finding what needs to be measured and what needs not- this all depends on the special needs of the eGovernment and eCommerce application. In the early 70s a framework for measuring such characteristics was defined by McCall, Richards and Walters who suggested the Factors Criteria Metrics model –also known as FCM model. This defines software quality in terms of sub-characteristics. Years later, the ISO standard ISO 9126 normalized what product quality is in terms of sub-characteristics – this included FCM and experience from similar proposals. Therefore the explanation of product quality is very significant because product metrics are used in the software development procedure to measure the product characteristics that are associated with product quality.
Once a clear understanding of the aims and reasons for measuring has been reached, the next question is: ‘when to measure?’ Although measurements should be directed throughout the entire eGovernment and eCommerce software development lifecycle, their scope differs depending on the development phase. Various measurement aims are defined at various development phases, thus resulting in various kinds of metrics. In the early phases of eGovernment and eCommerce software development, metrics are mainly used for software measurement and estimation purposes. It is valuable to select metrics connected with different projects as these can be useful as historical data for future projects and will ultimately contribute towards better results.
In the intermediate phases of the eGovernment and eCommerce development process, metrics are used for project monitoring aims while code metrics are used to avoid errors. Additionally, defect reports during testing are used for product quality evaluation and the calibration of measurement methods of the early phases. One more key purpose is the gathering of external measurement data following project delivery, more specifically during the beta testing or maintenance phases of an eGovernment or eCommerce project. So, it is important to note that the time to measure is specified by the necessities and the purposes of the measurement program and can vary from project to project.
To summarize, by using an oversimplified statement, it could be said that, metrics are greatly important for the development of software to be integrated into eGovernment and eCommerce systems. Metrics assist while performing a valuation function in the early phases of a project, avoiding problems in intermediate phases and estimating quality in the late project phases.
Using software measurement metrics for eGovernment and eCommerce systems development
Software product metrics can be characterized as external and internal product metrics. The internal ones are those used to measure qualities of the product that can be stated directly by investigating the product on its own irrespectively of its behavior. External metrics of the product are those used to measure characteristics of the product that can be stated only with respect to how the product relates to its environment.
Internal metrics can be divided into three categories based on the product qualities they measure. These categories are: size, complexity and data metrics. As far as internal product metrics in general are concerned, it is important to mention that one of their major benefits is that they are easy to systematize and therefore data collection can be made in an easy, automatic and profitable way. Additionally, the measurement conclusions can also be examined in a programmed way using statistical techniques and consequently results can be made swiftly. Tools such as QSUP, GQM automation, etc. have made internal measurements very easy to conduct.
On the other hand, it should also be noted that among the disadvantages of internal product measurements, is the fact that they are often hard to explain. In other cases, the internal quantities measured are not clearly associated with the external quality characteristics that should be assessed. Such problems can only be solved in the framework of a definite measurement method that unites internal and external metrics, as will be discussed hereinafter.
Based on the ISO 9126 standard and other related works, the external aspects affecting software quality are defined as; Functionality, Usability, Efficiency and Reliability. External metrics are used to measure these four factors or the qualities of which these factors are formed. This is important for stating the difference between generic metrics and metrics defined especially for eCommerce systems.
Unlike internal metrics (measuring software internal qualities and directing at connecting measurements of such characteristics with these factors), external metrics measure these factors or their characteristics. Such metrics can be made on subjective estimates. Among the means employed by external metrics are surveys on user opinion providing valuable measurements for a proper software usability measurement inventory, software system functionality or usability. Measures like defect reports, or mean time between failures are used to define product reliability, while measures like memory usage are used to define effectiveness.
As previously mentioned, the introduction of external metrics indicates that a definite degree of subjectivity is involved. Even metrics that appear to be objective are often considered by some degree of subjectivity. For example, defect reports seem to be a solid metric that can be used to factually measure reliability. But it is the time and the degree of product usage, the user experience and even the user’s motivation to manage and submit a defect report which can influence the number of defect reports submitted by a user. Therefore, such metrics must be investigated very intently and under a framework that will take such issues into consideration. One of the main advantages of external metrics is that they measure outright the chosen external product quality features, meaning that no further analysis or interpretation is needed.
Furthermore, external metrics greatly promote what is considered to be one of the main aims of eGovernment and eCommerce quality: user satisfaction. On the other hand, disadvantages and difficulties should be carefully considered when deciding to use external metrics. The most important of which is that such metrics are not objective and as a result extra effort is required to guarantee their objectivity. Additionally, they are not as cost effective as internal measurements and in many cases it is very hard to manage measurements due to high error rates, specifically in cases that error detection techniques have not been used during measurements.
How does one combine software measurement internal and external metrics into a measurement method?
As it was mentioned before, internal and external measurements must be managed under a well-defined framework with precise aims. Before choosing the fitting metrics for any project, a quality manual should be performed- this should include choosing and recording all metrics available for use in the software developing entity. This manual is a main component of the metrics implementation process and includes the metrics, the measurement techniques as well as the guidelines for the implementation of metrics, the data analysis and the improvement actions needed for enhancing the developing process of eGovernment and eCommerce systems. It should also be noted that the quality manual contains all metrics that are available irrespective of how many times they have been used or the availability of measurements data from past software development projects.
For each eGovernment or eCommerce project, a set of metrics adequate for this specific project is chosen from the quality manual. The principles on which the assortment of metrics is based are the special quality factors that the project places emphasis on. This set of metrics is documented – using the guidelines available in the quality manual– and contains the quality plan of the specific project. Therefore, an eGovernment or eCommerce project quality plan should contain all the metrics, measurement guidelines and appropriate goals for the project. It is self-evident that the project plan of a specific project may entirely differ from another project’s plan and may use an entirely different set of metrics.
The quality plan of each eGovernment or eCommerce project should contain internal metrics. This will secure an easy and reasonable way to identify possible causes for low product quality, as this might be perceived by the end-users who might take early corrective action. It should also contain external metrics – applied during alpha or beta testing and post shipment. This will measure external quality factors, as well as the soundness of the internal metrics and measurements results or even calibration of internal metrics.
The successful collection of metrics and measurement techniques to be included in an entity’s quality manual is greatly related to on the entity’s maturity. The adoption of sophisticated techniques and complex metrics by a company may prove to be unsuccessful and unproductive if it is not supported by years of experience with metrics and measurements and huge volumes of data from past project measurements. Software developing companies should always keep this fact in mind and should not set measurement goals too high at the early stages of metrics application.
Summary and Conclusions
Just as we typically need to define the weight, volume, and dynamic flight characteristics of a developmental aircraft as part of the planning process, we also need to define how much software to build. The importance of using software measurement tools has greatly increased over the past 30 years. Software quality measurement systems are valuable for gaining insight into software development; however, they are not a solution to issues in and of themselves. To implement a practical software measurement and metrics program effectively, we must be aware of limitations and constraints.