Cost of Poor Software Quality in the U.S.: A 2022 Report
In this 2022 update report we estimate that the cost of poor software quality in the US has grown to at least $2.41 trillion, but not in similar proportions as seen in 2020. The accumulated software Technical Debt (TD) has grown to ~$1.52 trillion.
Measuring the Structural Quality of Software Systems
The International Organization for Standardization recently published ISO/IEC 5055:2021 for measuring the reliability, security, performance efficiency, and maintainability of software systems by detecting and counting severe violations of good architectural and coding practice. Industry has needed an international standard defining measures derived from source code analysis covering both embedded and business systems. ISO 5055 was developed by 31 experts worldwide from OMG, Consortium for Information & Software Quality (CISQ), and published by ISO - International Organization for Standardization.
State of the Industry Report
The Consortium for Information & Software Quality™ (CISQ™) released its State of the Industry Report on Software Quality Analysis, the first comprehensive study covering tool vendors, system integrators, managers, and engineers.The impetus for this study was the alarming increase in software quality-related incidents and concerns that organizations are not getting the basics right when it comes to quality. Members of CISQ wanted to see how the move to agile and DevOps is changing not only software quality practices, but developer attitudes and behavior when it comes to code quality. It was also important to see how software quality standards are being utilized by system integrator and end-user organizations - which standards are being used, which sectors are driving adoption, and how organizations are deriving value from the standards.
Trustworthy Systems Manifesto
As businesses and governments automate more of their business and mission processes, the risks to which software-intensive systems expose the organization grows dramatically. In an era of 9-digit glitches (incidents with damages over $100,000,000), senior executives outside IT are held accountable, and some have lost their jobs as a result. Since senior executives are rarely IT experts, they need guidance on how to govern the risks of untrustworthy systems.
The Cost of Poor Software Quality in the US: A 2020 Report
This research report concludes that poor software quality cost the U.S. upwards of $2.08 trillion dollars in 2020 taking into account losses from operational software failures, poor quality legacy systems, and unsuccessful projects. The report examined how much the world is spending on IT software today and the fundamental issues causing problems. Looking backwards, legacy IT systems are holding us captive, looking forwards, technology innovations are coming faster and faster, and looking at present day, we're facing highly vulnerable and deficient systems-of-systems. The report was written by Herb Krasner, a member of CISQ’s Advisory Board and retired Professor of Software Engineering at the University of Texas at Austin. The report was underwritten by gold sponsor, Synopsys, and silver sponsors, OverOps and Undo.
How to Deliver Resilient, Secure, Efficient, and Easily Changed IT Systems in Line with CISQ Recommendations
In complex software applications, the same piece of code can be of excellent quality or highly dangerous - so, excellent code quality within an independent program does not guaranty a resilient, safe and efficient IT system. Correlations between architectural programming mistakes and production defects unveil something counter-intuitive. Studies show that basic coding errors within a program account for 92% of the total errors in source code but only account for 10% of production defects. Yet, software flaws at the Technology and System Level account for 8% of total errors, but consume over half the effort spent on fixing problems and lead to 90% of the most serious production issues. Engineering quality maturity grows exponentially with adherence to CISQ best practices.
How Do You Measure Software Resilience?
A resilient software-intensive system can experience failure in one or more of its constituent components, encounter unexpected inputs or external conditions, or come under malicious attack and yet continue to provide a useful level of functionality to the user and recover disrupted functions quickly after an incident. This whitepaper provides a definition of software resilience and discusses how to measure software resilience. Learn how software resilience relates to software quality standards and review the architectural attributes that affect resilience.
IEEE: Using Analytics to Guide Improvement During an Agile-DevOps Transformation
Fannie Mae IT has transformed from a waterfall organization to a lean culture enabled by Agile methods and DevOps. The article discusses how analytics were used, along with challenges in selecting measures and implementing analytics in an Agile–DevOps transformation. Authors: Dr. Bill Curtis, CISQ and Barry Snyder, Fannie Mae.
CISQ Recommendation Guide: Effective Software Quality Metrics for Use in ADM Service Level Agreements
Service Level Agreements (SLAs) are common for Application Development and Maintenance contracts. SLAs have been used to define the relationship between a service provider and customer since the early days of IT outsourcing, yet many of the contracts written, even in the last five years, use fundamentally the same time-based SLAs for software development. SLAs for responding to a high severity ticket or the turnaround on a work request are common. SLAs that contract around the quality of the code produced are rare in contrast. While many of the SLAs are appropriate for infrastructure, the SLAs for application software focus on relatively indirect measurements. Given the tight link between support cost, code quality, and subsequent risk to business, this must change. This paper discusses how to use software quality metrics in SLAs to manage software development and maintenance.
Using Software Measurement in SLAs: Integrating CISQ Size and Structural Quality Measures into Contractual Relationships
As more critical business functions within an organization are automated, IT is under increasing pressure to govern the quality of software received from suppliers, whether they are vendors, outsourcers, or system integrators while at the same time reducing cost. Better governance requires better measurement of application size, stability and quality in all of its manifestations—functional, non-functional or structural, and behavioral. These measures are being incorporated into contracts as the equivalent of Service Level Agreements (SLAs), targets that suppliers must achieve to avoid penalties. In some cases, these SLAs involve productivity targets measured by the amount of business functionality delivered compared to the effort expended. In other cases, they involve targets for the structural attributes of an application such as security or maintainability.
This paper talks about how to integrate size and software quality measures into SLAs.
Sample Acceptance Criteria with CISQ Standardized Metrics
This document contains sample contract language that you can use with software suppliers to establish acceptance criteria for software delivery. Use this language to ensure that the source code delivered is free from critical weaknesses as defined in industry standards for software structural quality. You as the buyer will evaluate the quality of the software prior to accepting delivery.
Contracting Best Practice - Improve Supplier Productivity Using the Automated Function Point Standard
This whitepaper contains sample contract language for software development outsourcing and specifically addresses how to measure development productivity using the Automated Function Point (AFP) standard. AFP is a measure of software size used in cost estimation, progress tracking, productivity measurement, and other software project management activities. The standard is used in contracts to specify a base level of productivity, set a rate invoiced per function point when software is delivered to the customer, ensure the quality of function points delivered, and set incentives for higher productivity.
Contracting Best Practice - Lower Risk and Improve Outcomes with Suppliers by Using Software Structural Quality Standards
This whitepaper contains sample contract language for software development outsourcing and specifically addresses how to lower risk and improve development outcomes by using software structural quality standards in contracts. The software structural quality standards developed by CISQ for Security, Reliability, Performance Efficiency and Maintainability are used to measure a system’s code quality and technical debt. The structual quality standards are used in outsourcing to check standards compliance, conduct systems assessment, measure the quality of delivered code, set acceptable levels of quality for delivered code, and measure software quality over time.
