Measuring the Structural Quality Attributes of a System
The Systems Engineering Quality Attributes relevant today continue to evolve as architectures and technologies change. Today’s software systems are built on an increasingly varied technology stack. In the “old days,” if you knew COBOL, you knew the entire system. A modern business application is a stack of technologies comprised of different languages, different layers, services, OSS components, and frameworks. The interactions between components need to be analyzed to measure the system’s structural quality. When we get into the domain of system-of-systems, with the development of cyber-physical systems and Internet of Things (IoT) technology, with multiple interacting services, the system’s end-to-end structural quality becomes ever more critical.
Complex technology stack
Standards for Measuring Systems Engineering Quality Attributes
Studies show that basic coding errors within an application account for 92% of the total errors one might find when reviewing source code, but only account for 10% of production defects. Yet, software flaws at the technology and system level account for 8% of total errors, and consume over half the effort spent on fixing problems and lead to 90% of the most serious production issues (Object Management Group whitepaper).
The code quality standards from CISQ (the set of coding rulesfor Security, Reliability, Performance Efficiency, and Maintainability) analyze weaknesses found at the code and system level to improve system structural quality. For example, the Reliability standard includes code-level weaknesses such as “Protecting state in multi-threaded environments” and “Safe use of inheritance and polymorphism” and system-level weaknesses such as “Multi-layer design compliance” and “Software manages data integrity and consistency” that need to be addressed at the system or architecture level.
These engineering quality attributes are used to measure structural quality during static analysis testing and also during integration and functional testing to ensure product reliability, security, and performance efficiency.
Emerging Standards for Model Based Systems Engineering
Models are used in systems engineering to test and validate quality attributes and system characteristics. Model Based Systems Engineering (MBSE) is the foundation for cyber-physical systems, IoT, and digital twin technology. The CISQ Quality Measures for MBSE working group is looking at system modeling and how to measure model quality earlier in the development lifecycle when models are less risky and expensive to repair. If a model is weak or vulnerable to exploit, any analyses or decisions based on it are flawed, as well as the system built from the model. The objective of this working group is to develop a standard to detect critical weaknesses in models at the architecture and design level.