David Herron, Co-Founder, David Consulting Group
Bill Curtis, Executive Director, CISQ
After requests from numerous commercial enterprises, the Consortium for IT Software Quality (CISQ) was formed in 2010 by the Software Engineering Institute at Carnegie Mellon University and the Object Management Group (OMG), an international IT standards organization. CISQ was chartered to create international standards for automating the measurement of size and structural quality from software source code. During early executive forums held in Washington DC, Frankfurt, and Bangalore, five measures were selected for initial specification, among which was a request to automate the counting of Function Points from source code based as closely as possible on counting guidelines from the International Function Points User Group (IFPUG).
The David Consulting Group, (DCG) a leader in Function Point analysis, was one of the founding members of CISQ. David Herron, co-founder of DCG, co-author of the Function Point Analysis, and a leader in IFPUG, was selected to head the international team chartered to develop a specification for automating Function Point counting. David’s team included members from North America, Europe, and India.
Function Points were originally defined by Allen Albrecht of IBM back in the 1970s to measure the functionality delivered by a software application. Traditionally Function Points are counting manually by trained Function Point experts. They are often counted from initial program specifications to estimate the size of the system and the effort required to build it. In 1986 the IFPUG was formed to support the Function Point counting community and formalize guidelines for counting Function Points.
While thorough in covering the many issues affecting Function Point counting, IFPUG counting guidelines leave some counting decisions to the judgement of the manual counter. These ambiguities had to be resolved to specify a consistent algorithm for automating the count. As a result, the OMG/CISQ Automated Function Point Sizing specification prioritizes repeatability and consistency over consistency with the IFPUG Function Point counting guidelines.
In certain counting situations, IFPUG guidelines are vague, leaving the interpretation to the judgment of the counter. In order to remove subjectivity, the specification makes explicit decisions about counting techniques in situations where the IFPUG guidelines were vague. Consequently some variation from the IFPUG guidelines were introduced in order to achieve the precision required for automation.
IFPUG functional sizing requires the identification of 5 types of functions; inputs, outputs, inquiries, external (referenced) data files and internal (stored) data. The challenge presented to the CISQ team was determining how to properly identify unique functions.
Identifying input and output functions is technically a simple process. Distinguishing the difference between an output and an inquiry can be a bit more difficult; however, the difference in functional value between the two is negligible. The real challenge is with identifying unique file types. Part of the solution is to collect specific inputs prior to automation. These include, along with the complete source code, a listing of all excluded files and libraries that don’t belong to the application. Additionally data definitions of data bases and flat files along with naming conventions are required. The result is an automated sizing capability that is consistent and verifiable.
How the automated counting tool performed relative to outcomes that were consistent with the IFPUG process of manual counting was a focus of the first round of analytics. It was always understood that a certain degree of calibration would be required. An independent study was performed on a random selection of 20 applications of varying sizes and technical profiles. Each application had been manually counted using current IFPUG guidelines. The objective was to understand the ‘accuracy’ of the automated counts relative to the manual IFPUG counts and to determine how many iterations were required until the automated tool had been properly calibrated for each application. The results were impressive.
The two critical variables that were analyzed were the variance between automated and manual counts and the number of iterations required to realize an acceptable size variance. The automated sizing on the first five applications resulted in a 300% variance between automated and manual counts and required 3 to 4 iterations each to calibrate the tool to result in an acceptable size variance of +/- 10%. After calibrating the tool for the first 10 applications the remaining 10 applications had a lower initial count variance of 13% (down from 300%) and required, on average, 1.5 iterations per application to realize an acceptable size range variance between automated and manual counts of -2.2% to 7.0%. This was a positive indicator that the initial calibrations included ‘standard’ adjustments that could be built into the automated tool and applied on subsequent calibrations thereby reducing the number of iterations and improving accuracy.
The benefits of functional size automation are many. It allows for the increased sizing at the application level providing the opportunity to more effectively and efficiently manage portfolios and better control production support costs. Organizations such as IFPUG, NESMA and Cosmic should continue to advocate and support the continued development of software that automates functional sizing and other software measurement practices.