IT Performance Improvement
Networking and Telecommunications
Share This Article
COSMIC Full Function Points (FFP) and the Worldwide Field Trials Strategy
1.1 Origin of Software Functional Size Measurement
Measuring the functional size of software was originally proposed in 1979 by Albrecht in a communication (Albrecht and Gaffney 1983) describing the results of an effort started in the mid-1970s in one IBM business unit. The overall goal of the work described in Albrecht's original paper was to measure the productivity of software development, as viewed from an economic perspective. His method, function points analysis, was proposed as a specific measure of the "output" of the development process, allowing a comparison of projects where software was developed using different programming languages.
The overall approach described in Albrecht's paper to achieve that goal was to select a specific subset of twenty-two software projects, mostly management information system (MIS) software, completed within this one organization. The measurement of the functional size of the software delivered by these projects consisted of a weighted sum of inputs, outputs, inquiries, and master files. The weights assigned to these items "were determined by debate and trial" (Albrecht and Gaffney 1983). Some extra adjustments (±25%) were provided for "extra complicated" (Albrecht and Gaffney 1983) items. In this communication (Albrecht and Gaffney 1983), Albrecht also offered a set of forms and rules to aid in calculating "function points." Many of the rules proposed originally were based on some aspects of the physical implementation of software.
In 1986, the International Function Point Users Group (IFPUG) was formed to foster and promote the evolution of the function point method. The group used Albrecht's revised version of the original method, using a fifth function type ("interface files") and a set of weight tables. Much work went into the subsequent releases of the method to include rules allowing an interpretation of functionality increasingly independent of the particular physical implementation of software.
The contribution of IFPUG to the field of functional size measurement has been the documentation of the measurement procedure, which enabled a certain level of uniformity in the application of the method. The basic concepts and implicit model of software, though, remained unchanged from what was proposed by Albrecht in 1984. However, it cannot be postulated that the sample of software used by Albrecht in 1984, which was developed between 1974 and early 1979, is representative of all software developed in the 1980s, 1990s, and 2000s.
1.2 A New Generation of Functional Size Measure
A group of experienced software measurers gathered in 1998 to form the Common Software Measurement International Consortium (COSMIC). This group aimed at designing and bringing to market a new generation of software measurement methods. With the support of industrial partners and tapping on the strengths of IFPUG, Mark II (Symons 1991), NESMA (NESMA 1997), and version 1.0 of full function point methods (Abran et al. 2000; Bosch 2003; Büren and Kroll 1999; Kecici et al. 1999; Oligny et al. 1998; Schmietendorf et al. 1999; St.-Pierre et al. 1997a), the group proposed some basic principles on which a new generation of software functional size measurement method could be based (Abran 1999; Synoms 1999; Symons and Rule 1999). In November 1999, the group published version 2.0 of COSMIC FFP, a measurement method implementing these principles. Overall, close to forty people from eight countries participated in the design of this measurement method. Key aspects of the COSMIC FFP measurement method are now highlighted.
1.2.1 Allocation of Functional User Requirements
From the perspective proposed by COSMIC, software is part of a product or service designed to satisfy functional user requirements. From this high-level perspective, functional user requirements can be allocated to hardware, to software, or to a combination of both. The functional user requirements allocated to software are not necessarily allocated to a single unit of software. Often these requirements are allocated to pieces of software operating at different levels of abstraction and cooperating to supply the required functionality to the product or service in which they are included.
In the context of the COSMIC FFP measurement method, which is aimed at measuring the functional size of software, only those functional user requirements allocated to software are considered. (The method may be applicable to size functional requirements for information processing that are allocated to hardware, but this needs further research.) For instance, as illustrated in Figure 1, the functional user requirements in this example are allocated to three distinct pieces, each exchanging data with another through a specific organization: One piece of the software lies at the application level and exchanges data with the software's users, and the second piece lies at the operating system level. In turn, this second piece of the software exchanges data with a third piece lying at the device driver level. This last piece then exchanges data directly with the hardware. The COSMIC FFP measurement method associates the functional user requirements for each piece with a specific layer. Each layer possesses an intrinsic boundary for which specific users are identified.
Figure 1. Example of functional user requirement allocation to different layers. (From Oligny, S., A. Abran, and D. St.-Pierre. 1999. Improving software functional size measurement. Proceedings of the 13th International Forum on COCOMO and Software Cost Modeling. With Permission.)
1.2.2 Representation of Functional User Requirements in Software
The functional user requirements allocated to one or more pieces of software are decomposed into and represented by functional processes. In turn, each functional process is represented by subprocesses. A subprocess can either be a data movement type or a data transform type. Version 2.0 of the COSMIC FFP measurement method recognizes only data movement type subprocesses. Further research is deemed necessary to incorporate data transform subprocess types in the measurement method. In the meantime, an approximating assumption is made that each data movement has an associated (small) amount of data transformation. This assumption, which should be valid for most MIS, real-time, and operating system software, is being tested in field trials (see Section 1.1.3), but will clearly not be valid for algorithm-intensive software as used in; e.g., scientific or engineering domains. The approach is illustrated in Figure 2.
Figure 2. COSMIC representation of functional user requirements within a piece of software. (From Symons, C. R. 1999. COSMIC aims, design principles, and progress. Proceedings of the IWSM'99. With Permission.)
Given the approximating assumption, a COSMIC FFP functional process is defined as a unique and ordered set of data movements that are triggered by an event outside the software being measured, and which, when complete, leave the software in a coherent state with respect to the external event.
1.2.3 COSMIC FFP Software Model
The COSMIC FFP measurement method defines an explicit model of software functionality, derived from the functional user requirements. Based on this explicit model of functionality, relevant functional attributes of software are identified. Their extent and limits are defined, and their generic interactions are described. Taken as a whole, these functional attributes form a generic model for any type of software that is not algorithm rich. The model is illustrated in Figure 3. Four types of data movement-ENTRY, EXIT, READ, and WRITE-are defined within this model. They form the basis for defining the standard unit of functional size. The four types of data movement are presented in Table 1.
Figure 3. COSMIC FFP software model.
Table 1. Definition of COSMIC FFP Data Movements
| Data Movement Type || Definition |
| ENTRY || An ENTRY (E) is a movement of the data attributes found in one data group from the user side of the software boundary to the inside of the software boundary. An ENTRY (E) does not update the data it moves. Functionally, an ENTRY sub-process brings data lying on the user's side of the software boundary within reach of the functional process to which it belongs. Note also that in COSMIC FFP, an entry is considered to include certain associated data manipulation (validation) sub-processes. |
| EXIT || An EXIT (X) is a movement of the data attributes found in one
data group from inside the software boundary to the user side of the software boundary. An EXIT (X) does not read the data it moves. Functionally, an EXIT sub-process sends data lying inside the functional process to which it belongs (implicitly inside the software boundary) within reach of the user side of the boundary. Note also that in COSMIC FFP, an exit is considered to include certain associated data manipulation sub-processes. |
| READ || A READ (R) refers to data attributes found in one data group. Functionally, a READ sub-process brings data from storage, within reach of the functional process to which it belongs. Note also that in COSMIC FFP, a READ is considered to include certain associated data manipulation sub-processes. |
| WRITE || A WRITE (W) refers to data attributes found in one data group. Functionally, a WRITE sub-process sends data lying inside the functional process to which it belongs to storage. Note also that in COSMIC FFP, a WRITE is considered to include certain associated data manipulation sub-processes. |
Source: Abran, A., J.-M. Desharnais, S. Oligny, D. St.-Pierre, and C. Symons. 2003. COSMIC-FFP Measurement Manual version 2.2. Common Software Measurement International Consortium. With Permission.
Version 2.0 of the COSMIC FFP measurement method uses only four base functional components: entry, exit, read, and write. Data manipulation subprocesses are not used as base functional components. The method assumes, as an acceptable approximation for many types of software, that the functionality of this type of subprocess is represented among the four types of subprocesses defined earlier.
In COSMIC FFP, the standard unit of measurement, that is, 1 Cfsu, is defined by convention as equivalent to one single data movement at the subprocess level. Another alternative, initially considered by the COSMIC group, was to define the standard unit of measurement based on the number of data element types moved by a data movement type subprocess. The study of such an alternative was one of the field trial aims.
1.3 Industrial Field Trials
Another key aspect in the development of the COSMIC FFP measurement method is the conduct of a formal field trial period designed to demonstrate that it can withstand being scaled up to industrial software from multiple and varied contexts. The field trial aims were as follows:
- To test for a common, repeatable interpretation of version 2.0 of the measurement manual under widely varying conditions: organizations, domains, development methods, and so forth
- To establish a detailed measurement procedure, where necessary, to ensure repeatable interpretation
- To test that the measures properly represent functionality
- To test that the measurement results correlate with development effort
- To enable a full transfer of technology to the trial "partners"
Starting at the end of 1999, a nine-month period was allocated for conducting formal and organized field trials of the COSMIC FFP measurement method in a significant number of organizations around the world. The data collection was completed in a formal context in a number of organizations: a European aerospace manufacturer, a UK bank with MIS applications, two European telecommunications manufacturers, and an Australian defense software contractor. Additional data were also received from Australia (a defense contractor, a real-time software house, and an aerospace manufacturer) and from Canada (a small software house, a defense contractor, and a public utility organization).
During this period, each participating organization received formal training on the application of the method. Furthermore, multiple items of software were selected from each organization's portfolio and their functional size measured. These results, along with some key data on effort and schedule involved in delivering each software, were registered and centralized for analysis.
Once analyzed, a specific report was prepared for each participating organization, offering: (1) guidelines for applying the method based on the organization's software engineering standards, and (2) some preliminary benchmarking information allowing the organization to leverage its investment in the new data and put it to use immediately. Consolidated highlights from the field trials are reported next, at a high level. Further work is required to formally document such observations.
1.3.2 Preliminary Results Highlights
The high-level preliminary results highlights are reported here using the structure of the field trials aims.
Common, repeatable interpretation. As can be observed in the number and origin of the field trial participants, COSMIC FFP was used under widely varying conditions: organizations, domains, and development methods. To ensure common implementation of the COSMIC FFP method at the trial partners' sites, training material was developed centrally at UQAM (Université du Québec à Montréal) and reviewed by the COSMIC team; "train the trainers" sessions were organized in three countries to ensure common training. The trainers then delivered the same training material to all trial partners. This training included repeatability exercises during the practical sessions.
Establishing detailed procedures, where necessary, to ensure repeatable interpretation. This was performed by each organization participating in the trials, sometimes with the help of the training and support staff. Some organizations already having significant experience in measurement did not have to ask for support.
Test that the measures properly represent functionality. The following were of interest:
- Can the method be applied equally to MIS and real-time software? Observations and feedback from the field trials indicated that it is easy to interpret the model in both domains. Where some parallel measurements were conducted with the IFPUG method, participants from the domain of real-time software reported significant difficulties at classifying functional processes in only one of the three IFPUG transaction types (inputs, outputs, or inquiries).
- Are the four data movement types of equal size? On very preliminary evidence, using one data set, the answer is yes; of course, such results still have to be formally documented and reported.
- The trials also included a participants' questionnaire to collect their feedback on their assessment of the ability of COSMIC FFP to adequately represent the size of the functionality of their software, reported on a Likert scale. Again, the preliminary analysis confirmed a significant alignment of the measurement results with participants' intuitive assessment of software size.
Correlation of the measurement results with development effort. The field trial participants, from both the formal and informal contexts, provided two sets of projects: development projects and maintenance projects. The development projects included eighteen projects from five organizations (sixteen new developments and two enhancements). They were developed on multiple platforms (seven PC, four DEC, two HP, two IBM mainframe, and one Compaq), and completed between March 1999 and May 2000 with a duration from 5 to 75 months.
The other set of maintenance requests provided twenty-one small functional enhancements completed in a single organization. For both sets of data, the correlations with effort were significant. Again, these results still have to be consolidated, formally documented, and reported.
Full transfer of technology to the trial partners. The general feedback is very positive, and the project teams were able to grasp the elements of the method easily, and were enthusiastic about the method. It was also reported that the documentation and effort needed are similar to those for applying the IFPUG method, though there is an extra step to identify layers. Also, one of the participating organizations decided early on to continue CFFP (COSMIC Full Function Points) measurements for all new projects and decided to implement this measurement technique as a standard procedure in its development process (a European participant). Other participants are either preparing or pursuing the deployment of this method within their respective organizations.
From the perspective of the participants in the field trials, the benefit of this approach lies in the availability, at the end of the field trial period, of a database of historical data useful for jump-starting the implementation of the measurement method within their own organizations while respecting the confidentiality of the sources. The benefit to the software engineering community will be the availability of the first functional size measurement method to be developed by an international group of experts and subjected to industrial field trials before finalization.
1.4 Summary and Conclusions
Albrecht proposed the function point method, more than twenty years ago, as a new way to measure the size of software. In the past fifteen years, although the method continues to give useful results for much MIS application software, many practitioners have found that Albrecht's measurement method cannot be applied satisfactorily to non-MIS software.
In 1998, building on the strengths of previous methods, the COSMIC group identified the principles on which the next generation of functional size measurement methods were to be built, offering applicability to MIS, real-time, and operating system software. A year later, the group published COSMIC FFP, a functional size measurement method implementing these principles.
Key aspects of this method were presented in this paper, and industrial field trials are under way to demonstrate that the method can withstand being scaled up to industrial software environments in multiple and varied contexts.
The COSMIC FFP method has achieved a number of firsts: it is the first functional sizing method to:
- Be designed by an international group of experts on a sound theoretical basis
- Draw on the practical experience of all the main existing FP methods
- Be designed to conform to ISO 14143 Part 1
- Be designed to work across MIS and real-time domains, for software in any layer or peer item
- Be widely tested in field trials before being finalized
Significant progress has been made, and the acceptance from those who have tried the method is good in both MIS and real-time environments. There is strong international interest: The COSMIC FFP measurement manual is already available in three languages (English, French, and Spanish), and translation into three additional languages is progressing well (Italian, Japanese, and German). The measurement manual has been downloaded to date in over thirty countries.
And, planning further ahead, the COSMIC FFP method was proposed in early 2000 to ISO/IEC/JTC1 SC7 (Software Engineering Subcommittee) for a new work item to introduce the COSMIC FFP method through the ISO standardizationprocess. In the July 2000 vote, it received an approval rate of over 90%.
In addition, research activities have been initiated to address the following themes:
- Convertibility studies with previous methods, such as FFP V1, Mark II, and IFPUG
- Estimation of functional size with COSMIC FFP, much earlier in the development cycle
- Mapping of measurement rules in the UML (Unified Modeling Language)-based specifications domain
- Measurement of functional reuse using COSMIC FFP
- Development of requirements identification and measurement with the computer-based reasoning (CBR) approach
- Website (standards and publications): www.lrgl.uqam.ca/ffp
- Website (generic information): www.cosmicon.com
The authors of this paper acknowledge the specific contributions of Jean-Marc Desharnais, Denis St.-Pierre, Pam Morris, Roberto Meli, Grant Rule, and Peter Fagg in the elaboration of the COSMIC FFP measurement method, the support of Risto Nevalainen and Jolijn Onvlee, and the thoughtful and generous comments from all the reviewers of the COSMIC FFP measurement manual (Abran et al. 2003b).
From COSMIC Function Points: Theory and Advanced Practices Edited by Reiner Dumke and Alain Abran. New York: Auerbach Publications, 2011.