A Reuse Metrics and Return on Investment Model Jeffrey S. Poulin Joseph M. Caruso Systems Software Development Tools International Business Machines Corporation Abstract: The key to reuse metrics is the accurate reflection of effort ________________________________________________________________ saved. The metrics may then provide input needed to develop a business case ____________________________________________________________________________ that estimates the financial benefit of an organizational reuse program. ________________________________________________________________________ This paper defines a reuse metrics and Return On Investment (ROI) model that ____________________________________________________________________________ distinguish the savings and benefits from those already gained through ______________________________________________________________________ accepted software engineering techniques. _________________________________________ Our experience is that establishing a realistic return on investment on a _________________________________________________________________________ reuse program is essential to inserting reuse into a corporate software _______________________________________________________________________ development process. Clearly stating the potential benefits of reuse in ________________________________________________________________________ financial terms has proven to be a powerful motivator. This paper describes ____________________________________________________________________________ the reuse metrics and return on investment process in place at IBM. Three __________________________________________________________________________ reuse metrics are derived from readily available and observable software data _____________________________________________________________________________ elements. The metrics are used in the return on investment model to estab- ___________________________________________________________________________ lish sound business justification for reuse. ____________________________________________ 1 OVERVIEW ________ Software metrics are an important ingredient in effective software manage- ment. Unfortunately, the lack of an industry standard for reuse metrics is one of the major inhibitors to a coordinated reuse program Ý4¨. Without a means to quantify the practice, development organizations are unable to judge their return on investment and are therefore reluctant to engage in an active reuse program. However, if metrics are used in a return on investment model to verify and demonstrate the substantial benefits of reuse, organizations may be less reticent to realize such potential. With published productivity gains commonly claimed from 20-40% Ý23¨ up to an order of magnitude Ý2¨, organizations should be anxious to take advantage of the increased output and corresponding lower costs reuse offers. The traditional role of metrics is to assist management by quantifying the software process. With an emerging technology, however, metrics must extend beyond their traditional role. Reuse metrics must also encourage the prac- tice of reuse. Most organizations do not practice formal reuse or are reluc- tant to invest in a formal reuse program. Reuse metrics must assist in the technology insertion process by providing favorable process improvement sta- tistics and by placing emphasis on activity conducive to reuse. Establishing a positive financial return on investment that is based on a sound ROI model is a proven motivator. Finally, reuse metrics must establish an effective standard that may be implemented by the development organizations in the enterprise. The data must be easily obtained, meaningful, and possible to implement in a uniform way. In summary, reuse metrics must: 1. quantify reuse, 2. encourage reuse, and 3. standardize reuse counting methods. Following this overview of the motivation and goals of reuse metrics is a discussion on measuring the different classes of reuse. The IBM return on investment model, including a description of tools developed to assist in developing project level reuse business cases, follows a description of the reuse metrics. Finally, the paper presents some related ROI models. USING VERSUS REUSING CODE _________________________ The most fundamental division in how organizations practice reuse is between simply recovering old code for later use (sometimes referred to as "unplanned reuse") and engaging in a formal, planned reuse program. When the reuse ____ decision is made is what distinguishes between these two classes of reuse Ý24¨. Planned reuse starts early in the software lifecycle; evidence of planning for reuse are a thorough requirements study and domain analysis of 2 the problem area. The goal of this additional planning and domain analysis is to identify the factors that normally change in later projects, such as: 1. Hardware or System Software, 2. User, Mission, or Installation, and, 3. Function or Performance. Early design and analysis will result in components that can accommodate these changes without modification. There is an increased level of cost and quality in planned reuse that is paid for once, in the initial development of the component. However, this cost is quickly recovered by subsequent projects that are able to reuse the generalized software and by reduced support costs resulting from having only one base product to maintain. Failing to plan for reuse, however, is the norm in traditional software development. In general, a software product is an original work except for the informal consideration of existing software for use in the new applica- tion. Although this informal use of previously developed software in new applications is widely practiced, the systematic reuse of existing code is not part of traditional software development methods. It is important to distinguish between these two classes, especially when determining the finan- cial return of a reuse program. Table 1 is a summary of how system changes are related to the class of reuse that is in practice. +---------------------------------------------------------------------------+ | Table 1. Relation between system changes and classes of reuse. | +------------------------+-------------------------+------------------------+ | REQUIRED CHANGE | CODE RECOVERY ONLY | WITH PLANNED REUSE | +------------------------+-------------------------+------------------------+ | Hardware or System | Rehosting | Porting | | Software | | | +------------------------+-------------------------+------------------------+ | User, Mission, or | Retargeting | Tailoring | | Installation | | | +------------------------+-------------------------+------------------------+ | Function or Perform- | Salvaging | Assembling | | ance | | | +------------------------+-------------------------+------------------------+ Overview 3 HOW TO MEASURE CODE RECOVERY AND PLANNED REUSE Over several development cycles, planned reuse provides the greatest cost and productivity benefits because there is only one resulting base product to maintain. Using this criteria, it is clear metrics should focus on planned reuse. It is also clear that although code recovery is a form of reuse, the benefits are somewhat limited to the development phase and are much less than with planned reuse. Therefore, the three techniques for code recovery should not factor into reuse metrics.(1) The current state of reuse technology is in assembling reusable components into new applications; metrics must capture this activity. The next most advanced form of reuse is tailoring. Tailoring normally occurs in organiza- tions with mature, formal, reuse programs, and is the result of thorough domain analysis and careful program design. The tremendously successful reuse experiences on the IBM Advanced Automation System (ASS) for the Federal Aeronautics Administration Ý17¨ are an example of tailoring; reuse metrics must also capture this activity. The third form of planned reuse is porting. However, porting is an anomaly for reuse metrics because it is already a standard part of the business plan- ning of products. The resource estimates for a product are normally based on development of the product on one hardware platform or operating system. A relatively nominal amount of resources are then allocated for changes required to adapt to other environments. Although porting is a form of reuse, reuse metrics should not claim these savings because porting is an established part of the business and development planning process. The anomaly occurs because it appears we are able to achieve reuse in-the-large but still have difficulty practicing reuse in-the-small. A further reason to exclude porting is that porting normally involves adapting a minor portion of a large product. To include ported code in reuse metrics would cause misleading results in the form of unrealistically high measures of reuse activity. For example, an organization making small changes to a large base might report levels of "reuse" close to 100%,(2) whereas an organization performing an equal amount of labor on an original project might do very well to to demonstrate reuse levels of 5-10%. To prevent this distortion, IBM does not include code porting in these reuse metrics. Organizations tasked with porting software separately track the amount of porting for which they are responsible. Table 2 is a summary of the measurement status for the various classes of code recovery and planned reuse. --------------- (1) Some organizations choose to track the amount of recovered code in their products to emphasize the amount of "total leverage" gained by copying and modifying old software but code recovery is not included in these reuse metrics. (2) The theoretical maximum is 85%. Ý15¨ 4 +---------------------------------------------------------------------------+ | Table 2. Reuse Techniques and Reuse Metrics | +-----------------+-----------------+---+-----------------+-----------------+ | CODE RECOVERY | MEASURED? | | PLANNED REUSE | MEASURED? | +-----------------+-----------------+---+-----------------+-----------------+ | Rehosting | No | | Porting | Separately | +-----------------+-----------------+---+-----------------+-----------------+ | Retargeting | No | | Tailoring | Yes | +-----------------+-----------------+---+-----------------+-----------------+ | Salvaging | No | | Assembling | Yes | +-----------------+-----------------+---+-----------------+-----------------+ MEASURING REUSE _______________ WHEN TO MEASURE REUSE Differentiating between code recovery, which results in new software to main- tain, and planned reuse, in which products are assembled or tailored from building blocks of reusable software, is important. Next, we define reuse based on "who" uses the component. Central to improving the practice of reuse is the understanding that good design and management is common within development organizations, but is less common between organizations. Communication, which is necessary for the simple exchange of information and critical to sharing software, becomes more difficult as the number of people involved grows and natural organizational boundaries emerge. Therefore, measurements must encourage reuse across these ______ organizational boundaries. A software component is reused when it is used by an organization that did not develop and does not maintain the component. Software development organ- izations vary, but for measuring reuse a typical organization is either a programming team, department, or functional group of about eight people or more. Also, although organizational size is a good indicator of how well communication within and between organizations takes place, functional bound- aries are equally important. For example, a small programming team may qualify as an organization if it works independently. For consistency, the type and size of the reporting organization is consid- ered part of the metrics. This provides an informal check on the flexibility allowed in selecting the most appropriate boundary for the organization. Selection of an inappropriately small boundary would distort the value of the metrics upward and an inappropriately large boundary would result in low reuse values. Changing the organizational boundary between reports would eliminate any possibility for comparisons and evaluation of the reuse program. Therefore, the organization is clearly indicated as part of the reuse measurement and is not changed between reporting periods. Overview 5 REUSING VERSUS USING COMPONENTS The most commonly reported reuse measurement is the reuse percent of a product or a new release of a product. The reuse percent is the portion of the product (normally expressed in lines of code, or LOC) that was saved by reusing software from other products or product releases. In both cases, the effort attributed to reuse comes from completely unmodified reusable compo- nents. Reusable components are easy to identify in new products because the criteria is straightforward; if use of a component saved having to develop a similar component, it is recorded as reuse. However, although a component may be "used" by an organization numerous times, a component can be "reused" by an organization only once. This distinction is critical for accurate estimates of the benefits of reuse and return on investment analysis of projects. Since we expect organizations to develop subprograms for often-needed services and to use components previ- ously developed for a product or previously developed by themselves, we do not credit them with savings resulting from this activity. UNITS OF MEASUREMENT These metrics use traditional "lines of code" to quantify the effort in soft- ware development. Although lines of code have well known deficiencies as a unit of measure Ý12¨, they are also simple to understand, they are easy to collect and compare, and they are difficult to distort. Nonetheless, there are actions that may be taken to increase confidence when using lines of code as a measure. One action is to use a standard code counting tool. Another action, taken with the metrics in this paper, is to use metrics derived from ratios or percentages of effort, and thereby eliminate the units of "LOC" from the metrics. The rationale is that if the units are a good indicator of overall effort Ý25¨, then the portion of units reused is a good indicator of effort saved. 6 REUSE METRICS _____________ OBSERVABLE DATA _______________ The reuse metrics presented in the next section are calculated from the fol- lowing observable data elements which have been in use within IBM for many years Ý11¨. Observable data may usually be directly measured from the product. For example, the different classes of source instructions are directly measurable. Observable data may also be historical data, collected for a variety of reasons related to managing the software development process. Costs for software development and statistical error rates are examples of historical data. Detailed descriptions of each of the required observable data elements follow the summary in Table 3. Shipped Source Instructions (SSI). The total lines of code in the product source files. New and Changed Source Instructions (CSI). The total lines of code new or changed in a new release of a product. Reused Source Instructions (RSI). The total lines not written but included in the source files. RSI includes only completely unmodified reused software components. Source Instructions Reused By Others (SIRBO). The total lines of code that other products reuse from a product. Software Development Cost. A historical average required for estimating reuse cost avoidance. Software Development Error Rate. A historical average required for esti- mating maintenance cost avoidance. Software Error Repair Cost. A historical average required for estimating maintenance cost avoidance. It is absolutely essential when acquiring the observable data elements, espe- cially RSI, to recognize when reuse actually saves effort. This requires the analyst to distinguish reuse from normal software engineering practices (e.g., structured programming) and to eliminate implementation-dependent options effecting the observable data. (e.g., static versus dynamic sub- program expansion). For example, the programmer's decision to implement a system service as a subroutine, which is expanded at runtime, or as a macro, which is expanded at compile time, should not affect the reuse metric. Reuse Metrics 7 +---------------------------------------------------------------------------+ | Table 3. Observable data | +------------------+------------------+------------------+------------------+ | DATA ELEMENT | SYMBOL | UNIT OF MEASURE | SOURCE | +------------------+------------------+------------------+------------------+ | Shipped Source | SSI | LOC | Direct Measure- | | Instructions | | | ment | +------------------+------------------+------------------+------------------+ | Changed Source | CSI | LOC | Direct Measure- | | Instructions | | | ment | +------------------+------------------+------------------+------------------+ | Reused Source | RSI | LOC | Direct Measure- | | Instructions | | | ment | +------------------+------------------+------------------+------------------+ | Source | SIRBO | LOC | Direct Measure- | | Instructions | | | ment | | Reused by Others | | | | +------------------+------------------+------------------+------------------+ | Software | Cost per LOC | $/LOC | Historical data | | Development Cost | | | | +------------------+------------------+------------------+------------------+ | Software Devel- | TVUA rate | TVUA/LOC | Historical data | | opment | | | | | Error Rate | | | | +------------------+------------------+------------------+------------------+ | Software Error | Cost per TVUA | $/TVUA | Historical data | | Repair Cost | | | | +------------------+------------------+------------------+------------------+ SHIPPED SOURCE INSTRUCTIONS (SSI) Shipped Source Instructions (SSI) are the number of non-comment instructions in the source files of a product. SSI does not include Reused Source ___ Instructions (RSI). A call to a reusable part counts as one SSI. When reporting reuse measures for development organizations, SSI are the source instructions the organization maintains. SSI are the lines of code actually written by someone for a product. CHANGED SOURCE INSTRUCTIONS (CSI) Changed Source Instructions (CSI) are the number of non-comment source instructions that are new, added, or modified in a product release. CSI does _______________________ not include Reused Source Instructions (RSI) or unchanged base instructions ___ from prior releases of the product. CSI does include source instructions from partially modified components that, had they not been modified, would have been considered "reused." A call to a reusable part counts as one CSI. CSI are the lines of code someone actually had to change or add for a product release. 8 REUSED SOURCE INSTRUCTIONS (RSI) Reused Source Instructions (RSI) are source instructions shipped but not developed or maintained by the reporting organization. RSI are from com- pletely unmodified components normally located in a reuse library. Base instructions from prior releases of a product and source instructions from partially modified parts are not RSI. Source Instructions from a reused part count once per organization inde- pendent of how many times one calls or expands the part. There are two reasons for this: 1. Metrics must accurately reflect effort saved. Programmers use subrou- tines and macros because many functions are repetitive. Their use is standard programming practice; not reuse. 2. Metrics must be implementation independent. The choice of using a sub- routine versus a macro is a design decision usually resulting from many considerations well outside the realm of reuse. The decision to use macros should not be made because multiple in-line expansions increase the amount of reuse reported on a project. SOURCE INSTRUCTIONS REUSED BY OTHERS (SIRBO) Source Instructions Reused by Others (SIRBO) are an organization's source instructions reused by other organizations and indicates how much an organ- ization contributes to reuse. SIRBO is important because for a reuse program to succeed organizations must not only reuse software but help other organ- izations reuse software. SIRBO not only measures the parts contributed for use by others but also the success of those parts. Organizations writing successful reusable parts will have a very high SIRBO because SIRBO increases every time another organization reuses their software. This encourages organizations to generate high quality, well-documented, and widely- applicable reusable components. SIRBO are a summation over all parts an organization contributes to a library of: cabove Example: An organization's contributions to a reuse library are: a 10kloc ________ module in use by 5 other departments, a 25kloc macro in use by 6 other departments, and an unused 75kloc macro. The organization's SIRBO is: < (5%depts. times 10%kloc) plus (6%depts. times 25%kloc)> Reuse Metrics 9 cabove SIRBO is independent of the number of times the same organization invokes or calls the part. The same rules apply that apply for counting RSI; use of a reusable part saved having to develop the part one time, not one time for every call to the part. SIRBO is also a dynamic measure. As more organiza- tions reuse the components, the SIRBO of the donating organizations increases. SOFTWARE DEVELOPMENT COST (COST PER LOC) To determine the financial benefit of reuse, the cost of developing software without reuse must be known. This new software development cost is a histor- ical average that is generally available from the financial planners and man- agement of the organization. The new software cost is normally found by adding all the expenses of the organization, including overhead, and dividing by the total output (in LOC) of the organization. SOFTWARE DEVELOPMENT ERROR RATE (TVUA RATE) No amount of testing, inspection, or verification can guarantee that a product is released without errors. Although emphasis on quality and strict adherence to development processes leads to better products, errors are inev- itably revealed once the product is released to the marketplace. Every development organization has a historical average number of errors, or TVUAs _____ ("Total Valid Unique Program Analysis Reports") uncovered in their products. Note that software components built for reuse are usually designed and tested to standards that are much more strict than those for normal program product components. The additional cost of this effort is justified by the savings gained when other organizations are not required to develop and maintain a similar component. The additional testing has an additional benefit in increasing the quality of the component, so we may expect fewer errors from reusable code. SOFTWARE ERROR REPAIR COST (COST PER TVUA) To quantify the benefit of the increased quality of reusable components, we need the historical average cost of maintaining components with traditional development methods. As with software development cost, this figure is gen- erally available from financial planners and management in the organization. The figure is found by taking the sum of all costs, including overhead, of repairing latent errors in software maintained by the organization and dividing by the number of errors repaired. 10 Although software maintenance includes enhancements to products, the cost of increasing function is not included in the software error repair cost. DERIVED METRICS _______________ The observable data elements combine to form three derived reuse metrics. The first two metrics indicate the level of reuse activity in an organization as a percentage of products and by financial benefit. The third metric includes recognition for writing reusable code. The three metrics, which are summarized in Table 4, are Ý21¨: 1. Reuse Percent; the primary indicator of the amount of reuse in a product or practiced in an organization. Reuse Percent is derived from SSI, CSI, and RSI. 2. Reuse Cost Avoidance; indicator of reduced total product costs as a result of reuse in the product. Reuse Cost Avoidance is derived from SSI, CSI, RSI, TVUA rates, software development cost (cost per LOC), and maintenance costs (Cost per TVUA). 3. Reuse Value Added; an indicator of leverage provided by practicing reuse and contributing to the reuse practiced by others. Reuse Value Added is derived from SSI, RSI, and SIRBO. +---------------------------------------------------------------------------+ | Table 4. Derived Metrics | +---------------------------+-------------+--------------------+------------+ | METRIC | SYMBOL | DERIVED FROM: | UNIT OF | | | | | MEASURE | +---------------------------+-------------+--------------------+------------+ | Reuse Percent | Reuse% | | Percent | | o for products | | o SSI, RSI | | | o product releases | | o CSI, RSI | | | o organizations | | o SSI, RSI | | +---------------------------+-------------+--------------------+------------+ | Reuse Cost Avoided | RCA | SSI or CSI, RSI, | Dollars | | | | Cost/LOC, | | | | | TVUA/LOC, | | | | | Cost/TVUA | | +---------------------------+-------------+--------------------+------------+ | Reuse Value Added | RVA | SSI, RSI, SIRBO | Ratio | +---------------------------+-------------+--------------------+------------+ REUSE PERCENT The purpose of this measurement is to indicate the portion of a product, product release, or organizational effort that can be attributed to reuse. Reuse Percent is an important metric because it is simple to calculate and it is easy to understand. Unfortunately, it is also easy to misrepresent without a supporting framework. Many companies report their reuse experi- ences in terms of "reuse percent" but few describe how they calculate the values. They commonly include informal reuse in the value, making it diffi- Reuse Metrics 11 cult to assess actual savings or productivity gains. Since RSI is clearly defined, the Reuse Percent metric is a reasonable reflection of effort saved. Reuse Percent of a product The Reuse Percent of a product (or first release of a product) is: Reuse Percent equals > times 100 percent Product Example: If a product consists of 65kloc SSI and an additional 35kloc ________________ from a reuse library, then the Reuse Percent of the product is: Reuse Percent equals <35%kloc over <35%kloc plus 65%kloc>> times 100 percent equals 35 percent Reuse Percent of a product release For a new release of a product, RSI comes from reusable components that are completely new to the product. A call to a component used in a previous release is a new or changed source instruction (CSI). The Reuse Percent of a product release is: Reuse Percent equals > times 100 percent Product Release Example: If a release of a product consists of 7k CSI plus 3k ________________________ "new" RSI from a reuse library, then the Reuse Percent for this product release is: Reuse Percent equals <3%kloc over <3%kloc plus 7%kloc>> times 100 percent equals 30 percent Reuse Percent for an organization All software developed and maintained by an organization is the organiza- tion's SSI. Any software used by the organization but maintained elsewhere is RSI. The Reuse Percent of an organization is: Reuse Percent equals > times 100 percent Organizational Example: If a programming team develops and maintains 80k SSI _______________________ and the team additionally uses 20k RSI from a reuse library, then the Reuse Percent for the team is: 12 Reuse Percent equals <20%kloc over <20%kloc plus 80%kloc>> times 100 percent equals 20 percent REUSE COST AVOIDANCE (RCA) The purpose of this measurement is to quantify the financial benefit of reuse. This is a particularly important metric because it shows the tremen- dous return on investment potential of reuse. Because RCA is a key metric for performing return on investment (ROI) analysis of reuse programs, RCA helps with the insertion of reuse technology. Reusing software requires less resources than new development, but is not free. The developer still must search for, retrieve, and assess the suit- ability of reusable components before finally choosing the appropriate part for integration into the product. Although reuse requires this effort to understand and integrate reusable parts, studies show that the cost of this effort is only about 20% of the cost of new development Ý26¨, Ý6¨. Based on this relative cost of reuse, the financial benefit attributable to reuse _______________________ during the development phase of a project is: cabove However, development is only about 40% of the software life cycle Ý9¨; sig- nificant maintenance benefit also results from reusing quality software. This benefit may be quantified as the cost avoidance of not fixing errors (TVUAs) in newly developed code Ý7¨. This savings is: cabove The total Reuse Cost Avoidance is then: cabove cabove Example: If an organization has a historical new code development cost of ________ $200 per line, a TVUA rate of 1.5/kloc, and a cost to fix a TVUA of $43k, then the RCA for integrating 20k RSI into a product is: Reuse Metrics 13 labove labove labove labove REUSE VALUE ADDED (RVA) The previous two metrics measure how much organizations reuse software. It is also important to motivate contributing software to reuse. The purpose of the Reuse Value Added is to provide a a metric that reflects positively on organizations that both reuse software and help other organizations by devel- oping reusable code. RVA is a ratio, or productivity index; organizations with no involvement in reuse have an RVA=1 . An RVA=2 indicates the organization is twice as effective as it would be without reuse. In this case the organization was able to double its productivity either directly (by reusing software) or indirectly (by maintaining software that other organizations are using). Therefore, the total effectiveness of a development group is: equals <(SSI plus RSI) plus SIRBO> over Example: A programming team maintains 80kloc and uses 20 kloc from a reuse ________ library. In addition, five other departments reuse a 10kloc module the pro- gramming team contributed to the organizational reuse library. The RVA of the programming team is: labove ' ' labove over 80%kloc equals 1.9> In this example, the RVA of 1.9 indicates the programming team is 1.9 times more effective than the team would be without reuse. Some organizations organize to obtain the most benefit possible from reuse. For example, the Mid-Hudson Valley Programming Laboratory and the IBM Federal Systems Company in Rockville, Maryland dedicate programming teams to develop and maintain shared software or site-wide reuse libraries. Corporate parts centers, such as the Boeblingen software center, also develop and maintain software for IBM-wide use. Experience shows that although these types of teams may have modest values for the Reuse Percent metric, they have extremely high values for the RVA metric. This high RVA indicates the tremen- dous programming leverage they provide to their organizations. 14 REUSE RETURN ON INVESTMENT __________________________ The reuse metrics quantify, encourage, and standardize counting methods for projects and development organizations. However, a more detailed accounting of costs is required when developing reuse business cases. Furthermore, a distinction must be made between project level business cases and the busi- ness case for the entire corporation. The corporate business case includes overhead costs which are not associated with any individual project. Both types of business cases are effective tools for promoting reuse throughout the corporation. PROJECT LEVEL ROI _________________ Product managers are often reluctant to invest in a comprehensive reuse program since the benefits of writing reusable code will often accrue to projects outside their realm of responsibility. Therefore, any definition of return on investment should include benefits which other projects reap as a result of efforts by the initiating project. A straightforward formulation for return on investment would be: ROI = RCA + ORCA - ADC Where: ROI is the Return on Investment that would occur in infinite time RCA is the Reuse Cost Avoidance for the initiating project ORCA is the Reuse Cost Avoidance for other projects benefiting from the reus- able code written by the initiating project ADC is the Additional Development Cost of writing reusable code to the ini- tiating project For simplicity, the above formula ignores the time value of money. RCA has previously been defined as the sum of Development Cost Avoidance and Service Cost Avoidance to the initiating project. ORCA is similar to RCA in computa- tion but it is based on SIRBO rather than RSI. ORCA is the sum of the RCA for each benefiting project: times <(1 - relative%cost%of%reuse sub i)>> Reuse Return on Investment 15 cabove ' ' cabove cabove ' ' cabove times times > Where: SIRBO sub i is the Source instructions reused by project i n is the number of projects reusing code written by the initiating project relative%cost%of%reuse sub i is the cost of integrating reusable code for project i relative to the cost of creating a new line of code, which is taken as 1 New%Code%Cost sub i is the cost per line of code for project i TVUA%Rate sub i is the number of TVUAs per kloc for project i Cost%per%TVUA sub i is the cost to repair an TVUA for project i Additional Development Cost is defined as: <(relative%cost%of%writing%reuse-1)> cabove < > Where: is the cost of writing reusable code relative to the cost of writing code for one time use, which is taken as 1. Usually this number is assumed to be 1.5. is the kloc of code intended for reuse that was written by the initi- ating project 16 Example: Assume an organization has a Reuse Cost Avoidance of $4.49 million ________ as in the previous example. Additionally the organization has developed 15 kloc of reusable code for other projects with a relative cost of writing reuse of 1.5. Two projects (Project A and B) have already agreed to reuse some components and have the following data: +---------------------+----------+----------+----------+----------+---------+ | PROJECT | SIRBO | COST/ | REL COST | TVUA | COST/ | | | | LOC | | RATE | TVUA | +---------------------+----------+----------+----------+----------+---------+ | A | 2 | 200 | .2 | 2 | 10 | +---------------------+----------+----------+----------+----------+---------+ | B | 8 | 80 | .3 | .5 | 18 | +---------------------+----------+----------+----------+----------+---------+ Reuse Return on Investment 17 Then labove <= $4.49%million plus ORCA minus 15 times (1.5-1) times $200%per%line> labove <= $4.49%million plus ORCA minus $1.5%million> labove <= $2.99%million> labove labove labove labove labove <= $3.87%million> Although the computations in the above example are not complex, the number of computations can make the final figure for ROI prone to error. For this reason a menu driven program was written which provides a business template for project level ROI analysis. An input menu is presented to the user with defaults for many of the input parameters previously described. The user alters parameters as required and enters the unique project level data. This input menu with the example data is shown in Figure 1. After pressing the enter key, the output shown in Figure 2 is presented to the user. In addition to providing the ROI for the project reuse program, all reuse metrics are displayed. The automation of the reuse metrics and ROI computations greatly assists the software project manager in developing jus- tification to implement a successful reuse program at the project level. CORPORATE LEVEL ROI ___________________ A corporate level reuse program may consist of many project-level reuse pro- grams. The benefits that accrue to the corporation are the sum of the bene- fits to the individual projects (RCA). From a cost perspective, however, there are start-up activities which must be considered. For instance, a group of people might be put in place to promote reuse programs across the corporation. Tools might be developed to store, search for, and retrieve reusable parts. A significant amount of disk storage may have to be pur- chased in order to store the reusable parts. Parts may also be purchased from outside vendors rather than being locally developed. Additionally, these start-up activities may require a significant period of time before the reuse infrastructure begins to yield productivity and quality savings. For this reason, a corporate level ROI should take into account the time value of money. The most common way to express this ROI is through the internal rate of return (IRR) approach where k is selected so that: 18 +---------------------------------------------------------------------------+ | +---------------------------------------------------------+ | | | | | | | Fill in all fields and then Press ENTER. | | | | | | | | SSI/CSI . . . . . . . . . . . . . 100_____ KLOC | | | | RSI . . . . . . . . . . . . . . . 20______ KLOC | | | | SSI/CSI written for reuse . . . . 15______ KLOC | | | | Cost/LOC. . . . . . . . . . . . $ 200_____ | | | | Relative cost of reuse (RCR). . . .2______ (0-1) | | | | Relative cost of writing reuse . 1.5_____ (1-2) | | | | APAR rate . . . . . . . . . . . . 1.5_____ TVUA/KCSI | | | | Cost/TVUA . . . . . . . . . . . $ 43______ K | | | | | | | | Data from other projects using your code: | | | | Project SIRBO(K) Cost/LOC RCR APAR/Rate Cost/TVUA | | | | A________ 2_______ 200_____ .2_ 2_______ 10______ | | | | B________ 8_______ 80______ .3_ .5______ 18______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | _________ 0_______ 100_____ .2_ ________ 20______ | | | | | | | +---------------------------------------------------------+ | +---------------------------------------------------------------------------+ Figure 1. Business Template Program = over <1 + k> plus over <(1 + k) sup 2> plus .%.%. plus over <(1 + k) sup n> Where: C sub 0 is the corporate reuse start-up costs R sub i is the revenue (savings) in year i C sub i is the costs in year i n is the number of years for which revenues are to be considered Reuse Return on Investment 19 +---------------------------------------------------------------------------+ | +---------------------------------------------------------+ | | | | | | | SSI/CSI . . . . . . . . . . . . . 100.00 KLOC | | | | * Reuse Percent . . . . . . . . . . 16.67 % | | | | RSI . . . . . . . . . . . . . . . 20.00 KLOC | | | | Percent SSI/CSI written for reuse 15.00 % | | | | KLOC of SSI/CSI written for reuse 15.00 KLOC | | | | Additional development cost (ADC) $1500.00 K | | | | SIRBO . . . . . . . . . . . . . . 10.00 KLOC | | | | * Reuse Value Added (RVA) . . . . . 1.30 | | | | Development cost avoidance (DCA). $3200.00 K | | | | Service cost avoidance (SCA). . . $1290.00 K | | | | * Reuse Cost Avoidance (RCA). . . . $4490.00 K | | | | | | | | (Savings for other projects) | | | | Development cost avoidance (ODCA) $768.00 K | | | | Service cost avoidance (OSCA) . . $112.00 K | | | | * Cost avoided by others (ORCA) . . $880.00 K | | | | | | | | Total RCA (RCA + ORCA). . . . . . $5370.00 K | | | | -> ROI (RCA+ORCA-ADC). . . . . . . . $3870.00 K | | | | | | | +---------------------------------------------------------+ | +---------------------------------------------------------------------------+ Figure 2. Output from Business Template Program k is the discount rate If we let k sub 0 equal the corporation's cost of capital then we can define the related net present value (NPV) metric which measures the value of the corporate reuse program in dollars: NPV = <- C sub 0> plus over <1 + k sub 0> plus over <(1 + k sub 0) sup 2> plus .%. plus over <(1 + k sub 0) sup n> An example of a corporate level ROI is displayed in Figure 3. In this example, business planning practices dictate that benefits are considered for 5 years into the future. The hypothetical ROI is a hefty $20 million net present value and with 104% internal rate of return. Although this ROI seems extraordinarily high by conventional business standards, it does not reflect the risk inherent in many of the underlying assumptions of the ROI. For instance, assumptions must be made about the growth of reuse over time, the relative cost of reuse, the cost of writing reusable code, the amount of vendor purchased reusable code versus code written internally within the cor- poration, etc. Since these assumptions are usually based on historical aver- 20 ages which can always be debated, it is important that these assumptions be varied to explore their effect on the ROI. Reuse Return on Investment 21 +------------------------+--------------------------------------------------+ | Benefits/Costs | Year | | +--------+-------+--------+-------+--------+-------+ | | Start | 1 | 2 | 3 | 4 | 5 | +------------------------+--------+-------+--------+-------+--------+-------+ | Benefits | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Total DCA | 0 | 6763 | 11870 | 17990 | 23713 | 30447 | +------------------------+--------+-------+--------+-------+--------+-------+ | Total SCA | 0 | 1646 | 877 | 395 | 169 | 71 | +------------------------+--------+-------+--------+-------+--------+-------+ | Total RCA | 0 | 8409 | 12747 | 18385 | 23882 | 30518 | +------------------------+--------+-------+--------+-------+--------+-------+ | | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Support Costs | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Reuse Technology | 85 | 88 | 65 | 41 | 29 | 30 | | Center | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Site Champions | 241 | 500 | 598 | 717 | 858 | 1024 | +------------------------+--------+-------+--------+-------+--------+-------+ | Disk Storage | 0 | 182 | 321 | 481 | 621 | 779 | +------------------------+--------+-------+--------+-------+--------+-------+ | Total Support | 326 | 770 | 984 | 1239 | 1508 | 1833 | +------------------------+--------+-------+--------+-------+--------+-------+ | | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Other Costs | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Total ADC | 0 | 3730 | 5299 | 6247 | 6009 | 5008 | +------------------------+--------+-------+--------+-------+--------+-------+ | Tool Development | 1200 | 1200 | 1200 | 1200 | 1200 | 1200 | +------------------------+--------+-------+--------+-------+--------+-------+ | Vendor Parts | 2400 | 1450 | 1450 | 1450 | 1450 | 1450 | +------------------------+--------+-------+--------+-------+--------+-------+ | | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Net Savings per year | -3926 | 1259 | 3814 | 8249 | 13715 | 21027 | +------------------------+--------+-------+--------+-------+--------+-------+ | Present Value | -3926 | 1049 | 2649 | 4774 | 6614 | 8450 | +------------------------+--------+-------+--------+-------+--------+-------+ | | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Net Present Value | $19,610| | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ | Internal Rate of | 104% | | | | | | | Return | | | | | | | +------------------------+--------+-------+--------+-------+--------+-------+ Figure 3. Example Corporate Level ROI (K$) 22 Notice that the sample ROI includes costs for tool development and support costs such as for personnel responsible to communicate the benefits of reuse thereby helping to spread reuse throughout the corporation and individual sites. These costs are not incurred by any individual project but must be included in the corporate ROI. Reuse Return on Investment 23 RELATED WORK ____________ Since quantifying a process is an essential step in assessing its success and effectiveness, there are several methods currently in use. However, the metrics in this paper are unique in the attention given to the definition of RSI and in attempting to present reuse as "real effort saved." Although Ý3¨ differentiates between reuse within an organization and reuse from sources external to the organization, no other paper addresses how to measure the classes of reuse nor do they provide a concentrated definition of RSI. There are other reports on reuse measurements available. Although reuse percent is most common metric, the majority of published methods focus on financial analysis. This is because the cost benefit of reuse is a highly convincing measure for project managers and planners. In 1988 Gaffney and Durek Ý7¨ published a comprehensive model addressing business case analysis of reuse. They premise their model on the need to amortize the cost of the reuse program, including the additional cost to build reusable components, across all projects using the component. When doing cost benefit analysis for software reuse, one needs to consider the long-term benefits and associated costs, which apply to every project using the component. Taking a short term approach to these costs, the additional cost of developing reusable components greatly overemphasizes their cost rel- ative to their benefit. They argue that a better economic estimate includes the number of times the component is reused. Gaffney and Durek define the cost of software development with reuse relative to the cost of software development with all new code. Their equation for this relative cost, C is: ________________ C equals R sub U times 1 plus R times (b plus ) Where: RÝU¨ is the portion of non-reused (newly written) code. R is the portion of reused code. b is the relative cost of integrating reused code. E is the relative cost of creating reusable code. n is the number of uses over which the reused code is to be amortized. As with the Reuse Value Added Metric, if the R value is zero (e.g., there is no reuse), the value of C is equal to 1. The equation also shows that for a reusable component to pay off, the component must be reused at least two times. The second metric defined by Ý7¨ is the Productivity Index, PI. PI is the ___________________ productivity relative to the productivity of creating the software product without reuse, and is defined as the inverse of C: 24 Productivity%Index equals <1 over C> A PI of 2.5 indicates the measured project was 150% more productive in terms of cost than the project would have been without reuse. Observing that the coefficient b varies depending on the type of reuse _ (recovering, porting), Ý17¨ extends the above model by defining additional values of b for the different types of reuse, R. C becomes C plus R sub i times b sub i for each (R sub i, b sub i) . For example: RÝ0¨ is the portion of reused code from other sources. bÝ0¨ is the relative cost of integrating reused code from other sources. RÝR¨ is the portion of code requiring re-engineering (copied and modified code). bÝR¨ is the relative cost of integrating re-engineered code. An additional cost-benefit model of reuse is presented in Ý19¨. The NATO model consists of listing the major benefits and costs of reuse, then applying time-value of money formulas to adjust for future values. The bene- fits are: Saving due to Avoided Cost, SÝR¨. The sum of costs avoided each time the _________________________________ component is reused. Service Life, L. The useful lifetime, in years, of the component (how _________________ long it will be maintained in the library). Demand. The number of times the component is likely to be reused during _______ its service life, L. The costs of reuse are: Cost to Reuse, CÝR¨. The cost incurred each time the component is reused ____________________ including identification, retrieval, familiarization, modification, and installation Ýthis is the relative cost of reuse¨. Accession Time, TÝA¨. The amount of time likely to elapse between the _____________________ decision to acquire the component and its availability in the library. Accession Cost, CÝA¨. The cost to add the component to the library _____________________ including obtaining raw material, developing the complete component, and installing it in the library. Maintenance Cost, CÝM¨. The cost to maintain the component in the _______________________ library, including maintenance and change distribution. The Net Saving to the Reuser (NSR) is the difference between the savings due to avoided cost and the cost to reuse: NSR equals S sub R minus C sub R Related Work 25 The Net Saving to the Supported Program (NSP) is the total savings from all instances of reuse of a component less the accession and maintenance costs. The total savings from all instances of reuse is the NSR multiplied by the number of reuses, N: NSP equals (NSR times N) minus (C sub A plus C sub M) Although the NATO model continues with adjustments for the time value of money, there is little guidance on collecting the data required for the model. For example, there are no details on accounting for the savings due to avoided cost, how to estimate the number of times a component is likely to be reused, nor estimating the service life of a product that does not "wear out." Finally, where data is not available or the analyst feels mitigating factors effecting the risk of the reusable product exist, statistical dis- tributions and estimates of risk factors may be used to adjust the inputs. An additional cost benefit model for reuse is presented in Ý13¨. The model begins with a thorough explanation of the cost benefit analysis procedure for reuse, consisting of five steps: 1. Select the alternatives to be analyzed. 2. Determine the organizational priorities, goals, requirements, and busi- ness strategies that will influence the investment decision. 3. Determine the time period for the analysis. 4. Identify and quantify the costs and benefits of the alternatives identi- fied in step 1. 5. Perform the cost benefit analysis. Step four in the above procedure is the key to the model; Hancock provides an extensive listing of each cost/benefit factor contributing to both reusing and producing reusable software. Once these factors have been tallied, the analyst applies the following Present Value (PV) equation over the time period (n years) identified in step 3 of the model: PV = <( minus )> over <(<1 plus d>)> sup t where d is the discount rate, BÝt¨ is the value of benefits in year t, and _ ____ _ CÝt¨ is the value of costs in year t. ____ _ 26 FUTURE WORK ___________ Future work includes further validation of these measures and ROI model. This includes comparing the predicted versus actual costs avoided and com- paring increased productivity rates with the values calculated in the metrics and the ROI model. Although the model uses industry experience for default values in the equations, actual values are used where available. For example, actual costs to develop new code and standard software development defect and maintenance data are usually known, and defect data (TVUAs) for reusable components are routinely gathered. However, data needs to be con- tinually collected and studied to compare with industry experience and to maintain the accuracy of the model. This paper discusses reuse measurements for software only. Future work will include methods to quantify reuse in areas other than software (e.g., Design, Test Case, Information Development). Many other areas for study remain. Much of the relative cost data is based on industry averages and needs to be captured, tracked, and validated. Other data of interest relate to the reuse process or to the success of the reuse library, including: The cost of developing Reuse components (estimated at 1.5-2 times normal development costs). The cost of certifying and testing Reuse components. Project costs for Reuse, including the cost to maintain a reuse library and to staff the support personnel. Future Work 27 CONCLUSION __________ Sound business decisions based on accurate measurements are essential to the management of any process. This paper introduces three metrics and a return on investment model for software reuse. The metrics are: Reuse Percent, Reuse Cost Avoidance, and Reuse Value Added. The metrics rely on easy to collect data, provide reasonable representations of reuse activity, and encourage reuse. Most importantly, the metrics provide reliable input to the corporate reuse ROI model, where the benefits attributed to reuse are carefully defined. With emerging technologies such as software reuse, the value of metrics and ROI analysis goes beyond the traditional benefits of assuring the quality of reusable components, demonstrating the success of a program, and improving the ability to plan and predict for future projects. They also serve to encourage reuse by providing feedback on the results of a reuse program and by highlighting the benefits of a organizational reuse effort. The ROI business template program has been made available to all sites to help convince project management that a formal reuse program should be imple- mented. The program has been well received and has been effectively put to use in a number of real cases. The corporate level ROI has been used by the IBM Reuse Technology Support Center to evaluate the relative benefit of formal reuse to other technologies which will improve programmer productivity and quality. Because this relative benefit is high, the corporate reuse program has been able to more effectively compete for corporate funds desig- nated for the promotion of new technologies. In addition, the analysis has been reviewed through various levels of executives to make them aware of the potentially large returns for reuse and the importance of increased focus on formal reuse programs. 28 CITED REFERENCES ________________ Ý1¨ Albrecht, A.J, "Measuring Application Development Productivity," in Pro- ____ ceedings of the Joint IBM/SHARE/GUIDE Application Symposium, October ____________________________________________________________ 1979, pp. 83-92. Ý2¨ Banker, Rajiv D. and Robert J. Kauffman, "Reuse and Productivity in an Integrated Computer Aided Software Engineering (ICASE) Environment: An Empirical Study at the First Boston Corporation," Technical Report, 10 _________________ July 1991. Ý3¨ Banker, Rajiv D., Robert J. Kauffman, Charles Wright, and Dani Zweig, "Automating Output Size and Reusability Metrics in an Object-Based Com- puter Aided Software Engineering (CASE) Environment," Technical Report, _________________ 25 August 1991. Ý4¨ Bowen, Gregory M, "An Organized, Devoted, Project-Wide Reuse Effort," Ada Letters, Volume 12, No. 1, January/February 1992, pp. 43-52. ____________ Ý5¨ Dreger, J.B. Function Point Analysis, Prentice-Hall, Englewood Cliffs, ________________________ NJ, 1989. Ý6¨ Favaro, John, "What price reusability? A case study," Ada Letters, Vol. ____________ 11, No. 3, Spring 1991, pp. 115-24. Ý7¨ Gaffney, John E., Jr. and Thomas Durek, "Software Reuse- Key to Enhanced Productivity; Some Quantitative Models," Software Productivity Consor- _____________________________ tium, SPC-TR-88-015, April 1988. ____________________ Ý8¨ Gaffney, J.E., Jr. and Durek, T.A., "Software Reuse- Key to Enhanced Productivity: Some Quantitative Models," Information and Software Tech- ______________________________ nology, 31:5, June 1989. _______ Ý9¨ "Software Engineering Strategies," Strategic Analysis Report, Gartner __________________________ Group, Inc., April 30, 1991. Ý10¨ Hayes, W.E., "Measuring Software Reuse," IBM Internal Document, Number ______________________ WEH-89001-2, 2 October 1989. Ý11¨ "Corporate Programming Measurements (CPM)," V 4.0, IBM Internal Docu- __________________ ment, dated 1 November 1991 _____ Ý12¨ Firesmith, D.G., "Managing Ada projects: the people issues," Proceedings ___________ of TRI Ada '88, Charleston, WV, USA 24-27 Oct. 1988, pp. 610-19 _______________ Ý13¨ Hancock, Debera R., "Reuse Investment Decisions for Building Competitive Software," draft submitted to IBM Systems Journal, 31 July, 1992. ____________________ Cited References 29 Ý14¨ "IBM Reuse Methodology: Qualification Standards for Reusable Components," IBM Internal Document, 19 December 1991. ______________________ Ý15¨ Jones, T.C. "Reusability in Programming: A Survey of the State of the Art," IEEE Transactions on Software Engineering, Vol. SE-10, No.5, Sep- __________________________________________ tember, 1984. Ý16¨ Jones, Capers. Applied Software Measurement: Assuring Productivity and _______________________________________________________ Quality. McGraw -Hill, Inc., NY, 1991. ________ Ý17¨ Margano, Johan, and Lynn Lindsey, "Software Reuse in the Air Traffic Control Advanced Automation System," paper for the Joint Symposia and __________________ Workshops: Improving the Software Process and Competitive Position, 29 ___________________________________________________________________ April-3 May 1991, Alexandria, VA. Ý18¨ Mills, E.E. "Software Metrics," Software Engineering Curriculum Module ______________________________________ SEI-CM-12-1.1, Carnegie Mellon University, Pittsburgh, PA, 1988. ______________ Ý19¨ "Standard for Management of a Reusable Software Component Library," NATO ____ Communications and Information Systems Agency, 18 August 1991. ______________________________________________ Ý20¨ "Standard for the Development of a Reusable Software Components," NATO ____ Communications and Information Systems Agency, 18 August 1991. ______________________________________________ Ý21¨ Poulin, Jeffrey S. and W.E. Hayes, "IBM Reuse Methodology: Measurement Standards," IBM Corporation Internal Document, 16 July 1992. __________________________________ Ý22¨ Reifer, Donald J., "Reuse Metrics and Measurement- A Framework," pre- sented at the NASA/Goddard Fifteenth Annual Software Engineering Work- ________________________________________________________ shop. _____ Ý23¨ Standish, T. "An Essay on Software Reuse," in IEEE Transactions on Soft- __________________________ ware Engineering, 10 (1984), No. 5, p. 494-497. _________________ Ý24¨ "Repository Guidelines for the Software Technology for Adaptable, Reli- able Systems (STARS) Program," CDRL Sequence Number 0460, 15 March 1989. Ý25¨ Tausworthe, R.C, "Information models of software productivity: limits on productivity growth," Journal of System Software, Vol 19, No. 2, Oct, ___________________________ 1992, pp. 185-201. Ý26¨ Tracz, Will, "Software Reuse Myths," ACM SIGSOFT Software Engineering ________________________________ Notes, Vol. 13, No. 1, Jan 1988 p. 17-21. ______ 30 BIOGRAPHY _________ JEFFREY S. POULIN is an advisory programmer at IBM's Reuse Technology Support Center, Poughkeepsie, New York, where his primary responsibilities include corporate standards for reusable component classification, certification, and measurements. The author has been active in the area of software reuse since 1985 and was key to the development and acceptance of the IBM software reuse metrics. He has conducted research in software measurement techniques and implemented a program measurement tool for the workstation platform. His interests include object-oriented database systems, semantic data modelling, CASE, and formal methods in software reuse. He is a member of the IBM Corpo- rate Reuse Council, the Association for Computing Machinery, and Vice- Chairman of the Mid-Hudson Valley Chapter of the IEEE Computer Society. He received his Bachelors degree from the United States Military Academy at West Point, New York, and his Masters and Doctorate degrees from Rensselaer Polytechnic Institute in Troy, New York. JOSEPH M. CARUSO is an advisory systems analyst for the IBM Systems Program- ming Tools organization, Poughkeepsie, New York, where his responsibilities include the determination of return on investment for the many technologies funded by his organization. He joined IBM in 1978 as a systems analyst for Material Requirements Planning Systems. In 1985 he moved into the Assurance Department in order to pursue his interests in statistics. While in assur- ance, he developed software statistical tools, projected quality levels of software projects, automated reporting and analysis systems, and provided statistical education for the quality assurance organization. His interests include applications of software reliability growth modelling, sample size determination, and Monte Carlo simulation. He received his Bachelors degree from S.U.N.Y. at Stony Brook, his Masters from Penn State University and is currently pursuing a Doctorate degree from Union College, New York. Biography 31