Adding Product Lines, Architectures, and Software Reuse to the SA-CMM -May 98

Adding Product Lines, Architectures, and
Software Reuse to the Software Acquisition
Capability Maturity Model

Tara Ragan, U.S. Army Space and Missile Defense Command
Donald J. Reifer, Reifer Consultants, Inc.

Under sponsorship of the Ballistic Missile Defense Organization's Small Business Innovative Research program, we have been actively developing and promoting changes to the Software Acquisition Capability Maturity Model that stimulate increased software reuse through revised and improved acquisition practices. Such practices are aimed at helping buyers incorporate product lines, architectures, and reuse considerations into their decision processes and products throughout the acquisition lifecycle. Such practices are not only aimed at improving the way government program offices do business, they are also directed at enhancing the manner in which contractors manage their suppliers, especially in acquisitions that involve commercial-off-the-shelf packages and strategic partnerships.

paradigm shift has occurred in the way systems are being built. As more systems are developed to open system standards, products once constructed from scratch using custom designs are being replaced by product lines that contain large quantities of commercial-off-the-shelf (COTS) hardware and software components. Commercial and aerospace firms, as part of their current best practices, have adopted these changes in the way systems are being built. For example, Lucent has modified its software process to review all projects before they are given go-ahead to ensure they conform to their architectural standards. Aerospace firms like Northrop Grumman have done the same for product lines they have developed, promoting software reuse across weapons systems within the radar and air defense application domains.
     As the emphasis has shifted to product lines and architectures, considerable changes have been occurring in the area of software process. The Software Engineering Institute (SEI) has been changing the frameworks many organizations use to assess the maturity of their software processes to include product line, architecture, and software reuse concepts. For example, the SEI is strongly considering including a new key process area (KPA) at Level 4 of its Software Capability Maturity Model (SW-CMM) [1] called "Organization Software Asset Commonality." This KPA requires organizations to exploit commonality that exists among software products using state-of-the-art techniques like domain engineering.
     To exploit commonality, this KPA requires organizations to identify the software product lines that constitute their core business and to populate these with reusable assets when appropriate. Other changes are also being incorporated into Levels 2 and 3 KPAs in the forthcoming Version 2 of the SW-CMM to support reuse. For example, the Level 2 KPA on subcontractor management is being broadened to encompass improved acquisition management and software supplier management practices. These improvements focus on improving the manner in which relationships with suppliers (subcontractors, strategic partners, COTS package vendors, etc.) are managed.
     Unfortunately, the Software Acquisition Capability Maturity Model (SA-CMM)[2] has not kept pace with the advances of the SW-CMM in the area of product lines, architectures, and software reuse. The SA-CMM is a sister framework to the SW-CMM that can be used by organizations that purchase development of their software from third parties to assess the maturity of processes used for software acquisition management. Such organizations include government program offices and commercial firms that contract for their software, e.g., many banks and insurance firms outsource their software to third parties, or buy it via strategic partnerships, e.g., the way firms like Boeing buy software for their commercial aircraft. Most of the process community would argue that these types of software acquisition organizations should be at the same level of maturity as those software organizations they are trying to manage.
     This article summarizes the results of a Ballistic Missile Defense Organization (BMDO)-sponsored Phase I effort conducted to determine what changes to the SA-CMM are needed to exploit the advances being made in the areas of product lines, architectures, and software reuse. Our Phase I effort recommended over 30 changes to the SA-CMM [3] and confirmed that there is a market for aligned products and services[4]. It also validated the promise of these changes using pilot project appraisals and ensured that these proposals are consistent with Version 2 of the SW-CMM. During Phase II, we will prototype these products and services and demonstrate their value by continued beta testing on trial projects.

The SA-CMM Framework
The SA-CMM describes the processes software buyers use to acquire, sustain, and maintain software. It provides a framework to establish benchmarks and improve an organization's software acquisition processes. It is a staged model in which processes are organized into KPAs with an architecture similar to the SW-CMM. There are five levels of process maturity through which software acquisition organizations evolve:

Level 1: The Initial Level - the organization does not have documented processes. It functions ad hoc and relies on crisis management techniques to address problems.
Level 2: The Repeatable Level - the organization fosters discipline through basic practices, which are followed at the project level.
Level 3: The Defined Level - acquisition practices are defined at the organization level and are tailored for use at the project level.
Level 4: The Quantitative Level - metrics-based practices are used to make decisions as processes are employed throughout the organization.
Level 5: The Optimizing Level - continual improvements are made to processes based on quantitative feedback flowing from early adopter projects.

Figure 1 illustrates the structure of the SA-CMM. It was primarily developed to help government program offices improve the way they manage organizations that develop software for them under contract. However, the use of the SA-CMM is not confined to situations where software is being acquired in this manner. It can also be used in commercial applications where software is subcontracted, outsourced, or acquired from vendors (COTS packages, tools, etc.).

Level	Focus	Key Process Areas
5. Optimizing	Continuous Process Improvement	Acquisition Innovation Management, Continuous Process Implementation
4. Quanitative	Quanitative Management	Quanitative Acquisition Management, Quanitative Process Management
3. Defined	Process Standardization	Training Program, Acquisition Risk Management, Contract Performance Management, Project Performance Management, Process Definition and Maintenance
2. Repeatable	Basic Acquisition Management	Transition to Support Evaluation, Contract Tracking and Oversight, Project Management, Requirements Development Management, Solicitation, Software Acquisition Planning
1. Initial	Competent People and Heroics

Figure 1. SA-CMM key process areas.

Recommended Changes to the Framework
As part of our Phase I efforts, we performed a detailed analysis of the SA-CMM to scope the changes needed to take advantage of product line, architecture, and reuse concepts. Our goal was to determine what changes were needed to stimulate adoption of software reuse concepts as part of the processes program offices use to manage the acquisition of their software. Definitions for key terms, desired outcomes, and best practices were those previously agreed to by Department of Defense (DoD) representatives, published in 1995 as the DoD Software Reuse Initiative's Strategic Plan [5].
As a result of this analysis, 34 changes were recommended to the existing SA-CMM framework. To develop our recommendations, each of the framework's KPAs was analyzed, along with related change requests submitted to the SEI during the past year. Based on discussions with the SEI principals, our suggested changes were structured as examples, elaborations, and extensions to existing material to minimize the impact of the changes on the overall document.
We did not believe the existing framework needed to be altered. Instead, we opted to provide its users with guidance on how to make the existing framework work for reuse with product lines and architectures in mind. For example, we suggested adding reuse considerations to acquisition plans and recommended that owners of the architecture be delineated in the software acquisition plan, with their roles and responsibilities.

Validation by Experts
The proposed changes were peer-reviewed by a group of software reuse experts from government and industry at the Reuse '97 Workshop held in Morgantown, W. Va. in July 1997. We wanted to ensure that the software reuse and process communities agreed with our changes. After reaching consensus, we submitted the changes to the SEI for incorporation into the next release of the SA-CMM. The experts who participated in the peer review included

Mary Beth Chrisis, SEI (guest)
Dixie Garr, Texas Instruments
Ted Lewiston, U.S. Air Force
Fred Maymir-Ducharme, Ph.D., Lockheed Martin
Stan McVay, West Virginia University (scribe)
Mark Paulk, SEI (guest)
Sabrina Raman, Boeing
Donald Reifer, Reifer Consultants, Inc. (facilitator)
Ken Song, Department of National Defense, Canada

This peer review exercise was extremely valuable because the group endorsed the changes we proposed to the SA-CMM without reservation. They also suggested additional changes to the SW-CMM that were aimed at helping the SEI guests overcome some problems they were having in tackling software reuse issues within the then current version.

Piloting the Results
We also wanted to ensure that our suggested changes stimulated increased reuse. We sought out organizations within government and industry that would let us prove the value of our ideas via what we called a "quick-look" appraisal of their programs. It is interesting to note that few of the program offices we approached were interested in conducting a complete, formal appraisal. They were either too busy or did not have the staff to support the formal appraisal process. They also were concerned about the increased workload and scrutiny that could result from the appraisal's findings. As such, appraisals turned out to be hard to sell.
     We began the piloting task by modifying the SEI-developed SA-CMM appraisal questionnaire to incorporate product line, architecture, and software reuse considerations. We soon concluded that it would be easier to replace the SEI-developed questionnaire with a new one because software reuse concepts could not easily be retrofitted into the document. In response to this need, we developed our own appraisal questionnaire [6]. We also developed a briefing that provided an overview of our Small Business Innovative Research (SBIR) effort and used it several times to solicit acquisition organization participation in pilot appraisals.
     Using our questionnaire, we conducted a series of "quick-look" appraisals of the acquisition processes used by two Northrop Grumman and two U.S. Army program management offices. These appraisals validated that the changes recommended to the SA-CMM are valuable and stimulate increased reuse. They had the added benefit of providing the pilot projects with constructive improvement recommendations in areas other than software reuse. They also proved useful in helping us identify priorities for candidate products and services we will provide during our Phase II effort to quicken the transfer of reuse technology via acquisition process alignment.
     The lessons we learned as we tried to get projects to participate in appraisals helped us understand the issues software acquisition managers have relative to the SA-CMM and reuse. Their concerns are summarized in the following five questions:

Why should I conduct an SA-CMM appraisal? What are the costs and benefits?
What can I do with the results of the appraisal? Where is the leverage?
Why should I be concerned with product lines, architectures, and software reuse?
What can I do to improve my management of COTS products? What processes make sense, and what can I do about them? What about enterprise-wide licensing?
How do I improve the way I manage my strategic partnerships with my suppliers?

Responses to these questions, which appear in our final report [7], were instrumental in getting program office agreement to participate in an appraisal. The most pervasive of these questions dealt with COTS management. Most organizations we talked to had experienced difficulties with COTS and were looking for ways to improve the practices used to manage its acquisition. For example, integrating COTS into the architecture using "glueware" sometimes seemed to create more problems than were solved by the use of COTS.
Another set of concerns stemmed from the fact that most offices we assessed had ongoing programs that manage existing contractor or supplier relationships. People in these offices were interested in knowing the answer to the question "How do I improve my acquisition management processes on contracts that have already been awarded?" Because they were comfortable with their existing practices, it was difficult to sell them on the need to insert new, reuse-based processes.

Market Survey
In parallel with these activities, we conducted a market analysis and developed a business plan and business case [8] to excite support among potential investors for our Phase II SBIR activities. We began this activity by scoping the market for prospective SA-CMM products and services. We wanted to ensure that whatever we proposed to develop during our Phase II SBIR effort had high commercialization potential. We developed a market survey questionnaire and used it to canvass targeted organizations, both industry and government, to determine the size and characteristics of the market for prospective SA-CMM products and services. The results of the survey were extremely encouraging. They indicated that the annual return on our projected Phase II investment should be close to 58 percent. These high returns justified our plans to pursue Phase II support.
We then used the results of the survey to develop a business plan and business case. This plan was used to show investors the potential returns if they elected to fund our future activities. Based on the positive results we projected in this plan and during the pilot appraisals, Northrop Grumman elected to partner with us to solicit Phase I Interim and Phase II fast-track funding from our BMDO sponsors. Their cash investments have been instrumental in helping us secure BMDO Phase I Interim and Phase II funding.

Potential Phase II Products and Services
Based on the results of our market survey and our piloting efforts, we can conclude that a market seems to exist for the following SA-CMM products and services:

Model Software Acquisition Processes - Model software acquisition management processes that respond to user requirements need to be developed for each identified market segment. Specifically, they are needed for the Acquisition Planning, Solicitation, and Evaluation SA-CMM KPAs. Model processes are also needed in COTS management; even though COTS management is not specifically addressed in the SA-CMM, it was the area where the need for additional guidance seemed most pressing during our piloting efforts.
Tailoring Guidelines - Related tailoring and scaling guidelines are needed so organizations can apply the model processes within their operations. Guidelines should be aimed at acquisition management, supplier management, and COTS management audiences.
Software Acquisition Education and Training - A variety of course materials are required to sell executives on the need to use the SA-CMM. Development of practitioner skills, knowledge, and abilities in the model processes and tailoring guidelines also seemed to be desired.
Appraisals - Appraisals need to be conducted to identify organizational strengths and weaknesses relative to the requirements of the SA-CMM. Organizations may also need help developing improvement plans that respond to the appraisals findings.
PC-Based Tools - A hypertext-based set of tools that run on a PC are needed to help users tailor Reifer Consultants, Inc. (RCI)-developed model processes to their needs using the tailoring guidelines developed for that purpose. This tool kit should make it easier to use the processes than to not use them.
Technology Transfer Kits - Everything needs to be packaged so a potential user of the SA-CMM can quickly develop the know-how to use the technology. The preparation of CD-ROMs with everything a user needs to use the processes, in kit form, is being pursued.
Consulting - Clients may need help using the products and services listed above to assess and develop their software acquisition management process improvement plans.

Our plan is to develop and pilot most of these products and services as part of our Phase II SBIR effort. Close coordination and cooperation with the SEI, BMDO, and our Phase II partners is essential. To keep the effort synchronized and keep all the key players involved, we plan to form an advisory council that will meet regularly to provide us with oversight, direction, and guidance. This council will be chaired by U.S. Army Space and Missile Defense Command, the BMDO-designated manager of this SBIR effort. Members of the council will be drawn from government, industry, and the SEI.
Organizations interested in participating in our Phase II SBIR effort are encouraged to contact us directly. Participation can be in the form of an appraisal or piloting the use of the model processes, guidelines, training materials, tools, or technology transfer kits we develop. Pilot projects will each be represented on our advisory council. They will help the team set priorities based on feedback from their trial use of Phase II products and services.

Findings and Conclusions
The four pilot appraisals proved beyond a doubt that it is feasible to use the modified SA-CMM (based on our recommended changes) to stimulate increased use of product line, architecture, and software reuse concepts. These appraisals also helped us understand where acquisition organizations need help, especially when they adopt the SA-CMM. These needs translate into the following process requirements for our Phase II efforts.

Acquisition processes should be compatible with those employed by suppliers, partners, or contractors. Synchronization points between processes used by these multiple parties, such as reviews, should be identified and well bounded.
For government organizations especially, developed processes should be geared to supporting major program reviews and any funding cycle requirements.
Leverage over suppliers should be gained primarily via strategic partnerships. Both acquirers and suppliers should invest their own resources as part of such partnerships. For the partnership to work, both sides should gain some advantage from the other. For example, the acquisition office might offer to market a supplier's product internationally if they make such reciprocal investments.
Leverage over contractors and subcontractors should be gained primarily via controlling the fee allocations and action items from program reviews. Using incentive or award fees to stimulate achievement of a goal should be pursued as part of the acquisition strategy, especially for reuse.
Because the management of relationships is the key to supplier management, techniques that improve such practices should be highlighted by our work. The strength of the relationship and the degree of confidence and trust that exists between parties effect leverage.

During Phase I, we also found that there were some basic things program offices could do to improve their acquisition management processes. For example, we found that the following guidelines need to be followed as processes are developed to mechanize the SA-CMM.

Make software reuse a concern in your and your supplier's software development plans.
Establish a software reuse working group to recommend how to put product lines, architectures, and software reuse concepts to work on the program.
Incorporate product line, architecture, and software reuse concepts into the checklists you use at reviews (both program and peer).
Empower your chief engineer to make decisions relative to the refinements and use of your product line architecture.
Use earned value, technical performance measures, and rate of progress information to determine how well suppliers or contractors are doing relative to plans.
Strengthen and use risk management concepts to identify, prioritize, and address the top 10 risk items on the project. Factor risk resolution into your fee plans.
Take advantage of the multitude of public resources that are available within the government and on the World Wide Web for help implementing these ideas (the Army Reuse Center, the SEI, etc.).
When appropriate, partner with contractors to accelerate their being awarded a higher SW-CMM level. For example, make their training costs allowable under the contract only if they achieve this higher level of process maturity.
Provide appropriate contractual incentives to stimulate increased levels of reuse.

In summary, our Phase I SBIR effort proved the feasibility of stimulating increased reuse by adding product line, architecture, and software reuse concepts to the SA-CMM. The Phase I effort also demonstrated that there is a market for related goods and services. The pilot appraisals conducted during Phase I helped us develop needs, priorities, and requirements for Phase II. We are encouraged by the results and are trying to fulfill these needs as part of our current Phase II SBIR efforts.

About the Authors
Tara Ragan is a computer engineer with the Advanced Technology Directorate of the U.S. Army Space and Missile Defense Command. She has 14 years progressive software engineering experience in government and industry, including involvement in all phases of the software acquisition and development lifecycle. She has performed SW-CMM-based Software Capability Evaluations for the Army and BMDO since 1991. She is currently the governmental technical representative on several SBIR efforts, including the subject BMDO-sponsored Phase II SBIR effort with RCI. She has a bachelor's degree in computer science from the University of Alabama and has done postgraduate work at the University of Alabama in Huntsville. She is an Army Acquisition Corps Level III professional, a member of the Institute of Electrical and Electronics Engineers (IEEE) Computer Society, and the Huntsville, Ala. area Software Process Improvement Network.

U.S. Army Space and Missile Defense Command
ATTN: SMDC-TC-AS/Ragan
P. O. Box 1500
Huntsville, AL 35807-3801
Voice: 205-955-3515
E-mail: ragant@smdc.army.mil

Donald J. Reifer is one of the leading figures in the fields of software engineering and management, with over 30 years of progressive experience in government and industry. From 1993 to 1995, he was chief of the Ada Joint Program Office, technical adviser to the Center for Software, and director of the DoD Software Reuse Initiative under an Intergovernmental Personnel Act assignment with the Defense Information Systems Agency. He is president of RCI, which specializes in helping clients improve the way they do business. He has a bachelor's degree in electrical engineering from New Jersey Institute of Technology, a master's degree in operations research from the University of Southern California, and a certificate in business management (MBA equivalent) from University of California at Los Angeles. His many honors include the Secretary of Defense's Medal for Outstanding Public Service, the NASA Distinguished Service Medal, the Frieman Award, and the Hughes Aircraft Fellowship.
Reifer has over 100 publications, including the popular IEEE Software Management Tutorial (5th ed.) and a new Wiley & Sons book entitled Practical Software Reuse.

Reifer Consultants, Inc.
P.O. Box 4046,
Torrance, CA 90505
Voice and fax: 310-530-4493
E-mail: d.reifer@ieee.org

References
To view the recommendations and final report, visit Don Reifer's Web site at http://www.reifer.com/.

Paulk, Mark C., Charles V. Weber, Bill Curtis, and Mary Beth Chrissis, The Capability Maturity Model, Addison-Wesley, Reading, Mass., 1995.
Ferguson, Jack, et al., Software Acquisition Capability Maturity Model, Version 1.01, CMU/SEI-96-TR-020, Software Engineering Institute, Pittsburgh, December 1996.
Reifer, Donald J., "Recommended Changes to the SA-CMM," Proceedings of the Reuse '97 Workshop, Morgantown, W. Va., July 1997.
Reifer, Donald J., SA-CMM Market Survey Results, Reifer Consultants, Inc., Report RCI-TN-150, June 30, 1997.
"DoD Software Reuse Initiative," Strategic Plan, June 16, 1995.
Reifer, Donald J., Questionnaire for Use with the Software Acquisition Capability Maturity Model (SA-CMM) "Quick-Look" Appraisals, RCI-TR-050, Reifer Consultants, Inc., June 30, 1997.
Reifer, Donald J., Final Report: Adding Product Lines, Architectures, and Software Reuse to the Software Acquisition Capability Maturity Model (SA-CMM), SBIR Phase I Report (CDRL A002), Dec. 27, 1997.
Reifer, Donald J., Business Plan/Case for Pursuit of Software Acquisition Capability Maturity Model (SA-CMM) Products and Services, Reifer Consultants, Inc., Aug. 1, 1997.