SOFTWARE QUALITY STRATEGY

Control - Assurance - Management

 

James A. Ward, PMP

 

 

There are  three primary strategies for achieving quality software:

 

·        Quality control tests the finished product of the software development process, with the goal of identifying and eliminating defects.

 

·        Quality assurance reviews the product throughout the software development life cycle to eliminate defects at critical points within the development process.

 

·        Quality management, a preventive strategy, concentrates on defect-proofing the software development process itself.

 

 


I will describe each of these three strategies, providing suggestions for the implementation of quality control and quality assurance, and the benefits to your organization of achieving quality management.  I will discuss the dynamic relationship between software quality and productivity, and  summarize all of the elements that must be addressed in designing a complete quality strategy within a software development organization.

 

THE QUALITY QUEST

 

By way of introduction and background I would like to review my own journey to an understanding of software quality strategy. First, any quality approach must be grounded in theory. As W. Edwards Deming, a founder of the modern quality movement has stated, "Experience  teaches  nothing  unless studied  with  the  aid  of  theory."

            While some people may say that what works in theory may not work in practice, I have a wonderful quote from Dr. Harry Markowitz, a Nobel laureate from the University of Chicago, “In theory, there is no difference between theory and practice. In practice, there is.”

            I first started, in the 1970s, working with software development organizations on improving their productivity. I thought I had a good approach for doing this, addressing five key areas:

 

But I had the cart before the horse. As Deming said, “Measures of productivity do not lead to improvement in productivity.  Concentrate on quality, and productivity will take care of itself.” Quality, not productivity, should be our improvement focus. To those of us who have done considerable software development work, it is evident that the greatest potential improvements in productivity lie in eliminating the destructive cycle of testing and rework that results from poor quality development work.

Quality can be defined as conformance to requirements or the absence of defects. The primary goal of any and all quality approaches is the elimination of errors. To be effective, it is helpful to understand where errors originate in the software development process.

According to several industry sources, insufficiently understood requirements account for 50 % of all errors. Design incorrectly understood or incorrectly translated from requirements account for 30 % of all errors, and coding (programming error or misunderstood design) contribute 20 % of all errors. Further, the longer an error has been in the product, the more expensive it will be to repair, so it behooves the organization to find and correct errors as soon as possible after they enter the software product.

Philip Crosby has said, “there are only two reasons errors are made, lack of knowledge and lack of attention.” The goal of a good software quality strategy is to prevent errors from being made by applying training, knowledge and attention.

For myself, the primary sources of my quality education, training and theory have been three.

First, I was formally trained in Philip Crosby’s “Quality Improvement Process” (QIP). Crosby concentrated on management attitudes toward quality and postulated the Quality Maturity Grid, a five stage model for assessing management’s commitment to quality. I became the chairman of the quality improvement committee for a 150 person I.T. organization and served on many corporate level quality committees for a $1 billion aerospace manufacturer. At that time, the late 1980s, it was difficult to understand how Crosby’s teachings should be applied to the software development process.

            Subsequently, I read Watts Humphrey’s “Managing the Software Process.” Humphrey adopted Crosby’s concept of the five stage maturity grid, concentrating on types of processes used in software development. This was prior to SEI’s publication of the “Capability Maturity Model.” It was not easy for me to discern exactly how Humphrey’s writings were to be applied.

            Then, I read Jerry Weinberg’s “Quality Software Management, Volume 1.” Jerry deals with the cultural patterns found in software development organizations. I will have to say that this book brought quality theory together for me. I understood how to apply the concepts discussed by both Crosby and Humphrey in a meaningful way. Shortly after reading Jerry’s book, I worked with a client that was having severe quality problems in developing software products. The engagement was very successful – and I came away with a cogent theory and approach to quality improvement in software development. I’d like to share with you what I’ve learned.

 

QUALITY CONTROL

 

Quality control (QC) involves a cycle of testing and rework. It concentrates on the finished product of the software development process. Testing is still the primary method for assuring software quality. However, quality can only be built it, not added on. “You burn, I’ll scrape” does not produce quality.

 

The traditional quality control cycle is:

 

OR

 

·        SHIP DATE (whichever come first!)

 

Quality Control Errors

 

In an organization that relies exclusively on quality control, the attitude toward errors is simply, “We don’t know why we have errors. Maybe if we ignore our problems, they will go away.” Worse, some organizations believe that, “Our people are lazy and incompetent, and that’s why they make errors.”

Crosby says that in organizations like these, “Management has no knowledge of quality as a positive management tool.” Weinberg goes on to say, “There is no knowledge of management as a software development tool.”

Results in this environment are unpredictable. Requirements are poorly documented, if at all. Measures, if taken, are mis-used. I once had a client who measured testers positively by the number of errors found, and measured developers negatively by the same measure, regardless of the source or validity of the errors, as these were never investigated. Most of the effort went into disagreements between developers and testers – you get what you measure.

Software projects are usually on schedule until the testing phase, where they then blow it – often by orders of magnitude. I once had a sincere developer tell me, “You never know what the requirements are until you get into the final testing phase, so the thing to do is code something and see what happens.”

Typically, developers and testers interface by “throwing it over the wall.” Testers (and users) are first exposed to the product when development is complete. As requirements are sketchy at best, testers are left to their own devices in finding errors, at which time they throw the software back over the wall.

 

The Bathtub Curve

 

Software normally has a useful life that can be illustrated in the shape of a bathtub curve where defects are graphed against time. During development and early use, defect rates are high. Then, as the software stabilizes, defect rates remain low for a period of use. Over time, as modifications are made to the software and user requirements change, defect rates (or perceived inadequacies) of the software start to increase, and the software should be retired and replaced.

However, in some organizations and on some software projects, errors are so prevalent that the software never achieves “bottom of the curve” stability. During testing, more errors are introduced than are corrected. These projects usually wind up being cancelled.

 

Quality Control Results

 

Quality is measured by whether the system works.  Some work is excellent, some is bizarre. Procedures are quickly abandoned in a crisis, and crises are frequent.  The organization has no real understanding of or commitment to procedures.

An environment that relies on quality control is analogous to CMM level 1, Initial, and organizations moving to level 2, Repeatable.

 

QUALITY ASSURANCE

 

Quality assurance (QA) is a “life cycle” approach to software quality. It focuses on the product at key points throughout the entire development life cycle. QA incorporates a system of walkthroughs and technical reviews as enforcement mechanisms.

 

Quality Assurance Errors

 

Errors are found as soon as possible, providing lower time and cost to fix. If errors are found in the phase in which they enter the software product, the cost to fix is minimized. The organization is actively trying to identify and address quality problems.

Schedules start to become consistent and predictable. Product development milestones and deliverables are specified. Standards and guidelines are created for deliverables. Enforcement mechanisms are implemented. Formal review procedures and acceptance criteria are established. Testers are involved from project inception. More time is spent in the earlier phases of software development projects and less time on testing and rework. Overall development cycle time is reduced.

 

Quality Assurance Results

 

Quality is measured by customer response, although usually not systematically. The organization enjoys consistent success in meeting commitments. The product is visible at key points throughout all stages of development.

Quality assurance is analogous to CMM level 2 (repeatable) and organizations in the process of moving to level 3 (defined).

 

QUALITY MANAGEMENT

 

Quality management (TQM – for lack of a better acronym) also operates throughout the entire product life cycle. TQM focuses, not on the product, but on the processes used in developing the product. Techniques such as root cause analysis and error cause removal are employed. Continuous process improvement is invoked. The goals of TQM are to implement a system of prevention and to eliminate variability, resulting in “zero defects.” A TQM testing cycle incorporates QC and QA approaches, but adds the following steps:

 

·        Test

·        Find errors

·        Capture and categorize errors

·        Perform root cause analysis

·        Identify point of introduction

·        Perform error cause removal

 

Quality Management Errors

 

Organizations practicing TQM believe that “Errors are golden, they represent opportunities for improvement.” Defect prevention is a routine part of the operation. Quality of all products is constantly being improved by improving the quality of the processes used to develop the products.

Processes are highly visible during the entire life cycle. Measurements are taken, as a byproduct of performing the processes. Remember - you can’t control what you can’t measure and you can’t measure what you can’t see.

Motorola, one of the early practitioners of the TQM approach to software development, found that “the system development process must be nudged from an individual art form to a measurable process before it can be rigorously controlled.”  The Motorola approach has been disseminated widely, known as “Six Sigma.”  While Motorola has certainly had its ups and downs in the competitive arena, they did get this one right. Bob Galvin, past CEO of Motorola, credits their quality program with preserving the very existence of the corporation.

 

Quality Management Results

 

The organization experiences consistent success, even on ambitious projects. They are constantly delighting their customers by exceeding expectations. The organization solicits and uses feedback from all stakeholders on a constant and consistent basis. Measurements are part of the process. This is where significant quality improvement really begins, and improvement can be dramatic.

The major difference between quality assurance and quality management is that QA concentrates on product visibility, while TQM concentrates on process visibility.

TQM is analogous to CMM level 3 (defined) and higher (managed, optimizing).

 

IMPLEMENTATION STRATEGIES

 

The foundations of Software Quality Management are:

 

 

 

 

 

Quality Training

 

Remember that “experience teaches nothing unless studied with the aid of theory (Deming).” A “critical mass” of the organization’s staff must be trained in quality management tools and techniques to be effective. Most organizations underestimate the extent of the need for this training.

 

SUMMARY, THE QUALITY JOURNEY

 

To achieve consistently high levels of software quality, an organization must exhibit what Deming has called “constancy of purpose.” This involves four distinct activities in my view, in the order of their importance:

 

  1. Quality Leadership
  2. Quality Culture
  3. Quality Environment
  4. Quality Process Management

 

Quality Leadership

 

Leadership must be consistent. Top management must both “talk the talk” and “walk the walk,” demonstrating a commitment to quality. Leadership activities will include:

Leadership must define a “Quality Mission And Policy” for the organization. It must also set Quality Goals, e.g. reduce costs, increase revenues, reduce development time, reduce defects, increase staff retention, increase customer satisfaction. Goals must be ambitious, with substantial, rather than incremental, improvement targets. Leadership must also become, and foster, change agents within the organization, individuals who are experienced and adept at fostering change.

 

Quality Culture

 

Many quality efforts fail because the culture will not support them. Jerry Weinberg, in his four volume “Software Quality Management” series, focuses on the culture as the defining factor in an organization’s quality efforts. A quality culture will need to address the following factors:

Is the organization structured in a way that will encourage quality work? Are employees empowered to perform their assigned responsibilities to the fullest of their abilities? Does the organization encourage team building, or does it foster competition rather than collaboration within its ranks? Is continuous process improvement part of the organizational culture, or does the attitude of “if it ain’t broke, don’t fix it” prevail? Is the physical environment conducive to productive work and mutual trust between employees and management? Are company and departmental policies and procedures aligned with quality goals and employee empowerment?

The organization should create Software Engineering Process Group (SEPG) and charge that group with the responsibility for promoting continuous improvement of software development processes. Many organizations have also benefited from an organizational assessment when beginning or renewing a commitment to TQM. This assessment is best based on criteria like those established in the Baldrige Quality Award, which covers all activities performed by the organization, not just focusing on internal processes, like the CMM criteria.

 

Quality Software Development Environment

 

Many factors, some already discussed, are important to the environment in which quality software is developed. Management and the culture create the context, but unless good tools and techniques are employed, the organization will not be able to produce quality software on a consistent basis.

 

Factors to be addressed include:

 

 

This is quite a laundry list, but all are important and all need to be addressed. There are probably more items that could be added to this list in creating an effective software development environment.

 

Quality Process Management

 

In addition to the above, there are quality process management tools and techniques that must be employed. These are not unique to the software development environment, and have the added benefit of being employable throughout an organization to increase effective communication and problem solving abilities.

Quality management tools and techniques include defining and establishing quality processes, e.g., CMM Key Process Areas; implementing standards and enforcement mechanisms for all processes; measuring the “Cost Of Quality” (as defined by Crosby), constantly tracking all defects; utilizing root cause analysis and error cause removal techniques.

Quality targets can be established at the process level, as long as the organization is careful to avoid concentrating on “local optimums” at the expense of the entire process. Key success factors should be defined and metrics established to measure progress against those factors. The organization should benchmark itself against industry best practices. Customer satisfaction surveys should be performed on a regular and consistent basis. The results of these surveys should be shared with all personnel.

The organization must also provide quality training on an ongoing basis. New personnel must be indoctrinated and the quality message must be constantly reinforced. 

            In closing, I hope that I have provided an understanding of the software quality universe, an awareness of the tools and techniques used in achieving software quality on a consistent basis, and, most important, an appreciation for the tasks which we all must face if we are progress on this journey. It is not easy. Most software development organizations do not even attempt the journey to Quality Management, let alone Quality Assurance, and many that do will flounder along the way. However, for those organizations that stay the course, the rewards are truly worth the effort.

 

 

ABOUT THE AUTHOR

 

James A. Ward is an independent management consultant specializing in System Development Project Management and implementation of Quality and Process Improvement Initiatives in Information Systems.  Mr. Ward has over thirty years of experience in information systems management and consulting.  He holds an MBA in Finance and Business Policy from the University of Chicago and a Bachelor's Degree in Economics and Mathematics from the University of Minnesota. He holds a PMP certification from PMI.

 

Mr. Ward resides in Jacksonville, Florida.  He can be contacted at (904) 273-8777 or via e-mail at soozward@earthlink.net. Visit his web site at http://www.JamesAWard.com.

 

 

REFERENCES

Jones, Capers (1994). Assessment and Control of Software Risks, PTR Prentice Hall, Upper Saddle River, New Jersey.

Weinberg, Gerald M. (1992), Quality Software Management Volume 1 Systems Thinking, Dorset House, New York.

McConnell, Steve (1998), Software Project Survival Guide, Microsoft Press, Redmond, Washington.

Carnegie Mellon University Software Engineering Institute (1995), The Capability Maturity Model, Addison Wesley, Reading, Massachusetts.

 

Crosby, Philip (1979). Quality Is Free, McGraw-Hill, New York.

 

Humphrey, Watts S. (1989). Managing The Software Process, Addison Wesley, Reading, Massachusetts.