There are 5 main principles to the Parasoft AEP Methodology:

1. Apply industry best practices to prevent common errors and establish a foundation for full-lifecycle error prevention.

2. Modify practices as needed to prevent unique errors.

3. Ensure that each group implements AEP correctly and consistently.

?3a. Introduce AEP on a group-by-group basis.

?3b. Ensure that each group has an appropriate supporting infrastructure.

?3c. Implement a group workflow that ensures error prevention practices are performed appropriately.

4. Phase in each practice incrementally.

5. Use statistics to stabilize each process, and then make it capable.

AEP Methodology Principle 1: Apply industry best practices to prevent common errors and establish a foundation for full-lifecycle error prevention

The Parasoft AEP Methodology was developed in recognition of the fact that different development processes share many of the same characteristics and pitfalls, but that each project and organization will also encounter its own unique challenges. Consequently, Parasoft AEP Methodology defines how to best prevent the errors common to most development groups, but also provides a blueprint for identifying and preventing recurrences of the errors that are unique to a specific development group, process, or project.

At a high-level, the software lifecycle always looks the same, whether it involves developing Java-enabled Web applications with Extreme Programming, developing C embedded system programs with a waterfall process, or anything in between. In all cases, software development involves the natural design> develop> deploy> manage lifecycle sequence illustrated in the left circle in Figure 2.

Figure 2: Anchoring and customizing AEP in the software development lifecycle

To anchor basic error prevention into the common software lifecycle, you integrate industry-accepted best practices (such as coding standards, unit testing, integration testing, and so on) into the lifecycle as described in the middle circle of Figure 2 and automate these practices as much as possible. This automated full-lifecycle error prevention is the basic foundation of the Parasoft AEP Methodology?

The integrated best practices are the product of software industry experts examining the most common errors for different languages, and then developing best practices designed to prevent these common errors. They represent a wealth of knowledge that was developed through many prior rounds through the five-step AEP Concept process. By leveraging these existing practices, you can start preventing many common and serious errors before you ever perform the five-step process yourself. Essentially, by adopting these "out-of-the-box" practices, you can instantly progress from following the few best practices that your organization has developed over the years to following a comprehensive set of best practices that have been developed, tested, and perfected by industry experts who have analyzed vast amounts of code梐nd errors?worldwide. You inherit the benefits of tremendous experience without having to perform any of the groundwork required to acquire that experience.

AEP Methodology Principle 2: Modify practices as needed to prevent unique errors

Inevitably, some errors will slip through these 搊ut-of-the-box? practices because every development process and project has its own unique challenges. The AEP Methodology expects these unique errors and provides a mechanism for customizing the practices to prevent these errors.

Each time you discover an error that evades the existing error prevention practices, you apply the five-step procedure that is the core of the AEP Concept. You identify the error, determine the cause of the error, look upstream in the software lifecycle to determine the root cause of the error, modify an existing practice (or implement a new one) to prevent that type of error from recurring, then check adherence to that practice to monitor whether it is being followed. This process is illustrated by the right circle in Figure 2, which shows how this principle might be applied to prevent the problem in the previous section抯 example?a performance problem that was identified during load testing, traced back to a resource leak caused by a development coding error, then prevented through the implementation of a new coding standard. When you apply this five-step process to each error that evades the 搊ut-of-the-box? practices, you not only fix each individual bug you encounter, but also modify the development methodology to prevent the same type of error from recurring. As a result, the development methodology becomes increasingly error-resistant with each bug identified.

AEP Methodology Principle 3: Ensure that each group implements AEP correctly and consistently

Even if a development group or organization has carefully implemented the "out-of-the-box" AEP practices and diligently modified them as needed to prevent their unique errors, this effort is useless if the vital AEP practices are not implemented correctly and consistently throughout the organization抯 development groups. This consistent application requires that you introduce AEP on a group-by-group basis, ensure that each group has an appropriate supporting infrastructure, then ensure that each group follows a group workflow that ensures error prevention practices are performed appropriately.

Principle 3a: Introduce AEP on a group-by-group basis

The most effective way to bring AEP into an organization is to implement it on a group-by-group basis. You start with one group, then, after this group is effectively practicing AEP, you start another group, and so on and so on.

The definition of "group" tends to vary from organization to organization. For the purposes of AEP, a "group" is a group of people that work on a common development project. This group typically includes between five to ten developers, an architect, a project manager, and any QA testers that were assigned to that project. AEP group definitions were developed by reviewing the natural structure of most groups, then formalizing that natural structure in a way that would best support AEP.

Principle 3b: Ensure that each group has an appropriate supporting infrastructure

Each group must have a functioning source control system and automated build process before its members can start practicing AEP. A source control system (also known as a configuration management system) is a database where source code is stored. Its purpose is to provide a central place where the team members can store and access the entire source base. An automated build process automatically executes the necessary build steps (compilations, transfers, etc.) at a scheduled time, without any human intervention.

Principle 3c: Implement a group workflow that ensures error prevention practices are performed appropriately

To ensure that error prevention techniques are used consistently, the Parasoft AEP Methodology describes how to build an automated support infrastructure to ensure that these practices become an integral and enduring part of the team members?day-to-day workflow.

Experience has taught us that these practices must be embedded into the software development groups that will use them. This means that everyone in the development group must understand their role ?be it Developer, Architect, QA, or Project Manager ?and that they must understand how to adhere to that role given the practices in place. For example, understanding who creates coding standards, where these standards are stored, and who uses them and when, is necessary for the group to successfully adopt automated coding standards. Most importantly, defining group behavior ensures that the practices adopted remained ingrained and that they do not deteriorate over time.

In a model AEP implementation, every developer and QA team member has a local installation of each AEP supporting technology; the technology settings are determined by the team architect and standardized across the team.

The development team members will have the appropriate AEP technologies installed on their workstations and integrated into their daily edit, compile, debug process. When a developer creates a new file or checks existing code out of source control and edits it, he must ensure that he has performed the practices defined by the team抯 architect before adding that new or modified code to source control. Practice compliance is checked by using the appropriate AEP technologies to automate the error prevention practices (static analysis, unit testing, etc.). Developers fix any reported errors so that their code complies with the required practices, and then check their code back into source control.

The QA team performs integration testing on the entire checked in project baseline (the checked in work of all project developers) by using workstation versions of the standard team AEP technologies and by manually exercising the application as needed. Preferably, QA tests new and modified functionality as soon as developers add it to source control. To streamline the QA testing process, QA抯 technologies are configured to access the same standard team test settings and files that the developers used and created. QA can then extend the developers?tests to perform more complex functional tests (such as system level tests that check real-life operations which span multiple developers?work) as well as advanced load tests (if applicable). If QA uncovers a new error through these or other tests, the QA team notifies the architect and then works with the architect to diagnose the root cause of the error, then design and implement an error prevention mechanism to prevent this error from recurring. This error prevention mechanism might involve requiring developers to follow a new practice, modifying how developers should perform a current practice, or simply changing the team抯 standard tool settings. QA also adds to the test suite one or more test cases that verify whether the error has been resolved. At that point, the architect asks the responsible developer to correct the problem. Once the new test cases pass, the error is deemed corrected.

To verify that the required practices are being implemented correctly and do not decay, the AEP technologies run as a batch process to test the entire project baseline at regularly scheduled intervals/frequencies (usually, at the same intervals/frequencies as the team抯 automated build). These tests verify whether the appropriate coding standards, unit testing, and integration testing practices have been implemented. They use the same test parameters and files used by development and QA and automatically generate and execute additional tests as needed. No errors should be reported at this point. If any coding standard or unit test errors are reported, the tools notify the architect and developer; this notification serves as an invitation for a code review. During the code review, the architect and developer review the violation of the practice and determine whether this was an appropriate violation of the team抯 required practices. If any other errors are reported, the tools notify the architect and the QA team; this notification serves as an invitation for a meeting to diagnose the root cause of the error, then design and implement an error prevention mechanism to prevent this error from recurring.

In addition, this infrastructure should store and analyze the information from this verification. Team members can access these analyses and use them to assess the effectiveness of the current practices and determine what additional practices would be helpful to implement. The team can then use this same infrastructure to implement and monitor any process improvements that they decide to adopt.

The recommended workflow is illustrated in Figure 3.

Figure 3: The recommended group workflow for success with AEP

AEP Methodology Principle 4: Phase in each practice incrementally

One reason why best practices (such as automated coding standard enforcement) have historically failed is due to the overwhelming amount of information that typically is delivered when developers first apply a new practice to their source code. In fact, the amount of information is usually so overwhelming that many developers reject the findings out-of-hand as 搉oise? and simply ignore the results altogether.

The key to overcoming these challenges and successfully implementing a practice is introducing it in phases. This prevents the team from being overwhelmed by having to learn and following an unmanageable number of new requirements at once梚n addition to performing their normal job responsibilities. One particularly helpful strategy is to divide each practice into several levels梒ritical, important, and recommended梩hen introduce each level incrementally. You would start by requiring that team members follow only the most 揷ritical?practice requirements. Once the team was able to comply with these requirements, you would then require that they also follow the 搃mportant?practice requirements. Once all of these tasks were mastered, you would phase in the 搑ecommended? practice requirements. Only at that point would the team members be required to follow the complete practice.

Another useful strategy is to apply the practice only to files that were created or modified after a predetermined "cutoff date," or?even better?to apply the practice to only the specific lines of code that were created or modified after the cutoff date. Having a reasonable implementation strategy facilitates practice implementation and enforcement, and also maximizes its benefits.

AEP Methodology Principle 5: Use statistics to stabilize each process, then make it capable

AEP will not deliver the maximum benefits until its processes are stable and capable. A stable process is predictable; its variation is under control (i.e., when representative variables are plotted on a control chart, they fall between the upper control limit and the lower control limit, which are based on quality controls such as Six Sigma). For a process to be considered capable, it must be stable and the average of its plotted variables must fall within the specification limits, which vary for each process. A process that is under control, but does not meet its target levels, would be considered stable but not capable.

AEP processes can and should be treated as statistical processes. To facilitate this, Parasoft has defined variables to represent each AEP process and defined the target average value used to assess each process's capability. Each group implementing AEP needs to monitor these predefined variables, plot them in a control graph, then use statistical control limits and target average values to statistically analyze the related processes. To make a process stable, the group must identify and remove the "special causes" that are responsible for the significant variance in the results. If the process is stable but not yet capable, the group must modify and improve the process so that the related variables are consistently at a more desirable level.

For example, the Confidence Factor is one variable that measures the group's adherence to all error prevention practices. When the Confidence Factor variable values are plotted in a control graph, the graph illustrates the typical day-to-day behavior of the development group. What you want to achieve is the long-term stability of this behavior, so that it is moving across time in a small range near the top of the scale. When the Confidence Factor is in this small range it indicates that the code is not being broken through feature additions, test cases are succeeding, and so on. When this is the case, the application can be released because everything is working as it should, and errors are not being found.

The process represented in the following graphic is statistically stable, since it remains well within the control limits. However, the average Confidence Factor is not terribly high; although the process is stable, it is not yet capable.

Figure 4: A control graph for the Confidence Factor Variable

To make this process capable, the group must determine what process modifications would raise their Confidence Factor measurements, modify the process, then continue plotting and analyzing this variable. Once this variable is both stable and at an acceptable level (for example, an average of 80), the process is capable.

Human Concerns Related to AEP

One goal of AEP is to automate error prevention as much as possible. However, in the end, there's no value to automation if team members do not take the appropriate action based on the data yielded from that automation. Ironically, while AEP strives to remove the dependence on human factors as much as possible, in the end, it is humans that will make or break AEP. The prime reasons why AEP may not deliver the desired results are:

?Insufficient management support: The manager does not frequently monitor the process and strictly require that team members follow the prescribed practices and correct all problems exposed by the practices.

?Insufficient architect and QA support: The architect and QA are not dedicated to ensuring that each time an error is detected, the process is modified to prevent all similar errors. If the architect and QA are not committed to this process improvement, the team will not be able to improve the way in which they produce software.

?Lack of buy-in to the importance of error prevention: The entire team does not recognize the value in identifying and correcting code does not yet produce a runtime error, but which is error-prone. Developers often argue that if a reported code problem does not result in a runtime error, the code does not need to be modified. They don't understand that in order to prevent errors, you need to write code in a way that reduces the possibility for errors梟ot just rid the code of full-fledged errors. Management needs to educate the team on why error-prone code needs to be taken just as seriously as error-causing code.

?An overwhelming introduction to AEP: All AEP practices are suddenly required for all code, which typically causes the team to become overwhelmed, reject the findings as 搉oise,?and simply ignore the results altogether. A better strategy is to introduce practices one by one, and introduce each practice in phases. One particularly helpful strategy is to divide each practice into several levels, then introduce each level incrementally. Another useful strategy is to apply the practice only to files that were created or modified after a predetermined "cutoff date", or?even better?to apply the practice to only the specific lines of code that were created or modified after the cutoff date.

Redefining Roles

One key requirement for success is that the team's architect and QA team members are willing and able to extend their role as defined by AEP.

QA team members need to adopt a new approach to errors. Each time an error is found, they need to consider what other errors might be related to that error, and how to abstract a general error prevention mechanism from that error. The most talented QA team members have a sharp intuition about where errors are coming from, and they write test cases to check whether the anticipated errors are actually present. They strive to break the system, not find positive reassurance that it is working. Every time they find an error, QA team members need to try to understand its root cause. This requires that they understand the system's overall system architecture and have an intimate understanding of the part of the architecture related to that error. Moreover, to help the team improve the process and monitor attempted process improvements, QA team members must understand the statistics behind measuring stabilizing, and managing processes, as well as how to identify whether a special cause or variation is affecting a process.

Under AEP, the architect not only needs to be able to design a system, but also to ensure that this system is built properly and works properly. He or she needs to have excellent vision, many years experience writing code, a keen sense of where problems can occur in code, the ability to determine algorithmic solutions for problems, and the ability to abstract from technical details to a higher-level understanding of code operations.

Both QA and the architect need the talent and drive to determine how the team can modify the way the team designs, writes, deploys, and monitors code in a way that will prevent errors from recurring; they need to adopt the mantra "I can find each type of error only once." To do this, they need to be very skilled at abstracting, at looking at a simple error, determining its root cause, and determining how this simple error can provide great insight into ways to improve the team's process. The team's ability to improve its process is directly related to the QA team members' and architect's ability to abstract.

Establishing Group Culture

For AEP to become an enduring part of the team, it must be supported by a strong group culture. Group culture is a development team's way of working together, including their shared habits, traditions, and beliefs. A positive group culture should promote code ownership, group cooperation, peer learning, common working hours, and mutual respect. When managers and leaders focus on developing and supporting a positive group culture, the team is typically more self-regulating, creative, effective, and satisfied.

The most important element of such a group culture is code ownership. Code is the group抯 greatest asset because it is the main thing that they have to show for all of their work. It also serves as means of communication: developers exchange the majority of their ideas by reading and writing code. Just as mathematicians communicate their ideas most precisely with equations, developers communicate their ideas most precisely with code. Thus, by protecting the quality of their code, developers can preserve their best ideas in the clearest, most concise way possible, as well as ensure that their communications are as effective as possible.

Because code is such an important expression and product of the group, caring about the quality and success of this code is fundamental to group culture. It is the glue that holds a group together. You want to build a culture where the developers?attitude towards the code reflects the code抯 importance. Developers should show that they care about the code because caring about the code is synonymous with caring about the group. If a developer cares about the code, he will care about the group, and if he cares about the group, he will care about the code. It抯 fundamental that everyone feels that they have a stake in maintaining high-quality code. This prevents group members from doing anything that harms code quality-- if they care about the code, they won抰 hack at it, shortchange it, wire it, etc.; they will always try to ensure that it is solid and working. This, in turn, helps filter 揵ad apples?out of your development group. In an environment where group members feel a strong investment in code quality, any developer who does not care about the code will alienate himself from the group. Most group developers will soon become frustrated with someone who constantly introduces problems into the code and try to help this developer improve. If he does improve, the group is stronger; if he does not, group conflict will generally lead him to leave the group.

Final Thoughts

The software industry must mature. There can be no maturation without effective error prevention and process improvement. We have no other options?we must do it. If we don't, our industry will pay dearly for it, much as the U.S. automotive industry did in the 1960s and '70s, when it lost nearly half of its market to foreign competitors that knew how to prevent errors and make better quality products. Offshore outsourcing is already starting to do this to us. How long will we wait before we make the necessary changes?

AEP, with its comprehensive collection of "ready-to-use" error prevention practices and its unique embracement of automation, offers the software industry a prime opportunity to start making the necessary changes in a way that is as painless as possible. By providing a feasible way to implement error prevention in the software industry, AEP helps software development organizations eliminate the errors that lie at the root of virtually every concern they are currently struggling with?from high development and labor costs, to quality issues, to security weaknesses, to the need for more (or more rapid) strategic improvements. Moreover, when an organization adopts the Parasoft AEP Methodology, it will also enjoy some pleasant side effects, including 1) the ability to capture and continuously reuse organizational knowledge, 2) improved process visibility and control, and 3) the ability to establish a quality "baseline" that can be improved upon to ensure that the organization is always moving forward and using past experience to improve its future.

Learning More About AEP

Additional Parasoft AEP documents provide detailed instructions for following the Parasoft AEP Methodology principles and explain how the Parasoft AEP Methodology supports industry quality control initiatives such as SEI-CMM, SEI-CMMI, and ISO 9000-3.

About Parasoft

We make software work.

Parasoft provides Automated Error Prevention solutions that combine advanced products, services and expertise to help companies automatically prevent errors throughout the software lifecycle, to improve software quality and reliability. Based on Parasoft AEP Methodology, the company抯 solutions and products automate practices such as coding standards, static analysis, unit testing, regression testing, load & stress testing, functional testing, integration testing, application testing and monitoring. The solutions enable software development and IT organizations to significantly reduce costs by shortening production cycles, improving overall quality and reducing time-to-market. Parasoft has been granted nine patents and numerous awards for the technology behind its innovative line of solutions and products. Founded in 1987, Parasoft is a privately held company whose clients include IBM, HP, Daimler Chrysler and over 10,000 companies worldwide. Parasoft is headquartered in Monrovia, CA. Telephone (888) 305-0041. Fax (626) 305-3036. Email to info@parasoft.com . URL: http://www.parasoft.com .

Notes