Best Practices Blog

Shaking out the bugs and the client’s role

Quite often, I attend Webinars. This has been a great way to stay up-to-date with trends, emerging technology and best practices.

A recent Webinar reminded me of a major point about application testing that I have shared with clients in the past: Bugs are inevitable.

The Webinar referenced a magnificent study along those lines. That study, "Understanding and Controlling Software Costs" by Boehm and Papaccio provided some wonderful statistics:

  • Defective and buggy code consumes 40% - 50% of project costs.
  • The earlier a bug is introduced into a project, and the later in the project the bug is repaired, the greater the cost to correct the bug.
  • IBM found that 5% of bugs originating in the development phase of a project will go undetected into and beyond the release phase. (This is a best case senario where best practices have been implemented in development and testing. This percentage could be higher in many cases.)
  • Every hour spent on bug detection during development and testing will reduce 3 to 10 hours of debugging after a project has been released.

For us, the development companies, we manage these eventualities by testing early and testing often. The sooner we can identify bugs and correct them, the less impact they have on the overall cost – in dollars and in time – of the project. We also follow the industry’s best practices for conducting testing.

So, what is the client’s role in shaking out the bugs?

For a typical project, we provide several milestones where the client can test and review:

  • Prototype - in this phase, we will build a portion of the application’s functionality and provide to the client for review. Client feedback is carried forward into the remainder of the project.
  • ALPHA version - this phase is the first completed version of the application. The client may test and review. We make revisions based on client feedback.
  • BETA version - incorporating all the revisions from the ALPHA version, we arrive at this phase. The client is allowed to test and review to ensure that we’ve understood them correctly.
  • Release or Release Candidate - we take any feedback from the BETA review, make revisions if necessary, and allow the client to review a final time. If all is a go, we release the application into whatever production environment has been specified by the client. If there are any show-stoppers detected, we go into another round of revision prior to final approval and launch.
  • Post-launch - we provide three months of support after release to cover bugs found by the end-user. And, we provide support upon request beyond this phase.

Bugs still slip through. And, that’s part of the 5% ceiling that seem destined to go undetected into the release phase. But, what can the client do to help get to that 5% or, maybe even to a lower percentage of errors?

I have some recommendations:

  • Allocate proper time and resources for testing. Don’t rush through it or approve just to keep the project moving. This is your time to help make the project go as smoothly as possible.
  • Test the application in the way you expect to use it. Developers can’t possibly predict all the ways that an end-user will try to use the application. Often times, the client is the end-user. If that’s not the case, consider using an actual sample of end-users to participate in the testing. We can help with choosing a sample group and provide direction for conducting testing.
  • Provide as much detail as possible about your findings. If you can provide steps to recreate the error, you can reduce the amount of search-time for the developer in recreating the error. This means the bug will be fixed faster.
  • If possible, build in some flexibility for your launch date early in the project. We can take a lesson from NASA here. Even though a shuttle mission’s launch date has been scheduled, the shuttle is on the launch pad, countdown has commenced – if there’s any indication of a problem – NASA scrubs the launch, starts working on a solution and reschedules.

"Be prepared to postpone a release date"? Absolutely. For many who are looking for a development partner, adherence to a predetermined launch date, or the speed with which an application can be launched are often major requirements for choosing a partner.

But, speeding up the development cycle or adhering to a launch-or-be-damned timeline ensures that more bugs will go undetected into your application’s release. Bugs in a live application can cost you more to repair in the end.

We work hard to ensure that every project is a success. Proper participation by the client in the testing phases of the project increase the likelihood of success while decreasing the likelihood of undetected bugs tarnishing that success.

Post a comment

Required Privacy Statement