ISBN:0201748681/p>
Reviews From AMAZON.COM
An Excellent "How To Implement" a Good Testing Process
Rex Black's latest book, "Critical Testing Processes" belongs in every test engineering library. Rex's book is logically organized; Part I, Prepare, embraces Chapters 1 through 7. He begins with a discussion of Planning, the first phase in the testing lifecycle, focusing on Quality Risks to assist test planning, He provides one of the best step-by-step examples I've seen to develop a test estimate. Rex sets the stage for us by showing how testing fits into the larger project context, and defining a 15-step testing process. He defines the context of his case study on the Sumatra project. In this chapter, Rex shows one of the nicer examples of the classic "V-Model" relating the development and testing lifecycles.
In Chapter 2, Rex explores the subject of Analyzing Quality Risks as the basis for his test planning, and shows a variety of templates and techniques that will assist the reader in performing this process, and links them to his case study. Note that using Quality Risks may have to be coupled with requirements/feature coverage on contracts with a government customer.
I believe that Chapter 3 (Gaze into the Crystal Ball; Estimate the Work Ahead), is one of the most helpful guides to Test Estimating that I've seen. Rex provides a variety of templates to assist folks unfamiliar with building a testing estimate.
Chapter 4 shows how testing provides "value-add", and gives the reader insight into aligning the initial estimate with the financial constraints of the project. Rex discusses ways to reduce cost by focusing testing based on the Quality Risks, in the context of the Sumatra case study. Templates, spreadsheets, and methods for calculating testing cost and return on investment are supplied.
In Chapter 5 Rex shows how to take the realistic estimate that was created and sell it to the project team. He also has a section titled "Predict with Honesty", which emphasizes the necessity for Ethics in our business.
Chapter 6 discusses the significant efforts in continuous test planning, It uses a variety of excellent figures to illustrate the author's points; however, note that there are 2 Figure 6-5's in this chapter, which results in the remaining figures being mis-numbered.
Chapter 7 goes into detail on how to get stakeholder support for the Test Plan. The Test Plan is the keystone for successful testing. Rex also provides an interesting section on what to do "When Key Players Don't Support the Test Plan" and how to work in a regulated (e.g., export restrictions) environment.
Chapters 8 through 11 are the section titled "Prepare". Chapter 8 on hiring testers is another area of the book not dealt with in many texts, but is extremely critical to successful testing. There are several parallels here to the Perry/Rice book "Surviving the Top Ten Challenges of Software Testing".
In Chapter 9 Rex provides pointers to career growth in a testing team, and assessing skills of the team members. He gives some tips on creating job definitions. He also addresses the challenges posed by having to accept team members other managers don't want, and use of contractors and consultants. Good chapter!
Chapter 10 gets back into the more technical side of test management. Through his Sumatra case study, he provides some insights into stress testing, and use and support of test tools.
In Chapter 11 Rex discusses details of approaches to test coverage, test design, selection and implementation of automated tools. He also takes some time to discuss approaches to "vague requirements", which is a problem most of us face.
Chapters 12 and 13 are Part III, Perform. Rick provides an excellent discussion of management of test releases in Chapter 12. This discussion is nicely illustrated through the case study. Note that you need to have the ability and a mechanism to UNINSTALL any release.
In Chapter 13 Rex defines a test execution process, and then goes on to show how that process works. He defines a set of test tracking worksheets, and illustrates use of the process and the worksheets on "the Big Build".
Chapters 14 through 17 are Part IV. Rex defines the Bug Reporting process. He describes use of this process through application to "the Big Build". He goes on to show how to, and how NOT to, report bugs, and emphasizes the necessity of describing one symptom per bug report. One caution in this chapter; Rex uses shaded boxes to emphasize points. Several of these boxes were misplaced in the book assembly process, so the reader needs to use care in navigation on pages 391 through 394.
The title of Chapter 15 seems a little misleading to me, in that it doesn't deal with writing a Test Summary Report (IEEE definition), but rather with the critical, on-going process of reporting test metrics. Rex focuses on describing the creation of a "test dashboard", and use of that dashboard for providing a succinct delivery of test status. Rex describes what I believe are a useful set of metrics charts. Unfortunately, the book assembly process resulted in Figures 15-2 and 15.3 being reversed, which led to much confusion on my initial read.
In Chapter 16 Rex discusses change management as it applies to bug fixes. The Sumatra case study is used to provide an informative view into the change management process. Rex notes that such a process must balance considerations of features, schedule, budget, and quality by assessing impacts from all project groups.
Rex concludes his book with Chapter 17 with some thoughts as to how testing can provide value.
Rex finishes up with a Glossary that defines his terms.
A suggestion for the interested reader. Read Rex's book in concert with the book from Drabick titled "Best Practices for the Formal Software Testing Process: A Menu of Testing Tasks". Drabick's book shows the reader "What to Do"; Rex Black's book shows "How to implement the What".
A creditable story
This book is written for the test manager and/or project manager who oversees one or more testing efforts. The author has done a great job of taking his own personal experiences (good and bad) gleaned from years of "in the trenches" work, and threading them together into a single fictional story that he uses to illustrate his proven method of managing testing projects.
Part 1 (Chapters 1 to 7) talks about the stuff you need to do to Plan for a testing effort, such as; convincing senior management that spending money on testing is a good Return On Investment (ROI), how to get a realistic ballpark estimate for how long the testing is likely to take, and when & who should get involved in the project.
Part 2 (Chapters 8 to 11) deals with Preparing for test execution; selecting test strategies & techniques, putting a testing team together (and keeping them together), and determining the appropriate measure(s) of test coverage
Parts 3 & 4 (Chapters 12 to 17) first focuses on perhaps the most stressful phase of a testing project - test execution. When the "heat of battle" is most intense; it's useful to have examples, templates, and checklists that a test manager can draw upon to help them manage and report on the test execution in a controlled and credible manner. The book then wraps-up by discussing process improvement; what went well, and what could have been improved (no witch hunts please!).
In summary, I believe Rex has built a very comprehensive and detailed composite documentary on how he has defined and deployed the most critical processes needed to support a software testing effort.
In the vain of "full disclosure", you should know that I've known Rex for many years, and consequently I cannot be considered a completely impartial reviewer.