Browse Books

Reviewer: Andrew Robert Huber

Empirical software engineering is the area where the practical, down-to-earth work of real engineers developing complex software collides with the research ideal of objective academics replicating controlled experiments in known environments. This book's goal is to provide a standard reference describing advanced techniques for running these empirical software-engineering studies. The intended target audience is software-engineering researchers, graduate students, practitioners who report or analyze data such as quality-control metrics, and reviewers of empirical research. Knowledge of software engineering, and other topics such as statistics, is necessary to follow this book. The book has 14 chapters by various authors, and is divided into three parts: research methods and techniques (five chapters), practical foundations (five chapters), and knowledge creation (four chapters). The order of the first two parts is questionable, since Part 2 covers basic issues necessary to conduct a meaningful study; beginning with the practical foundations would be more appropriate. However, for the most part, the chapters are self contained and can be read out of order. Four of the chapters are revised versions of symposium or journal papers, typically with updated references. The five chapters in Part 1 discuss various strategies for collecting and analyzing data, along with the most appropriate use of those strategies. The first chapter covers general techniques for gathering data in field studies, including direct methods ranging from brainstorming to joining a development team, and indirect methods such as instrumenting systems. The second chapter covers, in more detail, two qualitative methods: observation and interviewing. There is some duplication in these two chapters, many of the descriptions are too short to be very helpful, and neither chapter includes examples of using the methods. The other three chapters cover surveys, focus groups, and simulation. The chapter on surveys by Kitchenham and Pfleeger is particularly interesting because, as the authors note, "surveys are probably the most commonly used research method worldwide." In software engineering, surveys are certainly the method most commonly abused and poorly done. The discussion on what steps to take, what problems to avoid, and how to avoid common mistakes should be required reading for anyone developing or conducting a survey. Best of all, the issues are illustrated by showing the mistakes in three published surveys, including one conducted by the chapter's authors, who should be lauded for their honesty and integrity in displaying their own errors. I found the chapter on simulation the least satisfying. Many simulation techniques are described, such as deterministic versus stochastic and dynamic versus static, along with many purported simulation applications in software engineering, such as project and risk management, as well as product and process engineering. A discussion of three simulation tools includes an example of setting up a stochastic model of a generic software development process using one of the tools, but many of the techniques and applications are simply listed, with no discussion of their value. The example simulation does not seem to have any real-world applicability. There is no evidence of any insight or value obtained by simulating a software process. Part 2 includes chapters on statistical methods and measurement, handling missing data, and reporting experimental results. Another chapter covers ethical issues regarding research with human subjects, including a good discussion of the principles of informed consent, beneficence, confidentiality, and scientific value; it also provides good advice on dealing with the practical issues of these principles, such as informed consent and dealing with ethics review boards. Another practical and useful chapter deals with initiating and running empirical studies in collaboration with industry. The last part has four chapters on selecting empirical methods for software-engineering research, building theories, using multiple evidence sources in building theories, and replication. These chapters reflect one of the major weaknesses in software-engineering research: the lack of multiple sources of evidence and repeated experiments. For the most part, the book does a good job of minimizing the usual problems in a collection of papers by different authors. There is inevitably some overlap and duplication, but no glaring disparities in style, presentation, or level of detail. There are ample references in each chapter, as well as a bibliography. This is indeed a guide, as it often just points to other sources. It should be very useful to graduate students and researchers engaged in empirical software-engineering research. Online Computing Reviews Service