Browse Books

Reviewer: Michael J. Oudshoorn

This edition is a revision of the authors' book of the same name, originally published in 2000 [1]. The revised edition includes new chapters and examples, which further cements its place as the premier book on the topics of designing, constructing, executing, and assessing experiments in software engineering. The book has been used very successfully for many years in graduate-level courses, but is also a valuable resource for faculty and practitioners engaged in empirical software engineering studies. The revisions will make this book even more valuable. The first of the book's three parts recounts the background of empirical software engineering and introduces the terms needed for the remainder of the book. It sets the stage and discusses the importance of empirical software engineering, what can and cannot be learned from experiments, and what benefit these experiments provide by validating or disproving our beliefs about the software engineering process, its methodologies, and our intuitions about what does and does not work well. Part 2 contains the real contribution this book makes to the field. It focuses on the manner in which experimental questions need to be formulated, how the scope of the experiment is stated, how experiments are planned and designed, the way in which they are executed, and the manner in which the collected data can be validly analyzed and interpreted. Much of this part of the book is devoted to statistics, but it is presented very much in the context of a software engineering experiment. The kinds of experiments that lead to valid results are discussed, as is the way in which a hypothesis is formulated. The concepts of dependent and independent variables are introduced, so that requisite data can be collected using an appropriate sampling technique and then finally analyzed in a statistically valid manner, leading to reliable results. All concepts are presented in a well-motivated, logical, and clear manner. The final part of the book presents complete examples, including a description of the situation, formulation of a hypothesis to be tested, construction of the experiment, a description of the data collected, and the analysis and interpretation of that data. This part brings together the first two parts of the book to provide evidence that the methodologies are valid, appropriate, and applicable. The examples are based on real-life situations and are sufficiently complex, convincing those new to experimental software engineering that the approach is the right one to follow. This book is a valuable resource for anyone involved in software engineering. It would be particularly valuable to those actually engaged in empirical software engineering and the design of experiments. It ought to be required reading for all PhD students; every academic's bookcase should have a copy. Online Computing Reviews Service