Browse Books

Reviewer: Kathleen M. Swigger

The authors' purpose was to “produce a practical guide to the application of classification learning systems.” More specifically, the authors try to develop a technique for evaluating and comparing machine learning algorithms. Having described how statistical methods are used to estimate true error rates, they demonstrate how to use these methods to compare three distinct machine learning techniques—statistical pattern recognition, neural networks, and tree induction methods—using the same data. They then compare the three methods with respect to their prediction accuracy, speed, and explanation facilities. These learning systems are then contrasted with expert systems using similar criteria. The book is extremely useful in that it documents the statistical methods used to compare and evaluate different treatments. It also discusses sampling techniques and shows how to match the appropriate sampling technique with a population size. For example, the authors suggest that with limited samples, one should run repeated tests, leaving out one sample for each run. They also provide information about how to determine whether the sample is representative of the larger population and whether it contains enough features. The most significant portion of the book is dedicated to describing the major machine learning algorithms and showing how each algorithm performs on the same set of data. Statistical pattern recognition methods (linear discriminant, quadratic discriminant, nearest neighbor, and Bayes), neural nets (back propagation using a varying number of hidden layers), and rule-based solution methods (ID3 and AQ15) are compared using sample data from an iris classification exercise and appendicitis and thyroid classification problems. Although the authors failed to discover any major differences among the machine learning algorithms, they provide interesting insights into how to evaluate different methods. The book provides a good introduction to machine learning research and shows how one can compare different machine learning algorithms. Unfortunately, it discusses only three machine learning algorithms, and one of these is basically a mathematical solution. It will be interesting to see whether a case-based reasoner or a genetic algorithm can achieve similar performance. The authors fail to emphasize that a particular machine learning algorithm represents a specific model of learning. Indeed, the basic reason for building a machine learning system is to test a specific theory of learning. For example, neural networks are designed to model a biological view of learning. Comparing the performance of different machine learning algorithms is only one way to compare the different models of learning.