Browse Books

Reviewer: Toshiro Kubota

"Kernel methods" refers to a set of techniques for pattern analysis that became quite popular after the introduction of the support vector machine (SVM) in the 1990s. One of the most important pattern analysis problems is classification, where a classifier is designed based on a set of training data. Typically, a classifier first transforms data to a high-dimensional feature space, where separation between different classes of data becomes more prominent, and then uses linear decision boundaries in the feature space to map data into different classes. Kernel-based methods take advantage of the fact that the feature transformation, followed by the classification with linear decision boundaries, can be done by computing inner products between the data and a subset of training data (called support vectors) in the feature space. In addition, the inner product can be computed efficiently with a so-called kernel function. Thus, the computational complexity is not dependent on the dimension of the feature space (like back-propagational neural networks), but on the number of support vectors, which can be quite small or quite large, depending on the problems and the selection of kernel. Another benefit of kernel-based approaches is that the classifier can avoid overfitting to the training data, by controlling the margin of the decision boundaries with respect to the support vectors. Kernel methods are not limited to classification problems, but are also applicable to other pattern analysis problems, including regression, clustering, visualization, and novelty detection. The book describes each of these problems in great depth. The book has little overlap with another book on SVM [1], written recently by the same authors. It contains more up-to-date material, and provides a more in-depth discussion of kernel properties and nonclassification problems. It is divided into three parts: basic concepts, algorithms, and kernel construction. The first part presents an overview of the topic, and discusses the properties of kernels and the capacity of kernel-based learning. To discuss the capacity, the authors employed Rademacher complexity instead of a more conventional Vapnik-Chervonenkis (VC) dimension. I found that this treatment resulted in a more intuitive coverage of the topic. The second part offers a collection of algorithms, including SVM, kernel principal component analysis (PCA), and kernel-based clustering. This part is an excellent summary of recent developments in machine learning. I was most interested in the last part: kernel construction. The kernel approaches have many theoretical advantages over traditional back-propagation neural network approaches. However, their performance in real applications depends strongly on the selection of kernels, and it is often difficult to select and/or construct the right set of kernels. This part is split into four chapters. Chapter 9 describes polynomial-based kernels, including all-subsets and analysis of variance (ANOVA) kernels. Chapter 10 covers kernel construction for text processing. Chapter 11 discusses kernel construction for structured data, in particular strings. Chapter 12 looks at generative models, and centers the discussions on hidden Markov models and Fisher kernels. The authors discuss computational issues associated with evaluating these kernels, and present efficient computational approaches using recursion, dynamic programming, random sampling, and so forth. The chapters provide excellent coverage of kernels applicable to problems with one-dimensional data sets, such as text, deoxyribonucleic acid (DNA), and speech signals, but little coverage of multi-dimensional data sets, such as images and spatial data. The book comes with Matlab and Matlab-like pseudocode, to elucidate often-abstract algorithms. Additional resources, including Matlab code, are available on the authors' Web site. Overall, the book provides an excellent overview of this growing field. I highly recommend it to those who are interested in pattern analysis and machine learning, and especially to those who want to apply kernel-based methods to text analysis and bioinformatics problems. Since the book is highly theoretical, and does not come with many real-world examples, it may not be suited for practitioners. Online Computing Reviews Service