Pattern Recognition and Machine Learning (Information Science and Statistics): | Guide books

Pattern Recognition and Machine Learning (Information Science and Statistics)August 2006

Go to Pattern Recognition and Machine Learning (Information Science and Statistics)

August 2006

Author:
Christopher M. Bishop

Publisher:

Springer-Verlag
Berlin, Heidelberg

ISBN:978-0-387-31073-2

Published:01 August 2006

Available at Amazon

Bibliometrics

Sections

2006

Abstract

No abstract available.

Cited By

Contributors

Christopher M Bishop
Microsoft Research Cambridge
- Publication Years1992 - 2014
- Publication counts58
- Citation count4,701
- Available for Download4
- Downloads (cumulative)2,430
- Downloads (12 months)99
- Downloads (6 weeks)21
- Average Downloads per Article608
- Average Citation per Article81
View Full Profile

Index Terms

Pattern Recognition and Machine Learning (Information Science and Statistics)
1. Computing methodologies
  1. Machine learning

Recommendations

Statistics for Machine Learning: Techniques for exploring supervised, unsupervised, and reinforcement learning models with Python and R
Read More
Pattern Recognition & Machine Learning
Read More
3D Face Factorisation for Face Recognition Using Pattern Recognition Algorithms
Abstract
The face is the preferable biometrics for person recognition or identification applications because person identifying by face is a human connate habit. In contrast to 2D face recognition, 3D face recognition is practically robust to illumination ...
Read More

Reviewer: H. Van Dyke Parunak

Chris Bishop’s 1995 monograph [1] established his reputation for technical exposition that is at once lucid and mathematically rigorous. His latest volume sustains and extends that reputation. In more than 700 pages of clear, copiously illustrated text, he develops a common statistical framework that encompasses both pattern recognition and machine learning. This integrated approach promises to make the volume popular for its intended audience: it is a textbook, with a wide range of exercises, instructions to tutors on where to go for full solutions, and the color illustrations that have become obligatory in undergraduate texts. Yet its clarity and comprehensiveness will make it a favorite desktop companion for practicing data analysts. Chapter 1 introduces the book’s basic ideas with the example of fitting a polynomial curve. This mundane problem provides a framework for introducing the distinction between frequentist methods (which seek to minimize the error between the curve and the data) and a Bayesian approach (which maximizes the probability of the model parameters). The book repeatedly sets these two approaches alongside one another, helping readers understand their distinctive perspectives and thus make appropriate use of both families of methods. The curve-fitting example also provides a foundation for reviewing probability theory, the impact of dimensionality, and the need for model regularization, decision theory, and information theory. Chapter 2 provides an extensive review of probability distributions, with special emphasis on the exponential family of distributions, and a brief contrast with nonparametric methods. Chapters 3 and 4 deal with linear models for regression and classification, respectively. Chapter 5 summarizes neural networks. Chapters 6 and 7 deal with kernel methods (including sparse kernel machines in 7). Chapter 8 develops two kinds of graphical models for conditional probabilities: Bayesian networks and Markov random fields. In addition to the value of these models in their own right, they provide a context for introducing latent variables. Both graphical models and latent variables surface repeatedly in subsequent chapters, tying together the various concepts in the book. Chapter 9 introduces mixture models, motivating them by K-means clustering and then developing the expectation maximization algorithm as an important tool for assigning data points to different components of a mixture. Chapters 10 and 11 provide approximate methods for dealing with distributions that are not analytically tractable. Chapter 10 provides deterministic methods, including variational inference and expectation propagation. Chapter 11 discusses sampling methods that can be used to study a distribution stochastically. Chapter 12 extends the discussion of latent variables in previous chapters to include continuous latent variables, as a way of motivating reduction of dimensionality through various forms of principal component analysis. Up to this point, the book has focused on independent, identically distributed (IID) data. Chapter 13 introduces methods for dealing with sequential data, where the probability of each point depends on some of the preceding points. The discussion includes Markov models, hidden Markov models, and linear dynamical systems. For problems that do not yield to any one of the models considered up to this point, chapter 14 offers mechanisms for combining models, including Bayesian averaging, committees, boosting, and tree-based and conditional mixture models. The book concludes with five appendices, describing data sets that are available for experimentation, some commonly used probability distributions, basic properties of matrices, the calculus of variations, and Lagrange multipliers. The bibliography of over 400 entries includes some prepublication references as far ahead as 2008, and the book has a thorough index. In spite of his broad reach, Bishop does not treat every machine learning method. Reinforcement learning, briefly mentioned in the introduction, is explicitly declared out of scope. And much more can be said about any of the individual methods that are mentioned. The strength of this volume is in providing a common framework for understanding many different approaches. The reader who masters Bishop’s treatment of any of these topics will be able to make responsible use of their more basic techniques, and be fully prepared to follow his references to more extensive treatments in the literature. Online Computing Reviews Service

Reviewer: Luminita State

This accessible monograph seeks to provide a comprehensive introduction to the fields of pattern recognition and machine learning. It presents a unified treatment of well-known statistical pattern recognition techniques. This is accomplished by supplying a deep analysis of their underlying intimate structure, and presenting recent developments in these areas. The book is structured into 14 chapters and three appendices. The first chapter motivates the approaches developed later on by analyzing two typical examples: recognition of handwritten digits and polynomial curve fitting. A basic knowledge of probability theory and mathematical statistics is briefly exposed in the second chapter. The most commonly used linear models for regression and classification are presented in the next two chapters. The connectionist computational model, conventionally referred to as neural networks, and some of the basic learning algorithms are investigated next. The concept of a kernel formulated as an inner product in a feature space, and a variety of learning algorithms based on nonlinear kernels are explored in chapter 6. The discussion in chapter 7 focuses on sparse kernel machines, for example kernel-based algorithms that have sparse solutions, so that predictions for new inputs depend only on the kernel function evaluated at a subset of the training data points. Here, support vector machines (SVMs) and relevance vector machines (RLVs) are explained in detail, pointing out their relative advantages and respective pitfalls. Given the high complexity of most practical pattern recognition problems, the diagrammatic representations of probability distributions are useful for visualizing the structure of a given probabilistic model, designing new ones, and gaining insights into the properties of the model. Chapter 8 presents two of the basic graphical models: Bayesian networks and Markov random fields (Markov networks). Since the distribution of the observed variables results from a marginalization of the joint distribution over observed and latent variables, the introduction of latent variables allows representations of certain relatively complicated distributions in terms of simpler components. Chapter 9 is focused on mixture models and the expectation maximization (EM) method as a general technique for finding maximum likelihood estimators in latent variable models. The discussion on mixture distributions begins with the problem of finding clusters in a set of data points using the k -means algorithm. The latent variable view of mixture distributions corresponds to the view that latent variables define assignments of data points to specific components of the mixture. In the final section of the chapter, it is proven that the k -means algorithm corresponds to a particular nonprobabilistic limit of EM, applied to mixtures of Gaussians. In practice, sampling methods frequently prove to be computationally demanding, which limits their use to small-scale problems. In chapter 10, deterministic approximation schemes, called variational inferences or variational Bayes, are presented. The proposed techniques are based on analytical approximations to the posterior distribution, and it is argued that some of them scale well to large applications. Approximate inference algorithms based on numerical sampling (Monte Carlo techniques) are investigated in the next chapter. Several sampling strategies—for example, rejection sampling and importance sampling—for evaluating expectations of functions and their pitfalls in case of spaces of high dimensionalities are presented in Section 1 of the chapter. Next, the very general framework, Markov chain Monte Carlo (MCMC), which allows sampling from a large class of distributions and scales well with the dimensionality of the sample space, is presented. Chapter 12 investigates the properties of some of the most frequently used techniques for dimensionality reduction: feature extraction, and lossy data compression, known as principal component analysis (PCA). The discussion starts with a nonprobabilistic treatment of PCA; the author then explains how PCA arises naturally as the maximum solution to a particular form of linear-Gaussian latent variable model. The probabilistic formulation allows the use of EM for parameter estimation, principled extensions to mixtures of PCA models, and Bayesian formulations in order to automatically determine the number of principal components from data. Two of the most important examples of state space models are the hidden Markov model (HMM), in which the latent variables are discrete, and linear dynamical systems (LDS), in which the latent variables are Gaussian; these are investigated in the next chapter. The final chapter discusses combinations of models, which are known in the literature as committee machines. The book can be used by advanced undergraduates and graduate students who have completed courses in computer programming, probability, calculus, and linear algebra. The illustrative examples and exercises proposed at the end of each chapter are welcome, and substantially facilitate the understanding of the content. Moreover, the solutions of the proposed exercises and the sets of test data are available on the Web. The list of references contains the most significant published work in the field. The book, which provides several new views, developments, and results, is appropriate for both researchers and students who work in machine learning, statistics, computer science, data mining, computer vision, and signal processing. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Save to Binder

Sections

Cited By

Save to Binder

Index Terms

Recommendations

Statistics for Machine Learning: Techniques for exploring supervised, unsupervised, and reinforcement learning models with Python and R

Pattern Recognition & Machine Learning

3D Face Factorisation for Face Recognition Using Pattern Recognition Algorithms

Access critical reviews of Computing literature here