Many scientific and industrial problems can be better understood by learning from samples of the task at hand. For this reason, the machine learning and statistics communities devote considerable research effort on generating inductive-learning algorithms that try to learn the true "concept" of a task from a set of its examples. Often times, however, one has additional resources readily available, but largely unused, that can improve the concept that these learning algorithms generate. These resources include available computer cycles, as well as prior knowledge describing what is currently known about the domain. Effective utilization of available computer time is important since for most domains an expert is willing to wait for weeks, or even months, if a learning system can produce an improved concept. Using prior knowledge is important since it can contain information not present in the current set of training examples.In this thesis, I present three "anytime" approaches to connectionist theory refinement. Briefly, these approaches start by translating a set of rules describing what is currently known about the domain into a neural network, thus generating a knowledge-based neural network (KNN). My approaches then utilize available computer time to improve this KNN by continually refining its weights and topology. My first method, TopGen, searches for good "local" refinements to the KNN topology. It does this by adding nodes to the KNN in a manner analogous to symbolically adding rules and conjuncts to an incorrect rule base. My next approach, R scEGENT, uses genetic algorithms to find better "global" changes to this topology. R scEGENT proceeds by using (a) the domain-specific rules to help create the initial population of KNNs and (b) crossover and mutation operators specifically designed for KNNs. My final algorithm, A scDDEMUP, searches for an "ensemble" of KNNs that work together to produce an effective composite prediction. A scDDEMUP works by using genetic algorithms to continually create new networks, keeping the set of networks that are as accurate as possible while disagreeing with each other as much as possible. Empirical results show that these algorithms successfully achieve each of their respective goals.
Cited By
- Esmeir S and Markovitch S (2007). Anytime Learning of Decision Trees, The Journal of Machine Learning Research, 8, (891-933), Online publication date: 1-May-2007.
- Esmeir S and Markovitch S Any time induction of decision trees Proceedings of the 21st national conference on Artificial intelligence - Volume 1, (348-355)
- Esmeir S and Markovitch S Interruptible anytime algorithms for iterative improvement of decision trees Proceedings of the 1st international workshop on Utility-based data mining, (78-85)
- Burns B and Danyluk A (2000). Feature Selection vs Theory Reformulation, Machine Language, 38:1-2, (89-107), Online publication date: 1-Jan-2000.
Index Terms
- An anytime approach to connectionist theory refinement: refining the topologies of knowledge-based neural networks
Recommendations
Connectionist theory refinement: genetically searching the space of network topologies
An algorithm that learns from a set of examples should ideally be able to exploit the available resources of (a) abundant computing power and (b) domain-specific knowledge to improve its ability to generalize. Connectionist theory-refinement systems, ...
Anytime focal search with applications
IJCAI'18: Proceedings of the 27th International Joint Conference on Artificial IntelligenceFocal search (FS) is a bounded-suboptimal search (BSS) variant of A*. Like A*, it uses an open list whose states are sorted in increasing order of their f-values. Unlike A*, it also uses a focal list containing all states from the open list whose f-...
Predictive connectionist approach to speech recognition
Nonlinear Speech Modeling and ApplicationsThis tutorial describes a context-dependent Hidden Control Neural Network (HCNN) architecture for large vocabulary continuous speech recognition. Its basic building element, the context-dependent HCNN model, is connectionist network trained to capture ...