Quantitative results concerning the utility of explanation-based learning

doi:10.1016/0004-3702(90)90059-9

Artificial Intelligence

Volume 42, Issues 2–3, March 1990, Pages 363-391

https://doi.org/10.1016/0004-3702(90)90059-9 Get rights and content

Abstract

In order to solve problems effectively, a problem solver must be able to exploit domain-specific search control knowledge. Although previous research has demonstrated that explanation-based learning is a viable approach for acquiring such knowledge, in practice the control knowledge learned via EBL may not be useful. To be useful, the cumulative benefits of applying the knowledge must outweigh the cumulative costs of testing whether the knowledge is applicable. Unlike most previous systems that use EBL, the PRODIGY system evaluates the costs and benefits of the control knowledge it learns. Furthermore, the system produces useful control knowledge by actively searching for “good” explanations—explanations that can be profitably employed to control problem solving. This paper summarizes a set of experiments measuring the effectiveness of PRODIGY's EBL method (and its components) in several different domains.

References (40)

F.M. Brown
An experimental logic based on the fundamental deduction principle
Artificial Intelligence
(1986)
J.G. Carbonell et al.
Learning by experimentation
W.W. Cohen
Generalizing number and learning from multiple examples in explanation based learning
R. Fikes et al.
Learning and executing generalized robot plans
Artificial Intelligence
(1972)
C.L. Forgy
Rete: A fast algorithm for the many pattern/many object pattern matching problem
Artificial Intelligence
(1982)
C.A. Knoblock
Learning hierarchies of abstraction spaces
R.E. Korf
Macro-operators: A weak method for learning
Artificial Intelligence
(1985)
S. Markovitch et al.
The role of forgetting in learning
S. Minton et al.
PRODIGY 2.0: The manual and tutorial
R.J. Mooney
Generalizing the order of operators in macro-operators

R.J. Mooney et al.

A domain independent explanation-based generalizer

F. van Harmelen et al.

Explanation-based generalisation = partial evaluation

Artificial Intelligence

(1988)

M. Zweben et al.

Improving operationality with approximate heuristics

J.G. Carbonell

Derivational analogy: A theory of reconstructive problem solving and expertise acquisition

J.G. Carbonell et al.

The world modelers project: Objectives and simulator architecture

J.G. Carbonell et al.

Integrating derivational analogy into a general problem-solving architecture

P.W. Cheng et al.

Inducing iterative rules from experience: The FERMI experiment

S.A. Chien

Extending explanation-based learning: Failure-driven schema refinement

G.F. DeJong et al.

Explanation-based learning: An alternative view

Machine Learning

(1986)

M.R. Garey et al.

Computers and Intractability: A Guide to the Theory of NP-Completeness

(1979)

Cited by (193)

CPCES: A planning framework to solve conformant planning problems through a counterexample guided refinement
2020, Artificial Intelligence
We introduce cpces, a novel planner for the problem of deterministic conformant planning. cpces solves the problem by producing candidate plans based on a sample of the initial belief state, searching for counter-examples to these plans, and assigning these counter-examples to the sample, until a valid plan has been produced or the problem has been proved unfeasible. On top of providing a means to compute a conformant plan, the sample can also be understood as a justification for the plan being found, or relevant reasons why a plan cannot be found. We study the theoretical properties that cpces enjoys—correctness, completeness, and optimality—and how the several variants of cpces we describe differ in behaviour. Moreover, we establish a theoretical connection between the cpces framework and well-known concepts from the literature such as tags and width. With this connection we prove the worst case complexity for some variants of cpces.
Finally, we show how cpces can be used in a more incremental fashion by learning sequencing of actions from the previous plan being found. Such a technique mimics the use of macro-operators, widely used in automated planning to speedup resolution.
Our theoretical analysis is accompanied with a thorough experimental evaluation of the (many) possible incarnations of cpces. This not only proves our theoretical findings from an empirical perspective, but also shows that cpces is able to handle problems that have been traditionally hard to solve by the existing conformant planners, whilst remaining competitive over “easier” conformant planning problems. Importantly, cpces is able to prove many unsolvable conformant planning problems as such, extending substantially the reach of conformant planners.
Dynamic maintenance case base using knowledge discovery techniques for case based reasoning systems
2020, Theoretical Computer Science
Citation Excerpt :
Other researchers have been interested in solving the problem of CBR system's efficiency which can be defined as the average time for solving a problem [9]. We start this Section by talking about the traditional CBM methods such as random deletion policy [10], ironically policy and utility deletion [11]. The drawback of these methods is that the competence of the reduced case base is not always guaranteed to be preserved.
The achievement of a Case Based Reasoning (CBR) system is strongly related to the quality of case data and the rapidity of the retrieval process that depends on the quantity of the cases. This quality can diminish especially when the number of cases gets outsized. To guarantee this quality, maintenance the case base becomes essentially. Much existing maintenance CBR approaches focus on the performance of the CBR or the study of the case base (CB) competence. Even though the two points are directly related, there is a few research on using strategies at both points at the same time. Furthermore, the proposed methods are not dynamic, they are not suitable for the frequently change in learning process. In this paper, we propose maintenance CBR method based on well-organized machine learning techniques, in the process of improving the competence and the performance of the CB and can handle incremental cases which evolve over time. We support our approach with empirical evaluation using different benchmark data sets to show the effectiveness of our method.
Explanation-based learning with analogy for impasse resolution
2016, Expert Systems with Applications
This paper proposes an algorithm for the inclusion of analogy into Explanation-Based Learning (EBL). Analogy can be used when an impasse is reached to extend the deductive closure of EBL’s domain theory. This enables the generation of control laws, via EBL, for hardware which is not catered for in the domain theory. This advantage addresses a problem which represents a dearth in the current literature. Integrated Modular Avionics (IMA) literature has thus far been concerned with the architectural considerations. This paper seeks to address the impact of hardware changes on the controllers within an IMA architecture. An algorithm is proposed and applied to control an aviation platform with an incomplete domain theory. Control rules are generated when no deductive explanations are possible, which still reflect the intent of the domain theory.
Case-based maintenance: Structuring and incrementing the case base
2015, Knowledge-Based Systems
To avoid performance degradation and maintain the quality of results obtained by the case-based reasoning (CBR) systems, maintenance becomes necessary, especially for those systems designed to operate over long periods and which must handle large numbers of cases. CBR systems cannot be preserved without scanning the case base. For this reason, the latter must undergo maintenance operations.
The techniques of case base’s dimension optimization is the analog of instance reduction size methodology (in the machine learning community). This study links these techniques by presenting case-based maintenance in the framework of instance based reduction, and provides: first an overview of CBM studies, second, a novel method of structuring and updating the case base and finally an application of industrial case is presented.
The structuring combines a categorization algorithm with a measure of competence CM based on competence and performance criteria. Since the case base must progress over time through the addition of new cases, an auto-increment algorithm is installed in order to dynamically ensure the structuring and the quality of a case base. The proposed method was evaluated through a case base from an industrial plant. In addition, an experimental study of the competence and the performance was undertaken on reference benchmarks. This study showed that the proposed method gives better results than the best methods currently found in the literature.
Memory and forgetting: An improved dynamic maintenance method for case-based reasoning
2014, Information Sciences
Citation Excerpt :
Therefore, in order to improve the overall performance of the system, the establishment of a suitable case base maintenance method is urgently needed which can control the growth rate of case base while maintaining a high accuracy of problem solving. Many case base maintenance methods have been proposed, and these can be divided into efficiency-oriented methods [3,18] and performance-oriented [11,22–24] methods. For the former, a simple policy is random deletion.
The problem solving performance of a case-based reasoning (CBR) system is closely related to the quantity and quality of the cases stored in the case base. With the continuous growth of the size of the case base, the so called “swamping problem” may occur when the time cost of retrieval exceeds the benefit of the accuracy. From the perspective of cognitive science, a dynamic maintenance method improved by selective memory and intentional forgetting for CBR is proposed, which can imitate the memory function of the human brain to selectively save new cases, update the forgotten values and intentionally delete the old cases. The experiments show the effectiveness of the proposed method. The selective memory and international forgetting policy can significantly reduce the time and space complexity, and retain or improve the accuracy of the CBR classifier, thus improving the performance of CBR.
Evaluating Case-Base Maintenance algorithms
2014, Knowledge-Based Systems
Citation Excerpt :
Although the learning ability of CBR is an important advantage, this characteristic is not free of drawbacks. For example, the addition of new cases to the case-base could degrade a system’s performance [21,38,51] because case-bases with many cases need a long time to retrieve cases similar to an input query. Some reasons for increase in retrieval time are the scalability of the data structures that represent the case-base, and the case descriptions for both problem and solution.
The success of a Case-Based Reasoning (CBR) system closely depends on its knowledge-base, named the case-base. The life cycle of CBR systems usually implies updating the case-base with new cases. However, it also implies removing useless cases for reasons of efficiency. This process is known as Case-Base Maintenance (CBM) and, in recent decades, great efforts have been made to automatise this process using different kind of algorithms (deterministic and non-deterministic). Indeed, CBR system designers find it difficult to choose from the wealth of algorithms available to maintain the case-base. Despite the importance of such a key decision, little attention has been paid to evaluating these algorithms. Although classical validation methods have been used, such as Cross-Validation and Hold-Out, they are not always valid for non-deterministic algorithms. In this work, we analyse this problem from a methodological point of view, providing an exhaustive review of these evaluation methods supported by experimentation. We also propose a specific methodology for evaluating Case-Base Maintenance algorithms (the $α β$ evaluation). Experiment results show that this method is the most suitable for evaluating most of the algorithms and datasets studied.

View all citing articles on Scopus

^☆: Revised version of the paper that won the Artificial Intelligence Journal Best Paper Award at AAAI-88, St. Paul, MN.

^∗∗: Current address: Sterling Federal Systems, Artificial Intelligence Branch, NASA Ames Research Center, Mail Stop 244-17, Moffett Field, CA 94035, USA.

View full text

Quantitative results concerning the utility of explanation-based learning☆

Abstract

Artificial Intelligence

Artificial Intelligence

Artificial Intelligence

Artificial Intelligence

Artificial Intelligence

Derivational analogy: A theory of reconstructive problem solving and expertise acquisition

The world modelers project: Objectives and simulator architecture

Integrating derivational analogy into a general problem-solving architecture

Inducing iterative rules from experience: The FERMI experiment

Extending explanation-based learning: Failure-driven schema refinement

Explanation-based learning: An alternative view

Machine Learning

Computers and Intractability: A Guide to the Theory of NP-Completeness