Abstract
Rule-based systems that are easily testable are required for high reliability applications. However, as a rule base evolves, developers prefer incremental evaluation owing to the high cost of regression testing. For quality and reliability improvement, researchers advocate that the evaluation phase be integrated with development: thus, incremental evaluation becomes more important in this context. In this paper, we propose a three-tiered life-cycle model for integrating evaluation in a rule-based system life cycle. We then outline its use to facilitate knowledge acquisition using “goal specification” and its realization in a rule base using “paths.” Path-based validation has been well-accepted in the literature as a reliable method for structural testing. However, extracting paths for every rule base modification for its evaluation incurs an enormous effort. In this paper, we identify situations that can help in incremental path extraction and present some issues that are important for integrating evaluation into the life-cycle of rule-based systems. We also outline how our approach can facilitate handling these issues.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Andert Jr., E.P.: Integrated Design and V&V of Knowledge-Based Systems. In: Notes of the Workshop on Validation and Verification of Knowledge-Based Systems (Eleventh National Conference on Artificial Intelligence), Washington D.C., July 1993, pp. 127–128 (1993)
Ayel, M., Vignollet, L.: SYCOJET and SACCO: Two Tools for Verifying Expert Systems. International Journal of Expert Systems 6(3), 273–298 (1993)
Chander, P.G., Radhakrishnan, T., Shinghal, R.: Quality Issues in Designing and Evaluating Rule-based Systems. In: Notes of the Workshop on Verification & Validation of Knowledge-Based Systems (Thirteenth National Conference on Artificial Intelligence), Portland, Oregon, August 1996, pp. 33–42 (1996)
Chander, P.G., Radhakrishnan, T., Shinghal, R.: Design Schemes for Rule-based Systems. International Journal of Expert Systems: Research and Applications 10(1), 1–36 (1997)
Chander, P.G., Radhakrishnan, T., Shinghal, R.: Issues in Designing Rule-based Systems for Integrated Evaluation. In: Notes of the Workshop on Verification & Validation of Knowledge-Based Systems (Fifteenth National Conference on Artificial Intelligence AAAI-1998), Wisconsin, Madison, pp. 18–24 (July 1998)
Chander, P.G., Shinghal, R., Desai, B.C., Radhakrishnan, T.: An Expert System for Cataloging and Searching Digital Libraries. Expert Systems with Applications 12(4), 405–416 (1997)
Chander, P.G., Shinghal, R., Radhakrishnan, T.: Performance Assesment and Incremental Evaluation of Rule-based Systems. In: Notes of the Workshop on Verification & Validation of Knowledge-Based Systems (Fourteenth National Conference on Artificial Intelligence AAAI-1997), Providence, Rhode Island,, pp. 40–46 (1997)
Chander, P.G., Shinghal, R., Radhakrishnan, T.: Using Goals to Design and Verify Rule Bases. Decision Support Systems 21(4), 281–305 (1997)
Chander, P.G., Shinghal, R., Radhakrishnan, T.: Static Determination of Dynamic Functional Attributes in Rule-based Systems. In: Proceedings of the 1994 International Conference on Systems Research, Informatics and Cybernetics, AI Symposium (ICSRIC 1994), Baden Baden, Germany, pp. 79–84 (August 1994)
Chander, P.G.: On the Design and Evaluation of Rule-based Systems. PhD thesis, Department of Computer Science. Concordia University, Montreal (May 1996)
Chang, C.L., Combs, J.B., Stachowitz, R.A.: A Report on the Expert Systems Validation Associate (EVA). Expert Systems with Applications 1(3), 217–230 (1990)
Ginsberg, A.: Knowledge-Base Reduction: A New Approach to Checking Knowledge Bases for Inconsistency & Redundancy. In: Proceedings of the 7th National Conference on Artificial Intelligence (AAAI 1988), St. Paul, Minnesota, vol. 2, pp. 585–589 (1988)
Ginsberg, A., Williamson, K.: Checking for Quasi First-Order-Logic Knowledge Bases. Expert Systems with Applications 6(3), 321–340 (1993)
Grossner, C., Gokulchander, P., Preece, A., Radhakrishnan, T.: Revealing the Structure of Rule-Based Systems. International Journal of Expert Systems: Research and Applications 9(2), 255–278 (1996)
Grossner, C., Preece, A., Gokulchander, P., Radhakrishnan, T., Suen, C.Y.: Exploring the Structure of Rule Based Systems. In: Proceedings of the 11th National Conference on Artificial Intelligence (AAAI 1993), Washington D.C., pp. 704–709 (1993)
Guida, G., Mauri, G.: Evaluating Performance and Quality of Knowledge-Based Systems: Foundation and Methodology. IEEE transactions in Knowledge and Data engineering 5(2), 204–224 (1993)
Hamilton, D., Kelley, K., Culbert, C.: State-of-the-Practice in Knowledge-based System Verification and Validation. Expert Systems with Applications 3(3), 403–410 (1991)
Jacob, R.J.K., Froscher, J.N.: A Software Engineering Methodology for Rule-Based Systems. IEEE Transactions on Knowledge and Data Engineering 2(2), 173–189 (1990)
Kiper, J.D.: Structural Testing of Rule-Based Expert Systems. ACM Transactions on Software Engineering and Methodology 1(2), 168–187 (1992)
Lee, S., O’Keefe, R.M.: Developing a Strategy for Expert System Verification and Validation. IEEE Transactions on Systems, Man, and Cybernetics 24(4), 643–655 (1994)
Loiseau, S., Rousset, M.-C.: Formal Verification of Knowledge Bases Focused on Consistency: Two Experiments Based on ATMS Techniques. International Journal of Expert Systems 6(3), 273–298 (1993)
Meseguer, P.: Incremental Verification of Rule-based Expert Systems. In: Neumann, B. (ed.) 10th European Conference on Artificial Intelligence, Vienna, Austria, pp. 829–834 (1992)
Plant, R.T.: Expert System Development and Testing: A Knowledge Engineer’s Perspective. Journal of Systems Software 19(2), 141–146 (1992)
Preece, A., Gokulchander, P., Grossner, C., Radhakrishnan, T.: Modeling Rule Base Structure for Expert System Quality Assurance. In: Notes of the Workshop on Validation of Knowledge-Based Systems (Thirteenth International Joint Conference on Artificial Intelligence), Savoie, France, pp. 37–50 (August 1993)
Preece, A., Grossner, C., Gokulchander, P., Radhakrishnan, T.: Structural Validation of Expert Systems: Experience Using a Formal Model. In: Notes of the Workshop on Validation and Verification of Knowledge-Based Systems (Eleventh National Conference on Artificial Intelligence), Washington D.C., pp. 19–26 (July 1993)
Rousset, M.-C.: On the Consistency of Knowledge Bases: The COVADIS System. Computational Intelligence 4(2), 166–170 (1988); Also in ECAI 1988, Proc. European Conference on AI, Munich, August 1-5, pp. 79–84 (1988)
Rushby, J., Crow, J.: Evaluation of an Expert System for Fault Detection, Isolation, and Recovery in the Manned Maneuvering Unit. NASA Contractor a Report CR-187466, SRI International, Menlo Park CA (February 1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chander, P.G., Shinghal, R., Radhakrishnan, T. (1999). Incremental and Integrated Evaluation of Rule-Based Systems. In: Imam, I., Kodratoff, Y., El-Dessouki, A., Ali, M. (eds) Multiple Approaches to Intelligent Systems. IEA/AIE 1999. Lecture Notes in Computer Science(), vol 1611. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-48765-4_31
Download citation
DOI: https://doi.org/10.1007/978-3-540-48765-4_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66076-7
Online ISBN: 978-3-540-48765-4
eBook Packages: Springer Book Archive