Abstract
The application of the Radial Basis Function neural networks in domains involving prediction and classification of symbolic data requires a reconsideration and a careful definition of the concept of distance between patterns. This distance in addition to providing information about the proximity of patterns should also obey some mathematical criteria in order to be applicable. Traditional distances are inadequate to access the differences between symbolic patterns. This work proposes the utilization of a statistically extracted distance measure for Generalized Radial Basis Function (GRBF) networks. The main properties of these networks are retained in the new metric space. Especially, their regularization potential can be realized with this type of distance. However, the examples of the training set for applications involving symbolic patterns are not all of the same importance and reliability. Therefore, the construction of effective decision boundaries should consider the numerous exceptions to the general motifs of classification that are frequently encountered in data mining applications. The paper supports that heuristic Instance Based Learning (IBL) training approaches can uncover information within the uneven structure of the training set. This information is exploited for the estimation of an adequate subset of the training patterns serving as RBF centers and for the estimation of effective parameter settings for those centers. The IBL learning steps are applicable to both the traditional and the statistical distance metric spaces and improve significantly the performance in both cases. The obtained results with this two-level learning method are significantly better than the traditional nearest neighbour schemes in many data mining problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
S. Haykin, Neural Networks, 2nd edn., MacMillan College Publishing Company: London, 1999.
V.N. Vapnik, Statistical Learning Theory, Wiley: New York, 1998.
T. Poggio and F. Girosi, “Regularization algorithms for learning that are equivalent to multilayer perceptrons,” Science, vol. 247, pp. 978–982, 1990.
T. Poggio and F. Girosi, “Networks for approximation and learning,” Proceedings of the IEEE, vol. 78, pp. 1481–1497, 1990.
P. Baldi and S. Brunak, Bioinformatics, MIT Press: Cambridge, MA, 1998.
C. Stanfill and D. Waltz, “Toward memory-based reasoning,” Communications of the ACM, vol. 29, no. 12, pp. 1213–1228, 1986.
S. Berchtold, D.A. Keim, H.-P. Kriegel, and T. Seidl, “Indexing the solution space: A new technique for nearest neighbor search in high-dimensional space,” IEEE Transactions on Knowledge and Data Engineering, vol. 12, no. 1, Jan./Feb. 2000.
D.R. Wilson and T.R. Martinez, “Improved heterogenous distance functions,” Journal of Artificial Intelligence Research vol. 6, pp. 1–34, 1997.
F. Girosi, “An equivalence between sparse approximation and support vector machines,” Neural Computation, vol. 10, no. 6, pp. 1455–1480, 1998.
T. Elomaa and J. Rousu, “General and efficient multisplitting of numerical attributes,” Machine Learning, vol. 36, pp. 201–244, 1999.
J.C.W. Debuse and V. Jayward-Smith, “Discretisation of continuous commercial database features for a simulated annealing data mining algorithm,” Applied Intelligence, vol. 11, pp. 285–295, 1999.
S. Chen, Y. Wu, and B.L. Lu, “Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks,” IEEE Trans. on Neural Networks, vol. 10, no. 5, Sept. 1999.
S. Papadimitriou, A. Bezerianos, and A. Bountis, “Radial basis function networks as chaotic generators for secure communication systems,” International Journal on Bifurcation and Chaos, vol. 9, no. 1, pp. 221–232, 1999.
A. Bezerianos, S. Papadimitriou, and D. Alexopoulos, “Radial basis function neural networks for the characterization of heart rate variability dynamics,” Artificial Intelligence in Medicine, vol. 15, pp. 215–234, 1999.
P. Bartlett and J. Shawe-Taylor, “Generalization performance of support vector machines and other pattern classifiers,” in Advances in Kernel Methods, Support Vector Learning, MIT Press: Cambridge, MA, pp. 43–54, 1999.
S. Cos and S. Salzberg, “A weighted nearest neighbor algorithm for learning with symbolic features,” Machine Learning, vol. 10, pp. 57–78, 1993.
D. Wettschereck, D.W. Aha, and T. Mohri, “A review and empirical comparison of feature weighting methods for a class of lazy learning algorithms,” AI Review, vol. 11, pp. 273–314, 1997.
T. Hastie and R. Tibshirani, “Discriminant adaptive nearest neighbor classification,” IEEE Pattern Analysis and Machine Intelligence, vol. 18, pp. 607–616, 1996.
X.C. Ling and H. Wang, “Towards optimal weights setting for the 1-nearest neighbour learning algorithm,” AI Review, vol. 11, pp. 255–272, 1997.
T. Cain, M.J. Pazzani, and G. Silverstein, “Using domain knowledge to influence similarity judgement,” in Proceeding of a Case-Based Reasonong Workshop, Washington, DC, Morgan Kaufmann: San Mateo, CA, 1991, pp 191–202.
N. Howe and C. Cardie, “Examining locally varying weights for nearest neighbor algorithms,” in Second International Conference on Case-Based Reasoning, edited by D. Leake and E. Plaza, Lecture Notes in Artificial Intelligence, Springer: Berlin, 1997, pp. 445–466.
D.W. Aha, “Feature weighting for lazy learning algorithms,” in Feature Extraction, Construction and Selection: A Data Mining Perspective, edited by H. Liu and H. Motoda, Kluwer: Norwell, MA, 1998.
G.Towell, J. Shavlik, and M. Noordewier, “Refinement of aproximate domain theories by knowledge-based neural networks,” in Proceedings Eight National Conference on Artificial Intelligence, AAAI Press: Menlo Park, CA, pp. 861–866, 1990.
C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press: Oxford, 1996.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Papadimitriou, S., Mavroudi, S., Vladutu, L. et al. Generalized Radial Basis Function Networks Trained with Instance Based Learning for Data Mining of Symbolic Data. Applied Intelligence 16, 223–234 (2002). https://doi.org/10.1023/A:1014390017000
Issue Date:
DOI: https://doi.org/10.1023/A:1014390017000