Abstract
This paper describes the result of our study on neural learning to solve the classification problems in which data is unbalanced and noisy. We conducted the study on three different neural network architectures, multi-layered Back Propagation, Radial Basis Function, and Fuzzy ARTMAP using three different training methods, duplicating minority class examples, Snowball technique and multidimensional Gaussian modeling of data noise. Three major issues are addressed: neural learning from unbalanced data examples, neural learning from noisy data, and making intentional biased decisions. We argue that by properly generated extra training data examples around the noise densities, we can train a neural network that has a stronger capability of generalization and better control of the classification error of the trained neural network. In particular, we focus on problems that require a neural network to make favorable classification to a particular class such as classifying normal(pass)/abnormal(fail) vehicles in an assembly plant. In addition, we present three methods that quantitatively measure the noise level of a given data set. All experiments were conducted using data examples downloaded directly from test sites of an automobile assembly plant. The experimental results showed that the proposed multidimensional Gaussian noise modeling algorithm was very effective in generating extra data examples that can be used to train a neural network to make favorable decisions for the minority class and to have increased generalization capability.
Article PDF
Similar content being viewed by others
References
Y. Lu, H. Guo, and L. Feldkamp, “Robust neural learning from unbalanced data examples,” IEEE IJCNN, 1998.
C.H. Dagli (Eds.), Artificial Neural Networks for Intelligent Manufacturing, 1992.
B. Kosko, Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence, Prentice-Hall, 1992.
D. Mackay, “Bayesian methods for adaptive models,” Ph.D thesis, CIT, 1991.
B. Irie and S. Miyake, “Capabilities of three-layered perceptrons,” in Proc. of the International Conference on Neural Networks, pp. 641–648, 1988.
D.E. Rummelhart and J.L. McClelland. Parallel distributed processing: Explorations in the microstructure of cognition, vol 1: Foundations. MIT Press: Cambridge, MA, 1986.
J. Moody and C.J. Darken, “Fast learning in networks of locally tuned processing units,” Neural Computation, vol. 1, pp. 281–294, 1989.
G.A. Carpenter and S. Grossberg, “ART 2: Self-organization of stable category recognition codes for analog input patterns,” Applied Optics, pp. 4919–4930, 1987.
G.A. Carpenter, S. Grossberg, N. Markuzon et al., “Fuzzy ARTMAP: An adaptive resonance architecture for incremental learning of analog maps”, IJCNN, June 1992, pp. 309–314.
G.A. Carpenter, S. Grossberg, N. Markuzon, J.H. Reynolds, and D.B. Rosen, “Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps,” IEEE Trans. on Neural Networks, vol. 3, pp. 698–713, 1992.
K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press Inc., 1972.
S.M. Weiss and C.A. Kulikowski, Computer Systems that Learn, Morgan Kaufmann Publishers, Inc., 1991.
S. Amari, N. Murata, K.-R. Muller, M. Finke, and H. Yang, “Statistical theory of overtraining-Is cross-validation asymptotically effective?,” Advances in Neural Information Processing Systems 8, Proceedings of the 1995 Conference, David S. Touretzky, Michael C. Mozer, and Michael E. Hasselmo (Eds.), 1996, pp. 176–182.
R.Y. Rubinstein, Simulation and the Monte Carlo method, John Wiley & Sons, 1981.
M.T. Musavi, K.H. Chan, D.M. Hummels, and K. Kalantri, “On the generalization ability of neural network classifiers,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 16, no. 6, pp. 659–663, 1994.
J. Wang and J. Jean, “Resolve multifont character confusion with neural network,” Pattern Recognition, vol. 26, no. 1, pp. 173–187, 1993.
Rights and permissions
About this article
Cite this article
Murphey, Y.L., Guo, H. & Feldkamp, L.A. Neural Learning from Unbalanced Data. Applied Intelligence 21, 117–128 (2004). https://doi.org/10.1023/B:APIN.0000033632.42843.17
Issue Date:
DOI: https://doi.org/10.1023/B:APIN.0000033632.42843.17