Skip to main content
SpringerLink
Log in
Book cover

Mexican International Conference on Artificial Intelligence

MICAI 2011: Advances in Soft Computing pp 351–362Cite as

Improving the MLP Learning by Using a Method to Calculate the Initial Weights of the Network Based on the Quality of Similarity Measure

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7095))

Abstract

This work presents a technique that integrates the backpropagation learning method with a method to calculate the initial weights in order to train the Multilayer Perceptron Model. The method to calculate the initial weights of the MLP is based on the quality of similarity measure proposed on the framework of the extended Rough Set Theory. Experimental results show that the proposed initialization method performs better than other methods used to calculate the weight of the features, so it is an interesting alternative to the conventional random initialization.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Hocenski, Z., Antunovic, M., Filko, D.: Accelerated Gradient Learning Algorithm for Neural Network Weights Update. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds.) KES 2008, Part I. LNCS (LNAI), vol. 5177, pp. 49–56. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  2. Kim, M., Choi, C.: A New Weight Initialization Method for the MLP with the BP in Multiclass Classification Problems. Neural Processing Letters 6, 11–23 (1997)

    Article  Google Scholar 

  3. Fu, X., Zhang, S., Pang, Z.: A Resource Limited Immune Approach for Evolving Architecture and Weights of Multilayer Neural Network. In: Tan, Y., Shi, Y., Tan, K.C. (eds.) ICSI 2010. LNCS, vol. 6145, pp. 328–337. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  4. Almeida, L.M., Ludermir, T.B.: An Evolutionary Approach for Tuning Artificial Neural Network Parameters. In: Corchado, E., Abraham, A., Pedrycz, W. (eds.) HAIS 2008. LNCS (LNAI), vol. 5271, pp. 156–163. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  5. Hornik, K., et al.: Multilayer feedforward neural network are universal approximators. Neural Networks 2, 359–366 (1989)

    Article  Google Scholar 

  6. Faúndez-Zanuy, M.: Nonlinear Speech Processing: Overview and Possibilities in Speech Coding. In: Chollet, G., Esposito, A., Faúndez-Zanuy, M., Marinaro, M. (eds.) Nonlinear Speech Modeling and Applications. LNCS (LNAI), vol. 3445, pp. 15–42. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  7. Kim, Y.K., Ra, J.B.: Weight value initialization for improving training speed in the backpropagation networks. In: Kim, Y. (ed.) IEEE International Joint Conference on NN, pp. 2396–2401 (1991)

    Google Scholar 

  8. Drago, G., Ridella, S.: Statistically controlled activation weight initialization. IEEE Transaction on Neural Networks 3(4), 627–631 (1992)

    Article  Google Scholar 

  9. Wessels, L., Barnard, E.: Avoiding false local minima by proper initialization. IEEE Transaction on Neural Networks 3(6), 899–905 (1992)

    Article  Google Scholar 

  10. Denoeux, T., Lengelle, R.: Initializing back propagation networks with prototypes. Neural Networks 6, 351–363 (1993)

    Article  Google Scholar 

  11. Thimm, G., Fiesler, E.: High-order and multilayer perceptron initialization. IEEE Trans. on Neural Networks 2, 349–359 (1997)

    Article  Google Scholar 

  12. Almeida, L.M., Ludermir, T.B.: An Evolutionary Approach for Tuning Artificial Neural Network Parameters. In: Corchado, E., Abraham, A., Pedrycz, W. (eds.) HAIS 2008. LNCS (LNAI), vol. 5271, pp. 156–163. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  13. Adam, S., Karras, D.A., Vrahatis, M.N.: Revisiting the Problem of Weight Initialization for Multi-Layer Perceptrons Trained with Back Propagation. In: Köppen, M., Kasabov, N., Coghill, G. (eds.) ICONIP 2008. LNCS, vol. 5507, pp. 308–315. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  14. Nieminen, P., Kärkkäinen, T.: Ideas about a Regularized MLP Classifier by Means of Weight Decay Stepping. In: Kolehmainen, M., Toivanen, P., Beliczynski, B. (eds.) ICANNGA 2009. LNCS, vol. 5495, pp. 32–41. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  15. Rumelhart, D.E., Hilton, G.E., Williams, R.J.: Learning Representations by Backpropagation Errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

  16. Rocha, M., Cortez, P.C., Neves, J.: Simultaneous Evolution of Neural Network Topologies and Weights for Classification and Regression. In: Cabestany, J., Prieto, A.G., Sandoval, F. (eds.) IWANN 2005. LNCS, vol. 3512, pp. 59–66. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  17. Filiberto, Y., Bello, R., Caballero, Y., Larrua, R.: Using PSO and RST to Predict the Resistant Capacity of Connections in Composite Structures. In: González, J.R., Pelta, D.A., Cruz, C., Terrazas, G., Krasnogor, N. (eds.) NICSO 2010. SCI, vol. 284, pp. 359–370. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  18. Pawlak, Z.: Rough Sets. International Journal of Computer and Information Sciences 11, 341–356 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  19. Skowron, A.: Logic, algebra and computer science, Helena Rasiowa and Cecylia Rauszer in Memoriam. In: Bulletin of the Section of Logic, pp. 1–215 (1996)

    Google Scholar 

  20. Slowinski, R., Vanderpooten, D.: A generalized definition of rough approximations based on similarity. IEEE Transactions on Data and Knowledge Engineering 12(2), 331–336 (2000)

    Article  Google Scholar 

  21. Pawlak, Z., Skowron, A.: Rough sets: Some extensions. Information Sciences 177, 28–40 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Networks. IEEE Service Center, Piscataway (1995)

    Google Scholar 

  23. Asuncion, A., Newman, D.: UCI machine learning repository. A study of the behaviour of several methods for balancing machine learning training data. SIGKDD Explorations 6(1), 20–29 (2007)

    Google Scholar 

  24. Mitchell, T.: Machine Learning. In: Science/Engineering/Math., McGraw Hill, Portland (1997)

    Google Scholar 

  25. Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research (7), 1–30 (2006)

    Google Scholar 

  26. Myles, A., et al.: Estimating MLP generalisation ability without a test set using fast, approximate Leave-one-out cross-validation. Neural Computing and Applications 5, 134–151 (1997)

    Article  Google Scholar 

  27. Wettschereckd, D.: A description of the mutual information approach and the variable similarity metric. Technical report, Artificial Intelligence Research Division. German National Research Center for Computer Science, Sankt Augustin (1995)

    Google Scholar 

  28. Kononenko, I.: Estimating attributes: Analysis and extensions of RELIEF. In: Proc. European Conf. on Machine Learning (1994)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Sciences, University of Camaguey, Cuba

    Yaima Filiberto Cabrera & Yailé Caballero Mota

  2. Department of Computer Sciences, Central University of Las Villas, Cuba

    Rafael Bello Pérez

  3. Department of Languages and Computer Science, University of Malaga, Spain

    Gonzalo Ramos Jimenez

Authors
  1. Yaima Filiberto Cabrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rafael Bello Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yailé Caballero Mota
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gonzalo Ramos Jimenez
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Mexican Petroleum Institute (IMP), Eje Central Lazaro Cardenas Norte, 152, Col. San Bartolo Atepehuacan, CP 07730, Mexico DF, Mexico

    Ildar Batyrshin

  2. National Polytechnic Institute (IPN), Center for Computing Research (CIC), Av. Juan Dios Bátiz, s/n, Col. Nueva Industrial Vallejo, CP 07738, Mexico D.F., Mexico

    Grigori Sidorov

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Filiberto Cabrera, Y., Bello Pérez, R., Mota, Y.C., Jimenez, G.R. (2011). Improving the MLP Learning by Using a Method to Calculate the Initial Weights of the Network Based on the Quality of Similarity Measure. In: Batyrshin, I., Sidorov, G. (eds) Advances in Soft Computing. MICAI 2011. Lecture Notes in Computer Science(), vol 7095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25330-0_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-25330-0_31

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-25329-4

  • Online ISBN: 978-3-642-25330-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation