Skip to main content
SpringerLink
Log in

Hyperconic Multilayer Perceptron

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

This paper introduces the design of the hyperconic multilayer perceptron (HC-MLP). Complex non-linear decision regions for classification purposes are generated by quadratic hyper-surfaces spawned by the hyperconic neurons in the hidden layer (for instance, spheres, ellipsoids, paraboloids, hyperboloids and degenerate conics). In order to generate quadratic hyper-surfaces, the hyperconic neurons’ transfer function includes the estimation of a quadratic polynomial. The proper assignment of decision regions to classes is achieved in the output layer by using spheres to determine whether a point is inside or outside the spherical region. The particle swarm optimization algorithm is used for training the HC-MLP. The learning of the HC-MLP selects the best conic surface that separates the data set vectors. For illustration purposes, two experiments are conducted using two distributions of synthetic data in order to show the advantages of HC-MLP when the patterns between classes are contiguous. Furthermore a comparison to the traditional multilayer perceptron is carried out to evaluate the complexity (in terms of the number of estimated patterns) and classification accuracy. HC-MLP is the principal component to implement a diagnosis system to detect faults in an induction motor and to implement an image segmentation system. The performance of HC-MLP is compared to other leading algorithms by using 4 databases commonly used in related literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Arena P, Fortuna L, Occhipinti L, Xibilia M (1994) Neural networks for quaternion-valued function approximation. In: 1994 IEEE international symposium on circuits and systems, 1994. ISCAS ’94, vol 6, pp 307–310

  2. Astorino A, Fuduli A, Gaudioso M (2010) Dc models for spherical separation. J Glob Optim 48(4):657–669

    Article  MathSciNet  MATH  Google Scholar 

  3. Astorino A, Fuduli A, Gaudioso M (2012) Margin maximization in spherical separation. Comput Optim Appl 53(2):301–322

    Article  MathSciNet  MATH  Google Scholar 

  4. Bache K, Lichman M (2013) Uci machine learning repository. http://archive.ics.uci.edu/ml

  5. Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, Secaucus, NJ

    MATH  Google Scholar 

  6. Blake A, Rother C, Brown M, Perez P, Torr P (2004) Interactive image segmentation using an adaptive gmmrf model. In: Computer vision-ECCV 2004, pp 428–441. Springer, Berlin

  7. Bookstein FL (1979) Fitting conic sections to scattered data. Comput Gr Image Process 9(1):56–71

    Article  Google Scholar 

  8. Buchholz S, Sommer G (2000) A hyperbolic multilayer perceptron. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks, 2000. IJCNN 2000, vol 2, pp 129–133

  9. Cantu-Paz E, Kamath C (2005) An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Trans Syst Man Cybern B Cyber 35(5):915–927

    Article  Google Scholar 

  10. Carvalho M, Ludermir T (2007) Particle swarm optimization of neural network architectures and weights. In: 7th international conference on hybrid intelligent systems, 2007. HIS 2007. pp 336–339

  11. Casasent DP, Barnard E (1990) Adaptive-clustering optical neural net. Appl Opt 29(17):2603–2615

    Article  Google Scholar 

  12. Casasent D, Natarajan S (1995) A classifier neural net with complex-valued weights and square-law nonlinearities. Neural Netw 8(6):989–998

    Article  Google Scholar 

  13. Cheng H, Jiang X, Sun Y, Wang J (2001) Color image segmentation: advances and prospects. Pattern Recognit 34(12):2259–2281

    Article  MATH  Google Scholar 

  14. Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the sixth international symposium on micro machine and human science, 1995. MHS ’95. pp 39–43

  15. Engelbrecht AP (2006) Fundamentals of computational swarm intelligence. Wiley, New York

    Google Scholar 

  16. Garro B, Sossa H, Vzquez R (2011) Back-propagation vs particle swarm optimization algorithm: which algorithm is better to adjust the synaptic weights of a feed-forward ann? Int J Artif Intel 7(11A):208–218

    Google Scholar 

  17. Georgiou G, Koutsougeras C (1992) Complex domain backpropagation. IEEE Trans Circuits Syst II Analog Digit Sig Process 39(5):330–334

    Article  MATH  Google Scholar 

  18. Hernandez-Lopez FJ, Rivera M (2014) Change detection by probabilistic segmentation from monocular view. Mach Vis Appl 25(5):1175–1195

    Article  Google Scholar 

  19. Hong ZQ, Yang JY (1991) Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recognit 24(4):317–324

    Article  MathSciNet  Google Scholar 

  20. Kiranyaz S, Ince T, Yildirim A, Gabbouj M (2009) Evolutionary artificial neural networks by multi-dimensional particle swarm optimization. Neural Netw 22(10):1448–1462

    Article  Google Scholar 

  21. Lin S, Zeng J, Xu Z (2015) Error estimate for spherical neural networks interpolation. Neural Process Lett 42(2):369–379

    Article  Google Scholar 

  22. Lipson H, Siegelmann HT (2000) Clustering irregular shapes using high-order neurons. Neural Comput 12(10):2331–2353

    Article  Google Scholar 

  23. Liu R, Sun X, Jiao L (2010) Particle swarm optimization based clustering: a comparison of different cluster validity indices. In: Li K, Li X, Ma S, Irwin G (eds) Life system modeling and intelligent computing, communications in computer and information science, vol 98. Springer, Berlin, pp 66–72

    Google Scholar 

  24. Lowe D, Broomhead D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321–355

    MathSciNet  MATH  Google Scholar 

  25. Mabu S, Obayashi M, Kuremoto T (2015) Ensemble learning of rule-based evolutionary algorithm using multi-layer perceptron for supporting decisions in stock trading problems. Appl Soft Comput 36:357–367

    Article  Google Scholar 

  26. Minsky M (1969) Perceptrons. MIT Press, Cambridge, MA

    MATH  Google Scholar 

  27. Misra BB, Satapathy SC, Biswal B, Dash P, Panda G (2006) Pattern classification using polynomial neural network. In: 2006 IEEE conference on cybernetics and intelligent systems, pp 1–6

  28. Misra B, Satapathy S, Biswal B, Dash P, Panda G (2006) Pattern classification using polynomial neural network. In: 2006 IEEE conference on cybernetics and intelligent systems, pp 1–6. IEEE

  29. Natarajan SS, Casasent DP (1993) Piecewise quadratic optical neural network. In: San Diego’92, pp 142–147. International Society for Optics and Photonics

  30. Nejjari H, Benbouzid M (1999) Monitoring and diagnosis of induction motors electrical faults using a current park’s vector pattern learning approach. In: International conference IEMD ’99 electric machines and drives, 1999, pp 275–277

  31. Nitta T, Buchholz S (2008) On the decision boundaries of hyperbolic neurons. In: IEEE international joint conference on neural networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence), pp 2974–2980

  32. Oh SK, Pedrycz W (2002) The design of self-organizing polynomial neural networks. Inf Sci 141(34):237–258

    Article  MATH  Google Scholar 

  33. Park H, Ozeki T, Amari Si (2005) Geometric approach to multilayer perceptrons. In: Handbook of geometric computing, pp 69–96. Springer, Berlin

  34. Perwass C, Banarer V, Sommer G (2003) Spherical decision surfacesusing conformal modelling. In: Michaelis B, Krell G (eds) Pattern recognition. Lecture notes in computer science, vol 2781, pp 9–16. Springer, Berlin

  35. Pillay P, Xu Z (1996) Motor current signature analysis. In: Industry applications conference, 1996. Thirty-first IAS annual meeting, IAS ’96., Conference record of the 1996 IEEE, vol 1, pp 587–594

  36. Prechelt L et al (1994) Proben1: a set of neural network benchmarkproblems and benchmarking rules. Fakultät für Informatik, Univ. Karlsruhe, Karlsruhe, Germany, Tech Rep 21:94

  37. Rother C., Kolmogorov V, Blake A (2013) Image and video editing. http://research.microsoft.com/en-us/um/cambridge/projects/visionimagevideoediting/segmentation/grabcut.htm

  38. Salinas-Gutiérrez R, Hernández-Aguirre A, Rivera-Meraz MJ, Villa-Diharce ER (2010) Supervised probabilistic classification based on gaussian copulas. In: Sidorov G, Hernández Aguirre A, Reyes García CA (eds) Advances in soft computing. Lecture notes in computer science, vol 6438, pp 104–115. Springer, Berlin

  39. Serrano-Rubio J Color image segmentation. https://www.youtube.com/watch?v=sAVDPCed1qM

  40. Serrano-Rubio J Conic sections produced by hc-mlp. https://www.youtube.com/watch?v=A1Y2rJTSZbQ

  41. Serrano-Rubio J Linear surfaces produced by hp-mlp. https://www.youtube.com/watch?v=CfTL-AGmpzo

  42. Sexton RS, Dorsey RE (2000) Reliable classification using neural networks: a genetic algorithm and backpropagation comparison. Decis Support Syst 30(1):11–22

    Article  Google Scholar 

  43. Sossa, H., Garro, B., Villegas, J., Olague, G., Avils, C.: Evolutionary computation applied to the automatic design of artificial neural networks and associative memories. In: Schtze O, Coello Coello CA, Tantar AA, Tantar E, Bouvry P, Del Moral P, Legrand P (eds) Evolve—a bridge between probability, set oriented numerics, and evolutionary computation II. Advances in intelligent systems and computing, vol 175, pp 285–297. Springer, Berlin

  44. Tablada L, Valena M (2012) Self-organizing polynomial neuralnetworks based on matrix inversion and differential evolution. In: Yin H, Costa J, Barreto G (eds) Intelligent data engineering andautomated learning—IDEAL 2012. Lecture notes in computer science, vol 7435, pp 399–406. Springer, Berlin

  45. Taylor BJ (2005) Methods and procedures for the verification and validation of artificial neural networks. Springer, Secaucus, NJ

    Google Scholar 

  46. Thomson W, Fenger M (2001) Current signature analysis to detect induction motor faults. IEEE Ind Appl Mag 7(4):26–34

    Article  Google Scholar 

  47. Tufféry S (2011) Data mining and statistics for decision making. Wiley, New Yok

    Book  MATH  Google Scholar 

  48. UCI: Breast cancer wisconsin (diagnostic) data set—machine learning repository (1995). http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29

  49. UCI: Diabetes data set—machine learning repository. http://archive.ics.uci.edu/ml/datasets/Diabetes

  50. UCI: Heart disease data set—machine learning repository (1988). http://archive.ics.uci.edu/ml/datasets/Heart+Disease

  51. UCI: Lung cancer data set—machine learning repository (1992). https://archive.ics.uci.edu/ml/datasets/Lung+Cancer

  52. van den Bergh F, Engelbrecht A (2006) A study of particle swarm optimization particle trajectories. Inf Sci 176(8):937–971

    Article  MathSciNet  MATH  Google Scholar 

  53. Weber DM, Casasent DP (1998) The extended piecewise quadratic neural network. Neural Netw 11(5):837–850

    Article  Google Scholar 

  54. Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8(3):694–713

    Article  Google Scholar 

  55. Yu J, Xi L, Wang S (2007) An improved particle swarm optimization for evolving feedforward artificial neural networks. Neural Process Lett 26(3):217–231

    Article  Google Scholar 

  56. Zhang C, Shao H (2000) An ann’s evolved by a new evolutionary system and its application. In: Proceedings of the 39th IEEE conference on decision and control, 2000, vol 4, pp 3562–3563

Download references

Acknowledgments

The first author would like to thank to Antonio Zamarron for technical advice in induction motors. The third author would like to thank the International Centre for Theoretical Physics (ICTP) and the Institut Des Hautes Etudes Scientifiques (IHES) for its hospitality and support.

Author information

Authors and Affiliations

  1. Information Technologies Laboratory, Technological Institute of Irapuato (ITESI), Irapuato, Guanajuato, Mexico

    Juan Pablo Serrano-Rubio

  2. Computer Science Department, Center for Research in Mathematics (CIMAT), Guanajuato, Mexico

    Arturo Hernández-Aguirre

  3. Mathematics Department, Center for Research in Mathematics (CIMAT), Guanajuato, Mexico

    Rafael Herrera-Guzmán

Authors
  1. Juan Pablo Serrano-Rubio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arturo Hernández-Aguirre
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafael Herrera-Guzmán
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Pablo Serrano-Rubio.

Additional information

Juan Pablo Serrano-Rubio is partially supported by a PRODEP grant and Rafael Herrera-Guzmán is partially supported by a CONACYT grant.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Serrano-Rubio, J.P., Hernández-Aguirre, A. & Herrera-Guzmán, R. Hyperconic Multilayer Perceptron. Neural Process Lett 45, 29–58 (2017). https://doi.org/10.1007/s11063-016-9505-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-016-9505-2

Keywords

Search

Navigation