Skip to main content
SpringerLink
Log in

Object oriented design of a simulator for large BP Neural Networks

  • Neurosimulators
  • Conference paper
  • First Online:
From Natural to Artificial Neural Computation (IWANN 1995)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 930))

Included in the following conference series:

Abstract

In this paper we describe the implementation of the backpropagation algorithm by means of an object oriented library (ARCH). The use of this library relieves the user from the details of a specific parallel programming machines and at the same time allows a greater portability of the generated code.

To provide a comparison with existing solutions, we survey the most relevant implementations of the algorithm proposed so far in the literature, both on dedicated and general purpose computers.

Extensive experimental results show that the use of the library does not hurt the performance of our simulator, on the contrary our implementation on a Connection Machine (CM-5) is comparable with the fastest in its category.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. SNAP — SIMD Numerical Array Processor. HNC, 5930 Cornerstone Court West, S.Diego, CA, 1994.

    Google Scholar 

  2. J.M.Adamo. Object-Oriented Parallel Programming: Design and Development of an Object-Oriented Library for SPMD Programming. ICSI Technical Report TR-94-011, February 1994.

    Google Scholar 

  3. J.M.Adamo. Development of Parallel BLAS with ARCH Object-Oriented Parallel Library, Implementation on CM-5. ICSI Technical Report TR-94-045, August 1994.

    Google Scholar 

  4. D.Anguita, G.Parodi, and R.Zunino. An Efficient Implementation of BP on RISC-based Workstations. Neurocomputing 6, 1994, pp. 57–65.

    Google Scholar 

  5. K.Asanović, J.Beck, T.Callahan, J.Feldman, B.Irissou, B.Kingsbury, P.Kohn, J.Lazzaro, N.Morgan, D.Stoutamire and J.Wawrzynek. CNS-1 Architecture Specification. ICSI Technical Report TR-93-021, April 1993.

    Google Scholar 

  6. K.Asanović, J.Beck, B.E.D.Kingsbury, P.Kohn, N.Morgan, J.Wawrzynek. SPERT: A VLIW/SIMD Microprocessor for Artificial Neural Network Computations. ICSI Tech. Rep. TR-91-072, January 1992.

    Google Scholar 

  7. A.Corana, C.Rolando, S.Ridella. A Highly Efficient Implementation of Back-propagation Algorithm on SIMD Computers. In High Performance Computing, J.-L.Delhaye and E.Gelenbe (Eds.), Elsevier, 1989, pp.181–190.

    Google Scholar 

  8. A.Corana, C.Rolando, S.Ridella. Use of Level 3 BLAS Kernels in Neural Networks: The Back-propagation algorithm. Parallel Computing 89, 1990, pp.269–274.

    Google Scholar 

  9. J.Dongarra. Linear Algebra Library for High-Performance Computers. Frontiers of Supercomputing II. K.R.Ames and A.Brenner (Eds.), University of California Press, 1994.

    Google Scholar 

  10. B.Faure, G.Mazare. Implementation of back-propagation on a VLSI asynchronous cellular architecture. Proc. of Int. NN Conf., July 9–13, 1990, Paris, France, pp. 631–634.

    Google Scholar 

  11. K.A.Grajski, G.Chinn, C.Chen, C.Kuszmaul, S.Tomboulian. Neural Network Simulation on the MasPar MP-1 Massively Parallel Processor. Proc. of Int. NN Conf., July 9–13, 1990, Paris, France, p.673.

    Google Scholar 

  12. A.Hiraiwa, S.Kurosu, S.Arisawa, M.Inoue. A two level pipeline RISC processor array for ANN. Proc. of the Int. Joint Conf. on NN, January 15–19, 1990, Washinghton, DC, pp.II137–II140.

    Google Scholar 

  13. P.Ienne. Architectures for Neuro-Computers: Review and Performance Evaluation. Technical Report 93/21, Swiss Federal Institute of Technology, Lausanne, January 1993.

    Google Scholar 

  14. D.Jackson, D.Hammerstrom. Distributing Back Propagation Networks Over the Intel iPSC/860 Hypercube. Proc. of the Int. Joint Conf. on NN, July 8–12, 1991, Seattle, WA, pp. 1569–1574.

    Google Scholar 

  15. H.Kato, H.Yoshizawa, H.Iciki, K.Asakawa. A Parallel Neurocomputer Architecture towards Billion Connection Update Per Second. Proc. of the Int. Joint Conf. on NN, January 15–19, 1990, Washinghton, DC, pp.II47–II50.

    Google Scholar 

  16. X.Liu and G.L.Wilcox. Benchmarking of the CM-5 and the Cray Machines with a Very Large Backpropagation Neural Network. Proc. of IEEE Int. Conf. on NN, June 28–July 2, 1994, Orlando, FL, pp.22–27.

    Google Scholar 

  17. R.Means, L.Lisenbee. Extensible Linear Floating Point SIMD Neurocomputer Array Processor. Proc. of the Int. Joint Conf. on NN, July 8–12, 1991, Seattle, WA, pp. 1587–1592.

    Google Scholar 

  18. M.Moller. Supervised Learning on Large Redundant Training Sets. Int. J. of Neural Systems, Vol.4, No.1, 1993.

    Google Scholar 

  19. N.Morgan, J.Beck, P.Kohn, J.Bilmes, E.Allman, J.Beer. The Ring Array Processor: A Multiprocessing Peripheral for Connectionist Applications. Journal of Parallel and Distributed Computing, Vol.14, N.3, March 1992, pp.248–259.

    Google Scholar 

  20. U.A.Müller. A High Performance Neural Net Simulation Environment. Proc. of IEEE Int. Conf. on NN, June 28–July 2, 1994, Orlando, FL, pp.1–4.

    Google Scholar 

  21. U.A.Müller, M.Kocheisen, A.Gunzinger. High-Performance Neural Net Simulation on a Multiprocessor System with “Intelligent” Communication. Advances in Neural Information Processing Systems 6, J.D.Cowan, G.Tesauro, J.Alspector (Eds.), Morgan Kaufmann Publ., 1994, pp.888–895.

    Google Scholar 

  22. S.M.Müller. A Performance Analysis of the CNS-1 on Large, Dense Backpropagation Networks. ICSI Technical Report TR-93-046, September 1993.

    Google Scholar 

  23. A.Petrowsky, G.Dreyfus, Performance Analysis of a Pipelined Backpropagation Parallel Algorithm. IEEE Trans. on Neural Networks, Vol.4, No.6, Nov. 1993, pp.970–981.

    Google Scholar 

  24. U. Ramacher. SYNAPSE — A Neurocomputer That Synthesizes Neural Algorithms on a Parallel Systolic Engine. Journal of Parallel and Distributed Computing, Vol.14, N.3, March 1992, pp.306–318.

    Article  Google Scholar 

  25. E.Sànchez, S.Barro, C.V.Regueiro. Artificial Neural Networks Implementation on Vectorial Supercomputers. Proc. of IEEE Int. Conf. on NN, June 28–July 2, 1994, Orlando, FL, pp. 3938–3943.

    Google Scholar 

  26. T.J.Sejnowsky and C.R.Rosenberg. Parallel Networks that Learn to Pronounce English Text. Complex Systems 1, 1987.

    Google Scholar 

  27. A.Singer. Exploiting the Inherent Parallelism of Artificial Neural Networks to Achieve 1300 Million Interconnets per Second. Proc. of Int. NN Conf., July 9–13, 1990, Paris, France, pp.656–660.

    Google Scholar 

  28. M.A.Viredaz. MANTRA I: An SIMD Processor Array for Neural Computation. Proc. of the Euro-ARCH '93 Conf., München, October 1993.

    Google Scholar 

  29. J.Wawrzynek, K.Asanović, and N.Morgan. The Design of a Neuro-Microprocessor. IEEE Trans. on NN, Vol.4, No.3, May 1993.

    Google Scholar 

  30. X.Zhang, M.Mckenna, J.P.Mesirov, D.L.Waltz. An Efficient Implementation of the Back-Propagation Algorithm on the Connection Machine CM-2. Advances in Neural Information Processing Systems 2, D.S.Touretzky (Ed.), Morgan Kaufmann Publ., 1990, pp.801–809.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Université Claude Bernard & LISA CPE-Lyon, 25 rue du Plat, 69288, Lyon cedex 02, France

    J. M. Adamo

  2. D.I.B.E., University of Genova, Via Opera Pia 11a, 16145, Genova, Italy

    D. Anguita

Authors
  1. J. M. Adamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. Anguita
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

José Mira Francisco Sandoval

Rights and permissions

Reprints and permissions

Copyright information

© 1995 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Adamo, J.M., Anguita, D. (1995). Object oriented design of a simulator for large BP Neural Networks. In: Mira, J., Sandoval, F. (eds) From Natural to Artificial Neural Computation. IWANN 1995. Lecture Notes in Computer Science, vol 930. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-59497-3_233

Download citation

  • DOI: https://doi.org/10.1007/3-540-59497-3_233

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-59497-0

  • Online ISBN: 978-3-540-49288-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation