Skip to main content
SpringerLink
Log in

A highly scalable modular bottleneck neural network for image dimensionality reduction and image transformation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper presents a highly scalable modular bottleneck neural network and its application to image dimensionality reduction and image transformation. The network is a three-dimensional lattice of modules that implements a complex mapping with full connectivity between two high-dimensional datasets. These datasets correspond to input and output pixel-based images of three airplanes with various spatial orientations. The modules are multilayer perceptrons trained with Levenberg-Marquardt method on GPUs. They are locally connected together in an original manner that allows the gradual elaboration of the global mapping. The lattice of modules is squeezed in its middle into a bottleneck, thereby reducing the dimensionality of images. Afterward, the bottleneck itself is stretched to enforce a specific transformation directly on the reduced data. Analysis of the neural values at the bottleneck shows that we can extract from them robust and discriminative descriptors of the airplanes. The approach compares favorably to other dimensionality reduction techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37
Fig. 38
Fig. 39
Fig. 40
Fig. 41
Fig. 42
Fig. 43
Fig. 44
Fig. 45
Fig. 46
Fig. 47
Fig. 48
Fig. 49
Fig. 50
Fig. 51

Similar content being viewed by others

References

  1. Carcenac M (2008) A modular neural network applied to image transformation and mental images. Neural Comput Appl 17(5):549–568

    Article  Google Scholar 

  2. Osherson DN, Weinstein S, Stoli M (1990) Modular learning. In: Schwartz EL (ed) Computational neuroscience. MIT Press, Cambridge, MA, pp 369–377

    Google Scholar 

  3. Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE (1991) Adaptive mixtures of local experts. Neural Comput 3:79–87

    Article  Google Scholar 

  4. Jordan MI (1994) A statistical approach to decision tree modeling. Proceedings of the ACM Conference on Computer Learning Theory

  5. Wu CL, Chau KW, Fan C (2010) Prediction of rainfall time series using modular artificial neural networks coupled with data-preprocessing techniques. J Hydrol 389(1-2):146–167

    Article  Google Scholar 

  6. Carcenac M (2009) A modular neural network for super-resolution of human faces. Appl Intell 30(2):168–186

    Article  Google Scholar 

  7. Kramer MA (1991) Nonlinear principal component analysis using autoassociative neural networks. AIChE J 37(2):233–243

    Article  Google Scholar 

  8. CUDA Toolkit Documentation http://docs.nvidia.com/cuda, Nvidia Corporation

  9. Linsker R (1986) From basic network principles to neural architecture: emergence of spatial-opponent cells. P Natl Acad Sci USA 83:7508–7512

    Article  Google Scholar 

  10. Linsker R (1988) Self-organization in a perceptual network. Computer 21(3):105–117

    Article  Google Scholar 

  11. LeCun Y (1993) Efficient learning and second-order methods. A tutorial at NIPS 93, Denver

  12. Marquardt D (1963) An algorithm for least-squares estimation of nonlinear parameters. SIAM J Soc Ind Appl Math 11(2):431–441

    Article  MathSciNet  MATH  Google Scholar 

  13. Hinton GE (1989) Connectionist learning procedures. Artif Intell 40:185–234

    Article  Google Scholar 

  14. Hsieh WW, Tang B (1998) Applying neural network models to prediction and data analysis in meteorology and oceanography. B Am Meteorol Soc 79(9):1855–1870

    Article  Google Scholar 

  15. CUDA Toolkit Documentation http://docs.nvidia.com/cuda/cublas, Nvidia Corporation

  16. Carcenac M (2014) From tile algorithm to stripe algorithm: a CUBLAS-based parallel implementation on GPUs of Gauss method for the resolution of extremely large dense linear systems stored on an array of solid state devices. J Supercomput 68(1):365–413

    Article  Google Scholar 

  17. Buttari A, Langou J, Kurzak J, Dongarra J (2009) A class of parallel tiled linear algebra algorithms for multicore architectures. Parallel Comput 35:38–53

    Article  MathSciNet  Google Scholar 

  18. Agullo E, Augonnet C, Dongarra J, Faverge M, Langou J, Ltaief H, Tomov S (2011) LU factorization for accelerator-based systems AICCSA’ 11 Conference 217–224

  19. Hsieh WW (2001) Nonlinear principal component analysis by neural networks. Tellus 53A:599–615

    Article  Google Scholar 

  20. Monahan AH (2001) Nonlinear principal component analysis: tropical Indo-Pacific sea surface temperature and sea level pressure. J Climate 14:219–233

    Article  Google Scholar 

  21. Hsieh WW (2007) Nonlinear principal component analysis of noisy data. Neural Netw 20:434–443

    Article  MATH  Google Scholar 

  22. 3D models http://www.turbosquid.com, TurboSquid

  23. Monahan AD (2000) Nonlinear principal component analysis by neural networks: theory and application to the Lorentz system. J Climate 13:821–835

    Article  Google Scholar 

  24. Lorenz EN (1963) Deterministic nonperiodic flow. J Atmos Sci 20:130–141

    Article  Google Scholar 

  25. Kirby MJ, Miranda R (1996) Circular nodes in neural networks. Neural Comput 8:390–402

    Article  Google Scholar 

  26. Bengio Y, LeCun Y (2007) Scaling learning algorithms towards AI. In: Bottou L, Chapelle O, DeCoste D, Weston J (eds) Large Scale Kernel Machines. MIT, Press, Cambridge, MA, pp 321–360

    Google Scholar 

  27. Chen K (2015) Deep and Modular Neural Networks Kacprzyk J, Pedrycz W (eds)

  28. Bengio Y (2009) Learning Deep Architectures for AI. Found and Trends in Mach Learn 2(1):1–127

    Article  MathSciNet  MATH  Google Scholar 

  29. Hinton GE, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(9):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  30. Erhan D, Bengio Y, Courville A, Manzagol PA, Vincent P, Bengio S (2010) Why does unsupervised pre-training help deep learning J Mach Learn Res 11:625–660

    MathSciNet  MATH  Google Scholar 

  31. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS-10), 249–256

  32. Schmidhuber J (2014) Deep Learning in Neural Networks: An Overview. Technical Report IDSIA-03-14. arXiv:1404.7828v4

  33. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504– 507

    Article  MathSciNet  MATH  Google Scholar 

  34. Salakhutdinov RR, Hinton GE Training a deep autoencoder or a classifier on MNIST digits. http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html

  35. Kailath T (1980) Linear systems. Prentice-Hall, Englewood Cliffs, NJ

    MATH  Google Scholar 

  36. McWhirter JG, Baxter PD, Cooper T, Redif S, Foster J (2007) An EVD algorithm for para-Hermitian polynomial matrices. IEEE T Signal Proces 55(5):2158–2169

    Article  MathSciNet  Google Scholar 

  37. Redif S, McWhirter JG, Weiss S (2011) Design of FIR paraunitary filter banks for subband coding using a polynomial eigenvalue decomposition. IEEE T Signal Proces 59(11):5253–5264

    Article  MathSciNet  Google Scholar 

  38. Redif S, McWhirter JG, Baxter P, Cooper T (2006) Robust broadband adaptive beamforming via polynomial eigenvalues Proceedings of the IEEE Oceans Conference, 1–6

  39. Weiss S, Redif S, Cooper T, Liu C, Baxter PD, McWhirter JG (2006) Paraunitary oversampled filter bank design for channel coding. EURASIP J Appl Si Pr 2006:1–10

    Article  MATH  Google Scholar 

  40. Ta CH, Weiss S (2007) A design of precoding and equalisation for broadband MIMO systems. Proceedings of the 15th International Conference on Digital Signal Processing, Cardiff, UK, 571–574

  41. Alrmah MA, Weiss S, Redif S, Lambotharan S, McWhirter JG (2013) Angle of arrival estimation for broadband signals: A comparison. Proceedings of the Intelligent Signal Processing Conference, IET, London, UK

  42. Redif S, Weiss S, McWhirter JG (2015) Sequential matrix diagonalisation algorithms for polynomial EVD of parahermitian matrices. IEEE T Signal Proces 63(1):81–89

    Article  MathSciNet  Google Scholar 

  43. Corr J, Thomson K, Weiss S, McWhirter JG, Redif S, Proudler IK (2014) Multiple shift maximum element sequential matrix diagonalisation for parahermitian matrices. Proceedings of the IEEE Workshop on Statistical Signal Processing, Gold Coast, Australia, 312–315

  44. Krauth W, Mézard M, Nadal JP (1988) Basins of attraction in a perceptron-like neural network. Complex Syst 2:387–408

    MathSciNet  MATH  Google Scholar 

  45. Uppala SM, Kallberg PW, Simmons AJ, Andrae U, Da Costa Bechtold V, Fiorino M, Gibson JK, Haseler J, Hernandez A, Kelly GA, Li X, Onogi K, Saarinen S, Sokka N, Allan RP, Andersson E, Arpe K, Balmaseda MA, Beljaars ACM, Van De Berg L, Bidlot J, Bormann N, Caires S, Chevallier F, Dethof A, Dragosavac M, Fisher M, Fuentes M, Hagemann S, Holm E, Hoskins BJ, Isaksen L, Janssen PAEM, Jenne R, Mcnally AP, Mahfouf JF, Morcrette JJ, Rayner NA, Saunders RW, Simon P, Sterl A, Trenberth KE, Untch A, Vasiljevic D, Viterbo P, Woollen J (2005) The ERA-40 re-analysis. Q J Roy Meteor Soc 131(612):2961–3012

    Article  Google Scholar 

Download references

Acknowledgments

The authors are thankful to Nvidia Corporation for their donation of the C2075 and the GTX480 GPUs used for the experiments reported in this paper. We wish to thank the reviewers of this paper for their insightful comments and suggestions. We also thank Prof. Ian K. Proudler, of Loughborough University, UK, and Prof. Isaac Lerner for reviewing the manuscript and for offering suggestions toward its improvement.

Author information

Authors and Affiliations

  1. Faculty of Engineering, European University of Lefke, Gemikonaǧı, via Mersin 10, Turkey

    Manuel Carcenac & Soydan Redif

Authors
  1. Manuel Carcenac
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soydan Redif
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuel Carcenac.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Carcenac, M., Redif, S. A highly scalable modular bottleneck neural network for image dimensionality reduction and image transformation. Appl Intell 44, 557–610 (2016). https://doi.org/10.1007/s10489-015-0715-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-015-0715-5

Keywords

Search

Navigation