Skip to main content
SpringerLink
Log in

A deep stacked wavelet auto-encoders to supervised feature extraction to pattern classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The major issue in pattern classification is in the extraction of features in the training phase. The focus of this work is on combining the ability of wavelet networks and the deep learning techniques to propose a new supervised feature extraction method to pattern classification. This new approach allows the classification of all classes of the dataset by the reconstruction of a Deep Stacked wavelet Auto-Encoder. This Network is obtained after a series of wavelet Auto-Encoders followed by a Softmax classifier at the last layer. Finally, a fine-tuning is applied for the improvement of our result using a back propagation algorithm. Our approach is tested with different image datasets which are the COIL-100, the APTI and the ImageNet datasets and is also tested with two other audio corpuses that contain Arabic words and French words. The experimental test demonstrates the efficiency of our network for image and audio classification compared to other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Amin A, Al-Sadoun H, Fischer S (1996) Hand-printed arabic character recognition system using an artificial network. Pattern Recogn 29(4):663–675. doi:10.1016/0031-3203(95)00110-7. URL http://www.sciencedirect.com/science/article/pii/0031320395001107

    Article  Google Scholar 

  2. Ben Amar C, Zaied M, Alim A (2005) Beta wavelets. Synthesis and application tolossy image compression In: Adv Eng Softw 36:459–474. doi:10.1016/j.advengsoft.2005.01.013

    MATH  Google Scholar 

  3. Bengio Y (2009), "Learning Deep Architectures for AI", Foundations and Trends® in Machine Learning: 2(1), 1–127. doi:10.1561/2200000006

  4. Chen Z, Wang J, He H, Huang X (2014) A Fast Deep Learning System Using GPU (1), 1552–1555

  5. Dammak M, Mejdoub M, Zaied M, C. Ben Amar (2012) Feature vector approximation based on wavelet network, in: Proceedings of the 4th International Conference on Agents and Artificial Intelligence (ICAART), 394–399.

  6. Daubechies I (1992) Ten lectures on wavelets, number 61 in cbms-nsf series in applied mathematics

  7. Deng L, Yu D (2014) DEEP LEARNING: methods and applications. Found Trends SignalProcess 7(3–4):1–134

    Google Scholar 

  8. Ejbali R, Benayed Y, Zaied M, Alimi MA (2009) Wavelet networks for phonemes recognition, International conference on systems and information processing

  9. Ejbali R, Zaied M, Amar CB (2010) Intelligent approach to train wavelet networks forRecognition system of Arabic words. KDIR, In, pp 518–522

    Google Scholar 

  10. Ejbali R, Zaied M, C. Ben Amar (2012) Multi-input multi-output beta wavelet network: modeling of acoustic units for speech recognition Int. J. Adv. Comput. Sci. Appl. IJACSA12, Sci. Inf. Organ. (SAI), 3

  11. El Adel A, Zaied M, Amar CB (2011) Learning wavelet networks based on multiresolution analysis: application to images copy detection. International Conference on Communications, Computing and Control Applications, pp 1–6

    Google Scholar 

  12. Eladel A, Ejbali R, Zaied M, C. Ben Amar (2014a) A New Semantic Approach for CBIRBased on BetaWavelet Network Modeling Shape Refined by Texture and Color Features pp. 378–385

  13. Eladel A, Zaied M, C. Ben Amar (2014b) A new system for image retrieval using beta wavelet network for descriptors extraction and fuzzy decision. 232–236

  14. Guedri B, Zaied M, Amar CB (2011) Indexing and images retrieval by content, International Conference on High Performance Computing & Simulation, pp: 369–375

  15. Hassairi S, Ejbali R, Zaied M (2015a) A Deep Convolutional Neural Wavelet Network to supervised Arabic letter image classification. In: 15th Int. Conf. Intell. Syst. Des. Appl.

  16. Hassairi S, Ejbali R, Zaied M (2015b) Supervised Image Classification Using DeepConvolutional Wavelets Network. In: 2015 I.E. 27th Int. Conf. Tools withArtif. Intell., 265–271. IEEE. doi:10.1109/ICTAI.2015.49

  17. He K, Zhang X, Renand S, Sun J (2015) "Deep residual learning for image recognition." arXiv preprint arXiv:1512.03385

  18. Hinton GE, Osindero S, Teh YW (2006) A fastlearning algorithm for deep belief nets. Neural Comput 18(7):1527–1554. doi:10.1162/neco.2006.18.7.1527

    Article  MathSciNet  MATH  Google Scholar 

  19. Jemai O, Zaied M, C. Ben Amar and Alimi MA (2010) FBWN: an architecture of fast betawavelet networks for image classification. In: Proc Int Jt Conf Neural Networks. doi:10.1109/IJCNN.2010.5596876

  20. Jemai O, Zaied M, Ben Amar C, Alimi MA (2011a) Fast learning algorithm of WaveletNetwork based on fast wavelet transform. Int J Pattern Recognit Artif Intell 25(08):1297–1319. doi:10.1142/S0218001411009111

    Article  MathSciNet  MATH  Google Scholar 

  21. O. Jemai, M. Zaied, C. Ben Amar and A. Alimi (2011b): Pyramidal hybrid approach: wavelet network with OLS algorithm-based image classification, Int. J. Wavelets, Multiresolution Inf. Process 9 111–130.

  22. Jemai O, Ejbali R, Zaied M, Amar CB (2015) A speech recognition system based on hybrid wavelet network including a fuzzy decision support system, International Conference on Machine Vision (ICMV 2014), 944503–944503-7

  23. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst pp:1–9

  24. Le QV (2013) Building high-level features using large scale unsupervised learning. 2013 I.E. Int. Conf. Acoust. Speech Signal Process. pp. 8595–8598. doi:10.1109/ICASSP.2013.6639343. URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6639343

  25. LeCun Y (2012) Learning invariant feature hierarchies. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 7583 LNCS, 496–505

  26. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  27. Y. Lin, F. Lv, S. Zhu, M. Yang, T. Cour, K. Yu, L. Cao and T. Huang (2011) Large-scaleimage classification: Fast feature extraction and svm training. In: Computer Visionand Pattern Recognition (CVPR), 2011 I.E. Conference on, 1689–696. doi:10.1109/CVPR.2011.5995477

  28. Liou CY, Cheng WC, Liou JW, Liou DR (2014) Autoencoder for words. Neurocomputing 139:84–96. doi:10.1016/j.neucom.2013.09.055

    Article  Google Scholar 

  29. Mejdoub M, Ben Amar C (2013) Classification improvement of local feature vectors over the KNN algorithm Multimed. Tools Appl 64(1):197–218

    Article  Google Scholar 

  30. Mejdoub M, Fonteles L, C. Ben Amar, Antonini M (2008) Fast indexing method for image retrieval using tree-structured lattices, in: International Workshop on Content-Based Multimedia Indexing (CBMI), 2008, Conference Proceedings, art. no. 4564970, pp. 365–372.

  31. Nene S, Nayar S, Murase H (1996) Columbia Object Image Library (COIL-100). Tech.rep. (1996). URL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.5914

  32. X. Peng, R. Yan, B. Zhao, H. Tang and Z. Yi (2015) Fast low rank representation based spatialpyramid matching for image classification. Knowledge-Based Systems 90, 14–22

  33. Picone J (1993) Signal modeling techniques in speech recognition. Proc.IEEE 81(9), 1215–1247. doi:10.1109/5.237532. URLhttp://ieeexplore.ieee.org/articleDetails.jsp?arnumber=237532

  34. Rastegari M, Ordonez V, Redmon J, Farhadi A (2016) XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. arXiv preprint arXiv:1603.05279

  35. Said S, B. Ben Amor, Zaied M, C. Ben Amar, Daoudi M (2009) Fast and efficient 3D face recognition using wavelet networks, in: The 16th IEEE International Conference on Image Processing, Cairo, Egypt, November 4153–4156

  36. Sermanet P, Chintala S, LeCun Y (2012) Convolutional neural networks applied to housenumbers digit classification. Proc. Int. Conf. Pattern Recognit. ICPR12 pp. 10–13

  37. Shin HC, Orton MR, Collins DJ, Doran SJ, Leach MO (2013) Stacked autoencodersfor unsupervised feature learning and multiple organ detection in a pilot study using4D patient data. IEEE Trans. Pattern Anal. Mach. Intell 35(8):1930–1943. doi:10.1109/TPAMI.2012.277

    Article  Google Scholar 

  38. Slimane F, Kanoun S, Hennebert J, Alimi AM, Ingold R (2013) A study on font-family and font-size recognition applied to Arabic word images at ultra-low resolution. Pattern Recognit. Lett 34(2):209–218. doi:10.1016/j.patrec.2012.09.012

    Article  Google Scholar 

  39. Smagghe P, Buessler JL, Urban JP (2013) Novelty detection in image recognition usingirf neural networks properties. ESANN, In

    Google Scholar 

  40. Tsai Cw, Huang Wc, Chiang Mc (2014) Mobile, Ubiquitous, and Intelligent Computing. Lect Notes Electr Eng 274:629–636. doi:10.1007/978-3-642-40675-1

    Article  Google Scholar 

  41. P. Vincent: Stacked DenoisingAutoencoders: learning useful representations in aDeep network with a local denoising criterion 11, 3371–3408 (2010)

    Google Scholar 

  42. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked DenoisingAutoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408

    MathSciNet  MATH  Google Scholar 

  43. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: Fromerror visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. doi:10.1109/TIP.2003.819861

    Article  Google Scholar 

  44. Witkowski B (2015) Autoencoders for Image Classification.Jagiellonian University Faculty of Mathematics and Computer Science INSTITUTE OF COMPUTER SCIENCEApr 28

  45. Zaied M, C. Ben Amar, Alimi MA (2003) Award a new wavelet based beta function, International conference on signal, system and design, 185–191

  46. M. Zaied, S. Said, O. Jemai and C. Ben Amar (2011) a Novel Approach for Face RecognitionBased on Fast Learning Algorithm and Wavelet Network Theory. Int. J. Wavelets,Multiresolution Inf. Process 09(06), 923–945. doi:10.1142/S0219691311004389

  47. Zaied M, Mohamed R, C. ben Amar (2012) A power tool for content-based image retrieval using multiresolution wavelet network modeling and dynamic histograms. Int. REv.Comput. Softw. (IRECOS), 7

  48. Zeiler MD, Fergus R (2013) Visualizing and Understanding Convolutional Networks. arXivPrepr. arXiv1311.2901 (2013). URL http://arxiv.org/abs/1311.2901

  49. Zhang Q, Benveniste A (1992) Wavelet networks. IEEE Trans Neural Networks 3(6):889–898. doi:10.1109/72.165591

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUBprogram.

Author information

Authors and Affiliations

  1. RTIM: Research Team in Intelligent Machines, Gabes, Tunisia

    Salima Hassairi, Ridha Ejbali & Mourad Zaied

  2. National Engineering School of Sfax (ENIS), 1173, 3038, Sfax, BP, Tunisia

    Salima Hassairi, Ridha Ejbali & Mourad Zaied

Authors
  1. Salima Hassairi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ridha Ejbali
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mourad Zaied
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salima Hassairi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hassairi, S., Ejbali, R. & Zaied, M. A deep stacked wavelet auto-encoders to supervised feature extraction to pattern classification. Multimed Tools Appl 77, 5443–5459 (2018). https://doi.org/10.1007/s11042-017-4461-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4461-z

Keywords

Search

Navigation