Skip to main content
SpringerLink
Log in

Manifold Preserving CNN for Pixel-Based Object Labelling in Images for High Dimensional Feature spaces

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Deep CNN’s have achieved an excellent performance in computer vision and image processing methods, designating them as a state-of-art in this domain. CNN based applications have achieved tremendous advancement towards vision computing with high dimensional object labelling in images. The complex nature of High Dimensional (HD) images limits the performance of CNN’s. In high dimensional feature space, the pixel-based image labelling is a complex problem for the parsing of objects in an image. To overcome this issue, we have studied a two-stage end-to-end framework that uses manifold embedding based patch-wise CNN architecture to extract the features and classify the image for labelled classes. We have investigated the deep-features with an information fusion technique for low dimensional feature space compression by using pre-trained CNNs and spatiality preserving manifold embedding in the first stage. The cost of pixel-based labelling in HD feature space is very high, so researchers have tried to encapsulate maximum information within the minimum image size. Therefore, in this stage, we have first increased the valuable information by concatenating the deep spatial features and then embedding the massive dataset by using manifold preservation. In stage-2, the image patches are extracted and passed into three layers of convolution-pooling pair and two layers of fully connected pair using parameter tuning. The training dataset is prepared in the form of pixel-label pairs. Subsequently, the proposed method has been evaluated on publicly available images and compared with the previously proposed schemes. The proposed method has outperformed the previous techniques in accuracy and computation time with a significant margin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray DG, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X (2016) Tensorflow: A system for large-scale machine learning. In: Proceedings of the 12th USENIX conference on operating systems design and implementation, USENIX association, Berkeley, CA, USA, OSDI’16, pp 265–283, http://dl.acm.org/citation.cfm?id=3026877.3026899

  2. Li L, Ge H, Gao J, Zhang Y, Tong Y, Sun J (2019) A novel geometric mean feature space discriminant analysis method for hyperspectral image feature extraction. Neural Process Lett 51(1):515–542. https://doi.org/10.1007/s11063-019-10101-0

    Article  Google Scholar 

  3. Bhandari AK, Kumar A, Singh GK, Soni V (2016) Performance study of evolutionary algorithm for different wavelet filters for satellite image denoising using sub-band adaptive threshold. J Exp Theor Artif Intell 28(1–2):71–95. https://doi.org/10.1080/0952813X.2015.1020518

    Article  Google Scholar 

  4. Bian X, Zhang T, Zhang X, Yan L, Li B (2013) Clustering-based extraction of near border data samples for remote sensing image classification. Cogn Comput 5(1):19–31. https://doi.org/10.1007/s12559-012-9147-2

    Article  Google Scholar 

  5. Cai D, He X, Han J (2007) Spectral regression: a unified subspace learning framework for content-based image retrieval. In: ACM multimedia

  6. Cai Z, Shao L (2018) Rgb-d scene classification via multi-modal feature learning. Cogn Comput. https://doi.org/10.1007/s12559-018-9580-y

    Article  Google Scholar 

  7. Cao X, Zhou F, Xu F, Meng D, Xu Z, Paisley J (2018) Hyperspectral image classification with Markov random fields and a convolutional neural network. IEEE Trans Image Process 27(5):2354–2367. https://doi.org/10.1109/TIP.2018.2799324

    Article  MathSciNet  Google Scholar 

  8. Cheng G, Han J, Lu X (2017) Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE 105(10):1865–1883. https://doi.org/10.1109/JPROC.2017.2675998

    Article  Google Scholar 

  9. Cheng G, Li Z, Han J, Yao X, Guo L (2018) Exploring hierarchical convolutional features for hyperspectral image classification. IEEE Trans Geosci Remote Sens 56(11):6712–6722. https://doi.org/10.1109/TGRS.2018.2841823

    Article  Google Scholar 

  10. Ergul U, Bilgin G (2019) Hckboost: Hybridized composite kernel boosting with extreme learning machines for hyperspectral image classification. Neurocomputing 334:100–113. https://doi.org/10.1016/j.neucom.2019.01.010

    Article  Google Scholar 

  11. Han D (2013/03) Comparison of commonly used image interpolation methods. In: Proceedings of the 2nd international conference on computer science and electronics engineering. Atlantis Press, https://doi.org/10.2991/iccsee.2013.391

  12. Hu W, Huang Y, Wei L, Zhang F, Li H (2015) Deep convolutional neural networks for hyperspectral image classification. J Sens 258619:12. https://doi.org/10.1155/9161

    Article  Google Scholar 

  13. Imani M (2018) Anomaly detection using morphology-based collaborative representation in hyperspectral imagery. Eur J Remote Sens 51(1):457–471. https://doi.org/10.1080/22797254.2018.1446727

    Article  Google Scholar 

  14. Knöbelreiter P, Reinbacher C, Shekhovtsov A, Pock T (2017) End-to-end training of hybrid CNN-CRF models for stereo. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1456–1465. https://doi.org/10.1109/CVPR.2017.159

  15. Le THN, Duong CN, Han L, Luu K, Quach KG, Savvides M (2018) Deep contextual recurrent residual networks for scene labeling. Pattern Recognit 80:32–41. https://doi.org/10.1016/j.patcog.2018.01.005

    Article  Google Scholar 

  16. Li D, Tian Y (2018) Survey and experimental study on metric learning methods. Neural Netw 105:447–462. https://doi.org/10.1016/j.neunet.2018.06.003

    Article  Google Scholar 

  17. Li T, Leng J, Kong L, Guo S, Bai G, Wang K (2019) Dcnr: deep cube cnn with random forest for hyperspectral image classification. Multimed Tools Appl 78(3):3411–3433. https://doi.org/10.1007/s11042-018-5986-5

    Article  Google Scholar 

  18. Li W, Du Q, Zhang B (2015) Combined sparse and collaborative representation for hyperspectral target detection. Pattern Recognit 48(12):3904–3916. https://doi.org/10.1016/j.patcog.2015.05.024

    Article  Google Scholar 

  19. Li W, Wu G, Zhang F, Du Q (2017) Hyperspectral image classification using deep pixel-pair features. IEEE Trans Geosci Remote Sens 55(2):844–853. https://doi.org/10.1109/TGRS.2016.2616355

    Article  Google Scholar 

  20. Li W, Ding W, Sadasivam R, Cui X, Chen P (2019b) His-gan: a histogram-based gan model to improve data generation quality. Neural Netw 119:31–45. https://doi.org/10.1016/j.neunet.2019.07.001

    Article  Google Scholar 

  21. Li Y, Lu BL (2009) Feature selection based on loss-margin of nearest neighbour classification. Pattern Recognit 42(9):1914–1921. https://doi.org/10.1016/j.patcog.2008.10.011

    Article  MATH  Google Scholar 

  22. Gao H, Lin S, Li C, Yang Y (2018) Application of hyperspectral image classification based on overlap pooling. Neural Process Lett 49(3):1335–1354. https://doi.org/10.1007/s11063-018-9876-7

    Article  Google Scholar 

  23. Ma J, Yuan Y (2019) Dimension reduction of image deep feature using pca. J Vis Commun Image Represent 63:102578. https://doi.org/10.1016/j.jvcir.2019.102578

    Article  Google Scholar 

  24. Ma X, Liu W, Tao D, Zhou Y (2019) Ensemble p-Laplacian regularization for scene image recognition. Cogn Comput. https://doi.org/10.1007/s12559-019-09637-z

    Article  Google Scholar 

  25. Marinoni A, Gamba P (2017) Unsupervised data driven feature extraction by means of mutual information maximization. IEEE Trans Comput Imaging 3(2):243–253. https://doi.org/10.1109/TCI.2017.2669731

    Article  MathSciNet  Google Scholar 

  26. Menassel R, Nini B, Mekhaznia T (2018) An improved fractal image compression using wolf pack algorithm. J Exp Theor Artif Intell 30(3):429–439. https://doi.org/10.1080/0952813X.2017.1409281

    Article  Google Scholar 

  27. Li L, Ge H, Gao J, Zhang Y (2018) Hyperspectral image feature extraction using maclaurin series function curve fitting. Neural Process Lett 49(1):357–374. https://doi.org/10.1007/s11063-018-9825-5

    Article  Google Scholar 

  28. Nasrabadi NM (2014) Hyperspectral target detection: an overview of current and future challenges. IEEE Signal Process Mag 31(1):34–44. https://doi.org/10.1109/MSP.2013.2278992

    Article  Google Scholar 

  29. Cahill ND, Chew SE, Wenger PS (2015) Spatial-spectral dimensionality reduction of hyperspectral imagery with partial knowledge of class labels. https://doi.org/10.1117/12.2177139

  30. Pan E, Mei X, Wang Q, Ma Y, Ma J (2020) Spectral-spatial classification for hyperspectral image based on a single gru. Neurocomputing. https://doi.org/10.1016/j.neucom.2020.01.029

    Article  Google Scholar 

  31. Peng J, Jiang X, Chen N, Fu H (2019) Local adaptive joint sparse representation for hyperspectral image classification. Neurocomputing 334:239–248. https://doi.org/10.1016/j.neucom.2019.01.034

    Article  Google Scholar 

  32. Priego B, Duro RJ, Chanussot J (2017) 4dcaf: a temporal approach for denoising hyperspectral image sequences. Pattern Recognit 72:433–445. https://doi.org/10.1016/j.patcog.2017.07.023

    Article  Google Scholar 

  33. Rahimi SA, Sajedi H (2019) Monitoring air pollution by deep features and extreme learning machine. J Exp Theor Artif Intell 31(4):517–531. https://doi.org/10.1080/0952813X.2019.1572658

    Article  Google Scholar 

  34. Sharma A, Liu X, Yang X (2018) Land cover classification from multi-temporal, multi-spectral remotely sensed imagery using patch-based recurrent neural networks. Neural Netw 105:346–355. https://doi.org/10.1016/j.neunet.2018.05.019

    Article  Google Scholar 

  35. Srivastava V, Biswas B (2019) Cnn-based salient features in hsi image semantic target prediction. Connect Sci. https://doi.org/10.1080/09540091.2019.1650330

    Article  Google Scholar 

  36. Srivastava V, Biswas B (2019) Deep cnn feature fusion with manifold learning and regression for pixel classification in hsi images. J Exp Theor Artif Intell. https://doi.org/10.1080/0952813X.2019.1647566

    Article  Google Scholar 

  37. Srivastava V, Biswas B (2019) A subspace regression and two phase label optimization for high dimensional image classification. Multimed Tools Appl. https://doi.org/10.1007/s11042-019-08477-1

    Article  Google Scholar 

  38. Ho Tin Kam (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832–844. https://doi.org/10.1109/34.709601

    Article  Google Scholar 

  39. Trentin E, Cattoni R (1999) Learning perception for indoor robot navigation with a hybrid hidden Markov model/recurrent neural networks approach. Connect Sci 11(3–4):243–265. https://doi.org/10.1080/095400999116241

    Article  Google Scholar 

  40. Tu B, Li N, Fang L, Fei H, He D (2018) Classification of hyperspectral images via weighted spatial correlation representation. J Vis Commun Image Represent 56:160–166. https://doi.org/10.1016/j.jvcir.2018.09.010

    Article  Google Scholar 

  41. Xie W, Li Y, Hu J, Chen DY (2018) Trainable spectral difference learning with spatial starting for hyperspectral image denoising. Neural Netw 108:272–286. https://doi.org/10.1016/j.neunet.2018.08.021

    Article  Google Scholar 

  42. Yao Y, Guo P, Xin X, Jiang Z (2014) Image fusion by hierarchical joint sparse representation. Cogn Comput 6(3):281–292. https://doi.org/10.1007/s12559-013-9235-y

    Article  Google Scholar 

  43. Zhang A, Liu S, Sun G, Huang H, Ma P, Rong J, Ma H, Lin C, Wang Z (2018) Clustering of remote sensing imagery using a social recognition-based multi-objective gravitational search algorithm. Cogn Comput. https://doi.org/10.1007/s12559-018-9582-9

    Article  Google Scholar 

  44. Zhang L, Barnden J (2012) Affect sensing using linguistic, semantic and cognitive cues in multi-threaded improvisational dialogue. Cogn Comput 4(4):436–459. https://doi.org/10.1007/s12559-012-9170-3

    Article  Google Scholar 

  45. Zhang P, He H, Gao L (2019) A nonlinear and explicit framework of supervised manifold-feature extraction for hyperspectral image classification. Neurocomputing 337:315–324. https://doi.org/10.1016/j.neucom.2019.01.077

    Article  Google Scholar 

  46. Zhe X, Chen S, Yan H (2019) Directional statistics-based deep metric learning for image classification and retrieval. Pattern Recognit 93:113–123. https://doi.org/10.1016/j.patcog.2019.04.005

    Article  Google Scholar 

  47. Zhu X, Zhang X, Zhang XY, Xue Z, Wang L (2019) A novel framework for semantic segmentation with generative adversarial network. J Vis Commun Image Represent 58:532–543. https://doi.org/10.1016/j.jvcir.2018.11.020

    Article  Google Scholar 

  48. Zhu X, Zuo J, Ren H (2020) A modified deep neural network enables identification of foliage under complex background. Connect Sci 32(1):1–15. https://doi.org/10.1080/09540091.2019.1609420

    Article  Google Scholar 

  49. Peng Y, Long X, Lu BL (2014) Graph based semi-supervised learning via structure preserving low-rank representation. Neural Process Lett 41(3):389–406. https://doi.org/10.1007/s11063-014-9396-z

    Article  Google Scholar 

  50. Tzelepi M, Tefas A (2019) Class-specific discriminant regularization in real-time deep CNN models for binary classification problems. Neural Process Lett 51(2):1989–2005. https://doi.org/10.1007/s11063-019-10156-z

    Article  Google Scholar 

  51. Venugopal N (2020) Automatic semantic segmentation with DeepLab dilated learning network for change detection in remote sensing images. Neural Process Lett. https://doi.org/10.1007/s11063-019-10174-x

    Article  Google Scholar 

  52. Zhang Z, Zhang Y, Liu G, Tang J, Yan S, Wang M (2020) Joint label prediction based semi-supervised adaptive concept factorization for robust data representation. IEEE Trans Knowl Data Eng 32(5):951–970. https://doi.org/10.1109/TKDE.2019.2893956

    Article  Google Scholar 

  53. Zhang Y, Zhang Z, Li S, Qin J, Liu G, Wang M, Yan S (2019) Unsupervised nonnegative adaptive feature extraction for data representation. IEEE Trans Knowl Data Eng 31(12):2423–2440. https://doi.org/10.1109/TKDE.2018.2877746

    Article  Google Scholar 

  54. Zhang Z, Li F, Zhao M, Zhang L, Yan S (2017) Robust neighborhood preserving projection by nuclear/L2,1-norm regularization for image feature extraction. IEEE Trans Image Process 26(4):1607–1622. https://doi.org/10.1109/TIP.2017.2654163

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We are thankful to Prof. P.Gamba from Pavia University, Italy, for providing the ROSIS dataset and Prof. Landgrebe of Purdue University for contributing to the Indian Pines data set. The Salinas Valley dataset has obtained from “Grupo de Inteligencia Computacional (GIC) Buscar”. We are thankful to Dr. Wei Li from the School of Information and Electronics, Beijing Institute of Technology, for providing the code for CNN-PPF methods in comparative analysis. We would like to thank Cao et al, Department of Mathematics and Statistics,Xi’an Jiaoting University, China, for providing the python function codes to design the customised CNN architecture.

Author information

Authors and Affiliations

  1. Department of Computer Engineering IIT-BHU, Varanasi, India

    Vishal Srivastava & Bhaskar Biswas

Authors
  1. Vishal Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bhaskar Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vishal Srivastava.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Srivastava, V., Biswas, B. Manifold Preserving CNN for Pixel-Based Object Labelling in Images for High Dimensional Feature spaces. Neural Process Lett 53, 607–635 (2021). https://doi.org/10.1007/s11063-020-10415-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10415-4

Keywords

Search

Navigation