Skip to main content
SpringerLink
Log in

IaaS-Application Development for Paralleled Remote Sensing Data Stream Processing

  • Conference paper
  • First Online:
Lecture Notes in Data Engineering, Computational Intelligence, and Decision Making (ISDMCI 2022)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 149))

  • 476 Accesses

Abstract

During the last decade, huge advancements took place in remote sensing affecting speed and quality of data acquisition, processing and mapping. This rapid progress became possible especially due to shifts in software development involving cloud technologies as well as success of deep neural networks (DNN) in processing tremendous amount of data. In particular tasks such as image processing, DNNs achieve remarkable results considering accuracy and precision of pattern recognition. In order to achieve human-level performance solving image processing problem, Deep Learning (DL) solutions must be appropriately trained using powerful hardware worth of thousands of dollars which is often suboptimal to handle using on-premises servers. Current work suggests infrastructure-as-a-service (IaaS) solution for landing remote sensing (RS) data stream processing algorithm involving the latest advancements in image processing using DL solutions and cloud technologies optimized for work with the suggested algorithm such as Kubernetes and Apache Airflow hosted on Google Computing Platform (GCP). The suggested algorithm is represented as a directed acyclic graph (DAG) within IaaS-application. The mentioned cloud technologies are used for better representation of the workflow which implements a complex system for paralleling compute-heavy tasks of very high resolution (VHR) satellite imagery processing to provide the visualized segmentation results for urban development maps in a fast and efficient way.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baret, F., Buis, S.: Estimating canopy characteristics from remote sensing observations: review of methods and associated problems. Adv. Land Remote Sens. pp. 173–201 (2008). https://doi.org/10.1007/978-1-4020-6450-0_7

  2. Bisong, E.: Containers and google kubernetes engine. In: Building Machine Learning and Deep Learning Models on Google Cloud Platform, pp. 655–670. Apress, Berkeley, CA (2019). https://doi.org/10.1007/978-1-4842-4470-8_45

    Chapter  Google Scholar 

  3. Bruzzone, L., Demir, B.: A review of modern approaches to classification of remote sensing data. In: Manakos, I., Braun, M. (eds.) Land Use and Land Cover Mapping in Europe. RSDIP, vol. 18, pp. 127–143. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-007-7969-3_9

    Chapter  Google Scholar 

  4. BUBER, E., DIRI, B.: Performance analysis and CPU vs GPU comparison for deep learning. In: 2018 6th International Conference on Control Engineering Information Technology (CEIT), pp. 1–6 (2018). https://doi.org/10.1109/CEIT.2018.8751930

  5. Frogner, C., Zhang, C., Mobahi, H., et al.: Learning with a Wasserstein loss. In: Advances in Neural Information Processing Systems, vol. 28 (2015). https://doi.org/10.48550/arXiv.1506.05439

  6. Fu, Y., Guo, H., Li, M., et al.: Cpt: efficient deep neural network training via cyclic precision. arXiv preprint arXiv:2101.09868 (2021). https://doi.org/10.48550/arXiv.2101.09868

  7. Ghanbari, H., Mahdianpari, M., Homayouni, S., Mohammadimanesh, F.: A meta-analysis of convolutional neural networks for remote sensing applications. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 14, 3602–3613 (2021). https://doi.org/10.1109/JSTARS.2021.3065569

    Article  Google Scholar 

  8. Hnatushenko, V., Hnatushenko, V., Kavats, O., et al.: Pansharpening technology of high resolution multispectral and panchromatic satellite images. Sci. Bull. Nat. Min. Univ. 4, 91–98 (2015)

    Google Scholar 

  9. Hnatushenko, V., Zhernovyi, V.: Complex Approach of High-Resolution Multispectral Data Engineering for Deep Neural Network Processing. In: Lytvynenko, V., Babichev, S., Wójcik, W., Vynokurova, O., Vyshemyrskaya, S., Radetskaya, S. (eds.) ISDMCI 2019. AISC, vol. 1020, pp. 659–672. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-26474-1_46

    Chapter  Google Scholar 

  10. Hnatushenko, V., Zhernovyi, V.: Method of improving instance segmentation for very high resolution remote sensing imagery using deep learning. In: Babichev, S., Peleshko, D., Vynokurova, O. (eds.) DSMP 2020. CCIS, vol. 1158, pp. 323–333. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61656-4_21

    Chapter  Google Scholar 

  11. Hnatushenko, V., Zhernovyi, V., Udovyk, I., Shevtsova, O.: Intelligent system for building separation on a semantically segmented map. In: CEUR Workshop Proceedings, pp. 1–11 (2021)

    Google Scholar 

  12. Hong, S., Roh, B., Kim, K.H., et al.: PvaNet: lightweight deep neural networks for real-time object detection. arXiv preprint arXiv:1611.08588 (2016). https://doi.org/10.48550/arXiv.1611.08588

  13. Hordiiuk, D., Hnatushenko, V.: Neural network and local laplace filter methods applied to very high resolution remote sensing imagery in urban damage detection. In: 2017 IEEE International Young Scientists Forum on Applied Physics and Engineering (YSF), pp. 363–366 (2017). https://doi.org/10.1109/YSF.2017.8126648

  14. Hutchinson, M., Antono, E., Gibbons, B., et al.: Overcoming data scarcity with transfer learning. arXiv preprint arXiv:1711.05099 (2017). https://doi.org/10.48550/arXiv.1711.05099

  15. Jain, P., Mo, X., Jain, A., Subbaraj, H., et al.: Dynamic space-time scheduling for GPU inference. arXiv preprint arXiv:1901.00041 (2018). https://doi.org/10.48550/arXiv.1901.00041

  16. Kaab, A.: Remote sensing of permafrost-related problems and hazards. Permafrost Periglac. Process. 19(2), 107–136 (2008). https://doi.org/10.1002/ppp.619

    Article  Google Scholar 

  17. Kimmel, J., Mcdole, A., Abdelsalam, M., Gupta, M., Sandhu, R.: Recurrent neural networks based online behavioural malware detection techniques for cloud infrastructure. IEEE Access 9, 68066–68080 (2021). https://doi.org/10.1109/ACCESS.2021.3077498

    Article  Google Scholar 

  18. Madiajagan, M., Raj, S.: Parallel computing, graphics processing unit (GPU) and new hardware for deep learning in computational intelligence research. In: Deep Learning and Parallel Computing Environment for Bioengineering Systems, pp. 1–15. Elsevier (2019). https://doi.org/10.1016/B978-0-12-816718-2.00008-7

  19. Mueller, P.: Cryptocurrency mining: asymmetric response to price movement. Available at SSRN 3733026 (2020). https://doi.org/10.2139/ssrn.3733026

  20. Natarajan, A., Ganesan, D., Marlin, B.: Hierarchical active learning for model personalization in the presence of label scarcity. In: 2019 IEEE 16th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 1–4. IEEE (2019). https://doi.org/10.1109/BSN.2019.8771081

  21. Ranjit, M.P., Ganapathy, G., Sridhar, K., Arumugham, V.: Efficient deep learning hyperparameter tuning using cloud infrastructure: Intelligent distributed hyperparameter tuning with Bayesian optimization in the cloud. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 520–522 (2019). https://doi.org/10.1109/CLOUD.2019.00097

  22. Sethi, K., Kumar, R., Prajapati, N., Bera, P.: Deep reinforcement learning based intrusion detection system for cloud infrastructure. In: 2020 International Conference on Communication Systems Networks (COMSNETS), pp. 1–6 (2020). https://doi.org/10.1109/COMSNETS48256.2020.9027452

  23. Sovrano, F., Palmirani, M., Vitali, F.: Combining shallow and deep learning approaches against data scarcity in legal domains. Gov. Inf. Quart. 39(3), 101715 (2022). https://doi.org/10.1016/j.giq.2022.101715

    Article  Google Scholar 

  24. Strom, N.: Scalable distributed DNN training using commodity GPU cloud computing. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)

    Google Scholar 

  25. Sze, V., Chen, Y.H., Yang, T.J., Emer, J.: Efficient processing of deep neural networks: a tutorial and survey. Proc. IEEE 105(12), 2295–2329 (2017). https://doi.org/10.1109/JPROC.2017.2761740

    Article  Google Scholar 

  26. Sze, V., Chen, Y.H., Yang, T.J., Emer, J.: Efficient processing of deep neural networks. Synth. Lect. Comput. Archit. 15(2), 1–341 (2020). https://doi.org/10.2200/S01004ED1V01Y202004CAC050

    Article  MATH  Google Scholar 

  27. Teerapittayanon, S., McDanel, B., Kung, H.T.: Distributed deep neural networks over the cloud, the edge and end devices. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 328–339. IEEE (2017). https://doi.org/10.1109/ICDCS.2017.226

  28. Walter-Tscharf, V.: Implementation and evaluation of a MLaaS for document classification with continuous deep learning models. In: Architecture, Engineering, and Technology (AET), p.55

    Google Scholar 

  29. Wiedemann, S., Mehari, T., Kepp, K., Samek, W.: Dithered backprop: A sparse and quantized backpropagation algorithm for more efficient deep neural network training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 720–721 (2020). https://doi.org/10.48550/arXiv.2004.04729

  30. Yao, Y., Deng, J., Chen, X., et al.: Deep discriminative CNN with temporal ensembling for ambiguously-labeled image classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12669–12676 (2020). https://doi.org/10.1609/aaai.v34i07.6959

  31. You, Y., Zhang, Z., Hsieh, C.J., Demmel, J., Keutzer, K.: Fast deep neural network training on distributed systems and cloud tpus. IEEE Trans. Parallel Distrib. Syst. 30(11), 2449–2462 (2019). https://doi.org/10.1109/TPDS.2019.2913833

    Article  Google Scholar 

  32. Yu, Z., Fu, Y., Wu, S., et al.: Ldp: learnable dynamic precision for efficient deep neural network training and inference. arXiv preprint arXiv:2203.07713 (2022). https://doi.org/10.48550/arXiv.2203.07713

  33. Zheng, H., Xu, F., Chen, L., Zhou, Z., Liu, F.: Cynthia: Cost-efficient cloud resource provisioning for predictable distributed deep neural network training. In: Proceedings of the 48th International Conference on Parallel Processing, pp. 1–11 (2019). https://doi.org/10.1145/3337821.3337873

Download references

Author information

Authors and Affiliations

  1. Oles Honchar Dnipro National University, Dnipro, Ukraine

    Vadym Zhernovyi

  2. Dnipro University of Technology, Dnipro, Ukraine

    Volodymyr Hnatushenko & Olga Shevtsova

Authors
  1. Vadym Zhernovyi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Volodymyr Hnatushenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Olga Shevtsova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Volodymyr Hnatushenko .

Editor information

Editors and Affiliations

  1. Jan Evangelista Purkyně University in Ústi nad Labem, Ústi nad Labem, Czech Republic

    Sergii Babichev

  2. Kherson National Technical University, Kherson, Ukraine

    Volodymyr Lytvynenko

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhernovyi, V., Hnatushenko, V., Shevtsova, O. (2023). IaaS-Application Development for Paralleled Remote Sensing Data Stream Processing. In: Babichev, S., Lytvynenko, V. (eds) Lecture Notes in Data Engineering, Computational Intelligence, and Decision Making. ISDMCI 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 149. Springer, Cham. https://doi.org/10.1007/978-3-031-16203-9_39

Download citation

Publish with us

Policies and ethics

Search

Navigation