Skip to main content

Deep Learning in Computer Vision Through Mobile Edge Computing for IoT

  • Chapter
  • First Online:
Mobile Edge Computing

Abstract

The success of Artificial Intelligence (AI) through Deep Learning (DL) and Computer Vision has inspired many researchers to work on many real-life and human-centered tasks. These current AI systems are in use to augment the intelligence of IoT. IoT devices are equipped with very low computing and fewer storage resources. In the case of visual computing, a massive number of images or video data are needed to be processed, which seems to be not feasible for an IoT device. Therefore, those data are needed to transfer to a cloud machine for computation. However, in this case, bandwidth scarcity is a huge problem. Real-time computation and security and privacy of data are also very challenging issues. To handle this problem, Mobile Edge Computing (MEC) is used in IoT to perform the real-time computation locally. Combining state-of-the-art computer vision algorithms such as DL, especially Deep Convolutional Neural Network (CNN) based algorithms and MEC, can be a smart solution for onsite visual computing. This chapter scholarly discussed how deep CNN through MEC could be a potential technique for IoT based solutions. It also discuss how a deep transfer learning procedure can be applied in this method. This chapter proposes how different layers of deep CNN can be split up among Edge devices, Fog gateway, and Cloud servers to do visual computing at IoTs. Relevant technical backgrounds, current state-of-the-art, and future scopes are also emphasized in this chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://fas.org/irp/nic/disruptive.pdf accessed on 11/02/2020.

References

  1. J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of things (iot): A vision, architectural elements, and future directions,” Future generation computer systems, vol. 29, no. 7, pp. 1645–1660, 2013.

    Article  Google Scholar 

  2. C. Granell, A. Kamilaris, A. Kotsev, F. O. Ostermann, and S. Trilles, “Internet of things,” in Manual of Digital Earth. Springer, 2020, pp. 387–423.

    Google Scholar 

  3. P. Schulz, M. Matthe, H. Klessig, M. Simsek, G. Fettweis, J. Ansari, S. A. Ashraf, B. Almeroth, J. Voigt, I. Riedel et al., “Latency critical iot applications in 5g: Perspective on the design of radio interface and network architecture,” IEEE Communications Magazine, vol. 55, no. 2, pp. 70–78, 2017.

    Article  Google Scholar 

  4. M. Youssef and M. Hassan, “Next generation iot: Toward ubiquitous autonomous cost-efficient iot devices,” IEEE Pervasive Computing, vol. 18, no. 4, pp. 8–11, 2019.

    Article  Google Scholar 

  5. B. Afzal, M. Umair, G. A. Shah, and E. Ahmed, “Enabling iot platforms for social iot applications: vision, feature mapping, and challenges,” Future Generation Computer Systems, vol. 92, pp. 718–731, 2019.

    Article  Google Scholar 

  6. L. Atzori, A. Iera, and G. Morabito, “The internet of things: A survey,” Computer networks, vol. 54, no. 15, pp. 2787–2805, 2010.

    Article  MATH  Google Scholar 

  7. M. Mohammadi, A. Al-Fuqaha, S. Sorour, and M. Guizani, “Deep learning for iot big data and streaming analytics: A survey,” IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 2923–2960, 2018.

    Article  Google Scholar 

  8. H. Li, K. Ota, and M. Dong, “Learning iot in edge: Deep learning for the internet of things with edge computing,” IEEE Network, vol. 32, no. 1, pp. 96–101, 2018.

    Article  Google Scholar 

  9. R. F. Molanes, K. Amarasinghe, J. Rodriguez-Andina, and M. Manic, “Deep learning and reconfigurable platforms in the internet of things: Challenges and opportunities in algorithms and hardware,” IEEE industrial electronics magazine, vol. 12, no. 2, pp. 36–49, 2018.

    Article  Google Scholar 

  10. F. Xia, L. T. Yang, L. Wang, and A. Vinel, “Internet of things,” International Journal of Communication Systems, vol. 25, no. 9, p. 1101, 2012.

    Google Scholar 

  11. A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of things: A survey on enabling technologies, protocols, and applications,” IEEE communications surveys & tutorials, vol. 17, no. 4, pp. 2347–2376, 2015.

    Article  Google Scholar 

  12. L. Atzori, A. Iera, and G. Morabito, “Understanding the internet of things: definition, potentials, and societal role of a fast evolving paradigm,” Ad Hoc Networks, vol. 56, pp. 122–140, 2017.

    Article  Google Scholar 

  13. R. Khan, S. U. Khan, R. Zaheer, and S. Khan, “Future internet: the internet of things architecture, possible applications and key challenges,” in 2012 10th international conference on frontiers of information technology. IEEE, 2012, pp. 257–260.

    Google Scholar 

  14. J. Lin, W. Yu, N. Zhang, X. Yang, H. Zhang, and W. Zhao, “A survey on internet of things: Architecture, enabling technologies, security and privacy, and applications,” IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1125–1142, 2017.

    Article  Google Scholar 

  15. X. Liu, K. Lam, K. Zhu, C. Zheng, X. Li, Y. Du, C. Liu, and P. W. Pong, “Overview of spintronic sensors, internet of things, and smart living,” arXiv preprint arXiv:1611.00317, 2016.

    Google Scholar 

  16. M. Wu, T.-J. Lu, F.-Y. Ling, J. Sun, and H.-Y. Du, “Research on the architecture of internet of things,” in 2010 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), vol. 5. IEEE, 2010, pp. V5–484.

    Google Scholar 

  17. A. Botta, W. De Donato, V. Persico, and A. Pescapé, “Integration of cloud computing and internet of things: a survey,” Future generation computer systems, vol. 56, pp. 684–700, 2016.

    Article  Google Scholar 

  18. P. Bellavista, J. Berrocal, A. Corradi, S. K. Das, L. Foschini, and A. Zanni, “A survey on fog computing for the internet of things,” Pervasive and mobile computing, vol. 52, pp. 71–99, 2019.

    Article  Google Scholar 

  19. W. Yu, F. Liang, X. He, W. G. Hatcher, C. Lu, J. Lin, and X. Yang, “A survey on the edge computing for the internet of things,” IEEE access, vol. 6, pp. 6900–6919, 2017.

    Article  Google Scholar 

  20. R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, “Cloud computing and emerging it platforms: Vision, hype, and reality for delivering computing as the 5th utility,” Future Generation computer systems, vol. 25, no. 6, pp. 599–616, 2009.

    Article  Google Scholar 

  21. Y. Zhao, W. Wang, Y. Li, C. C. Meixner, M. Tornatore, and J. Zhang, “Edge computing and networking: A survey on infrastructures and applications,” IEEE Access, vol. 7, pp. 101 213–101 230, 2019.

    Google Scholar 

  22. K. Jain and S. Mohapatra, “Taxonomy of edge computing: Challenges, opportunities, and data reduction methods,” in Edge Computing. Springer, 2019, pp. 51–69.

    Google Scholar 

  23. F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, “Fog computing and its role in the internet of things,” in Proceedings of the first edition of the MCC workshop on Mobile cloud computing, 2012, pp. 13–16.

    Google Scholar 

  24. P. Escamilla-Ambrosio, A. Rodríguez-Mota, E. Aguirre-Anaya, R. Acosta-Bermejo, and M. Salinas-Rosales, “Distributing computing in the internet of things: cloud, fog and edge computing overview,” in NEO 2016. Springer, 2018, pp. 87–115.

    Google Scholar 

  25. P. Garcia Lopez, A. Montresor, D. Epema, A. Datta, T. Higashino, A. Iamnitchi, M. Barcellos, P. Felber, and E. Riviere, “Edge-centric computing: Vision and challenges,” 2015.

    Google Scholar 

  26. Y. C. Hu, M. Patel, D. Sabella, N. Sprecher, and V. Young, “Mobile edge computing-a key technology towards 5g,” ETSI white paper, vol. 11, no. 11, pp. 1–16, 2015.

    Google Scholar 

  27. E. Ahmed and M. H. Rehmani, “Mobile edge computing: opportunities, solutions, and challenges,” 2017.

    Google Scholar 

  28. Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Communications Surveys & Tutorials, vol. 19, no. 4, pp. 2322–2358, 2017.

    Article  Google Scholar 

  29. H. Elazhary, “Internet of things (iot), mobile cloud, cloudlet, mobile iot, iot cloud, fog, mobile edge, and edge emerging computing paradigms: Disambiguation and research directions,” Journal of Network and Computer Applications, vol. 128, pp. 105–140, 2019.

    Article  Google Scholar 

  30. S. Li, L. Da Xu, and S. Zhao, “The internet of things: a survey,” Information Systems Frontiers, vol. 17, no. 2, pp. 243–259, 2015.

    Article  Google Scholar 

  31. X. Sun and N. Ansari, “Edgeiot: Mobile edge computing for the internet of things,” IEEE Communications Magazine, vol. 54, no. 12, pp. 22–29, 2016.

    Article  Google Scholar 

  32. D. Sabella, A. Vaillant, P. Kuure, U. Rauschenbach, and F. Giust, “Mobile-edge computing architecture: The role of mec in the internet of things,” IEEE Consumer Electronics Magazine, vol. 5, no. 4, pp. 84–91, 2016.

    Article  Google Scholar 

  33. N. Ansari and X. Sun, “Mobile edge computing empowers internet of things,” IEICE Transactions on Communications, vol. 101, no. 3, pp. 604–619, 2018.

    Article  Google Scholar 

  34. H. Khelifi, S. Luo, B. Nour, A. Sellami, H. Moungla, S. H. Ahmed, and M. Guizani, “Bringing deep learning at the edge of information-centric internet of things,” IEEE Communications Letters, vol. 23, no. 1, pp. 52–55, 2018.

    Article  Google Scholar 

  35. B. Blanco-Filgueira, D. García-Lesta, M. Fernández-Sanjurjo, V. M. Brea, and P. López, “Deep learning-based multiple object visual tracking on embedded system for iot and mobile edge computing applications,” IEEE Internet of Things Journal, vol. 6, no. 3, pp. 5423–5431, 2019.

    Article  Google Scholar 

  36. A. Sufian, A. Ghosh, A. S. Sadiq, and F. Smarandache, “A survey on deep transfer learning to edge computing for mitigating the covid-19 pandemic,” Journal of Systems Architecture, vol. 108, p. 101830, 2020.

    Article  Google Scholar 

  37. R. Szeliski, Computer vision: algorithms and applications. Springer Science & Business Media, 2010.

    Google Scholar 

  38. M. Nixon and A. Aguado, Feature extraction and image processing for computer vision. Academic Press, 2019.

    Google Scholar 

  39. Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, and M. S. Lew, “Deep learning for visual understanding: A review,” Neurocomputing, vol. 187, pp. 27–48, 2016.

    Article  Google Scholar 

  40. A. Voulodimos, N. Doulamis, A. Doulamis, and E. Protopapadakis, “Deep learning for computer vision: A brief review,” Computational intelligence and neuroscience, vol. 2018, 2018.

    Google Scholar 

  41. Y. Shen, T. Han, Q. Yang, X. Yang, Y. Wang, F. Li, and H. Wen, “Cs-cnn: Enabling robust and efficient convolutional neural networks inference for internet-of-things applications,” IEEE Access, vol. 6, pp. 13 439–13 448, 2018.

    Google Scholar 

  42. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.

    Article  Google Scholar 

  43. I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press, 2016.

    MATH  Google Scholar 

  44. A. Ghosh, A. Sufian, F. Sultana, A. Chakrabarti, and D. De, “Fundamental concepts of convolutional neural network,” in Recent Trends and Advances in Artificial Intelligence and Internet of Things. Springer, 2020, pp. 519–567.

    Google Scholar 

  45. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012.

    Google Scholar 

  46. F. Sultana, A. Sufian, and P. Dutta, “A review of object detection models based on convolutional neural network,” in Intelligent Computing: Image Processing Based Applications. Springer, 2020, pp. 1–16.

    Google Scholar 

  47. F. Sultana, A. Sufian, and P. Dutta, “Evolution of image segmentation using deep convolutional neural network: A survey,” Knowledge-Based Systems, vol. 201–202, p. 106062, 2020.

    Article  Google Scholar 

  48. N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, and F. Kawsar, “An early resource characterization of deep learning on wearables, smartphones and internet-of-things devices,” in Proceedings of the 2015 international workshop on internet of things towards applications, 2015, pp. 7–12.

    Google Scholar 

  49. F. Alam, R. Mehmood, I. Katib, and A. Albeshri, “Analysis of eight data mining algorithms for smarter internet of things (iot),” Procedia Computer Science, vol. 98, pp. 437–442, 2016.

    Article  Google Scholar 

  50. N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, L. Jiao, L. Qendro, and F. Kawsar, “Deepx: A software accelerator for low-power deep learning inference on mobile devices,” in 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 2016, pp. 1–12.

    Google Scholar 

  51. D. Ravi, C. Wong, B. Lo, and G.-Z. Yang, “A deep learning approach to on-node sensor data analytics for mobile or wearable devices,” IEEE journal of biomedical and health informatics, vol. 21, no. 1, pp. 56–64, 2016.

    Article  Google Scholar 

  52. S. Bhattacharya and N. D. Lane, “Sparsification and separation of deep learning layers for constrained resource inference on wearables,” in Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, 2016, pp. 176–189.

    Google Scholar 

  53. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.

    Google Scholar 

  54. J. Tang, D. Sun, S. Liu, and J.-L. Gaudiot, “Enabling deep learning on iot devices,” Computer, vol. 50, no. 10, pp. 92–96, 2017.

    Article  Google Scholar 

  55. H.-Y. Kim and J.-M. Kim, “A load balancing scheme based on deep-learning in iot,” Cluster Computing, vol. 20, no. 1, pp. 873–878, 2017.

    Article  Google Scholar 

  56. S. Teerapittayanon, B. McDanel, and H.-T. Kung, “Distributed deep neural networks over the cloud, the edge and end devices,” in 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2017, pp. 328–339.

    Google Scholar 

  57. Z. Zhao, K. M. Barijough, and A. Gerstlauer, “Deepthings: Distributed adaptive deep learning inference on resource-constrained iot edge clusters,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37, no. 11, pp. 2348–2359, 2018.

    Article  Google Scholar 

  58. M. Song, K. Zhong, J. Zhang, Y. Hu, D. Liu, W. Zhang, J. Wang, and T. Li, “In-situ ai: Towards autonomous and incremental deep learning for iot systems,” in 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 2018, pp. 92–103.

    Google Scholar 

  59. M. Ali, A. Anjum, M. U. Yaseen, A. R. Zamani, D. Balouek-Thomert, O. Rana, and M. Parashar, “Edge enhanced deep learning system for large-scale video stream analytics,” in 2018 IEEE 2nd International Conference on Fog and Edge Computing (ICFEC). IEEE, 2018, pp. 1–10.

    Google Scholar 

  60. Z. Tao and Q. Li, “esgd: Communication efficient distributed deep learning on the edge,” in {USENIX} Workshop on Hot Topics in Edge Computing (HotEdge 18), 2018.

    Google Scholar 

  61. K. Bhardwaj, C.-Y. Lin, A. Sartor, and R. Marculescu, “Memory-and communication-aware model compression for distributed deep learning inference on iot,” ACM Transactions on Embedded Computing Systems (TECS), vol. 18, no. 5s, pp. 1–22, 2019.

    Article  Google Scholar 

  62. A. Ashiquzzaman, L. Van Ma, S. Kim, D. Lee, T.-W. Um, and J. Kim, “Compacting deep neural networks for light weight iot & scada based applications with node pruning,” in 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). IEEE, 2019, pp. 082–085.

    Google Scholar 

  63. J. Zhou, Y. Wang, K. Ota, and M. Dong, “Aaiot: Accelerating artificial intelligence in iot systems,” IEEE Wireless Communications Letters, vol. 8, no. 3, pp. 825–828, 2019.

    Article  Google Scholar 

  64. M. Min, L. Xiao, Y. Chen, P. Cheng, D. Wu, and W. Zhuang, “Learning-based computation offloading for iot devices with energy harvesting,” IEEE Transactions on Vehicular Technology, vol. 68, no. 2, pp. 1930–1941, 2019.

    Article  Google Scholar 

  65. J. Kang and D.-S. Eom, “Offloading and transmission strategies for iot edge devices and networks,” Sensors, vol. 19, no. 4, p. 835, 2019.

    Google Scholar 

  66. S. Tuli, N. Basumatary, and R. Buyya, “Edgelens: Deep learning based object detection in integrated iot, fog and cloud computing environments,” arXiv preprint arXiv:1906.11056, 2019.

    Google Scholar 

  67. J. Chen, K. Li, Q. Deng, K. Li, and S. Y. Philip, “Distributed deep learning model for intelligent video surveillance systems with edge computing,” IEEE Transactions on Industrial Informatics, 2019.

    Google Scholar 

  68. X. Wang, Y. Han, C. Wang, Q. Zhao, X. Chen, and M. Chen, “In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning,” IEEE Network, vol. 33, no. 5, pp. 156–165, 2019.

    Article  Google Scholar 

  69. F. Liang, W. Yu, X. Liu, D. Griffith, and N. Golmie, “Towards edge-based deep learning in industrial internet of things,” IEEE Internet of Things Journal, 2020.

    Google Scholar 

  70. E. Kristiani, C.-T. Yang, and C.-Y. Huang, “isec: An optimized deep learning model for image classification on edge computing,” IEEE Access, vol. 8, pp. 27 267–27 276, 2020.

    Google Scholar 

  71. J. Azar, A. Makhoul, R. Couturier, and J. Demerjian, “Robust iot time series classification with data compression and deep learning,” Neurocomputing, 2020.

    Google Scholar 

  72. X. Wang, Y. Han, V. C. Leung, D. Niyato, X. Yan, and X. Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” IEEE Communications Surveys & Tutorials, vol. 22, no. 2, pp. 869–904, 2020.

    Article  Google Scholar 

  73. M. Aazam, S. Zeadally, and K. A. Harras, “Offloading in fog computing for iot: Review, enabling technologies, and research opportunities,” Future Generation Computer Systems, vol. 87, pp. 278–289, 2018.

    Article  Google Scholar 

  74. M. Verhelst and B. Moons, “Embedded deep neural network processing: Algorithmic and processor techniques bring deep learning to iot and edge devices,” IEEE Solid-State Circuits Magazine, vol. 9, no. 4, pp. 55–65, 2017.

    Article  Google Scholar 

  75. S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on knowledge and data engineering, vol. 22, no. 10, pp. 1345–1359, 2009.

    Article  Google Scholar 

  76. K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” Journal of Big data, vol. 3, no. 1, p. 9, 2016.

    Google Scholar 

  77. F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, “A comprehensive survey on transfer learning,” Proceedings of the IEEE, 2020.

    Google Scholar 

  78. C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, “A survey on deep transfer learning,” in International conference on artificial neural networks. Springer, 2018, pp. 270–279.

    Google Scholar 

  79. M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” Neurocomputing, vol. 312, pp. 135–153, 2018.

    Article  Google Scholar 

  80. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255.

    Google Scholar 

  81. I. Krasin, T. Duerig, N. Alldrin, V. Ferrari, S. Abu-El-Haija, A. Kuznetsova, H. Rom, J. Uijlings, S. Popov, A. Veit et al., “Openimages: A public dataset for large-scale multi-label and multi-class image classification,” Dataset available from https://github.com/openimages , vol. 2, no. 3, p. 18, 2017.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Abu Sufian .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Sufian, A., Alam, E., Ghosh, A., Sultana, F., De, D., Dong, M. (2021). Deep Learning in Computer Vision Through Mobile Edge Computing for IoT. In: Mukherjee, A., De, D., Ghosh, S.K., Buyya, R. (eds) Mobile Edge Computing. Springer, Cham. https://doi.org/10.1007/978-3-030-69893-5_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69893-5_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69892-8

  • Online ISBN: 978-3-030-69893-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics