Abstract
The success of Artificial Intelligence (AI) through Deep Learning (DL) and Computer Vision has inspired many researchers to work on many real-life and human-centered tasks. These current AI systems are in use to augment the intelligence of IoT. IoT devices are equipped with very low computing and fewer storage resources. In the case of visual computing, a massive number of images or video data are needed to be processed, which seems to be not feasible for an IoT device. Therefore, those data are needed to transfer to a cloud machine for computation. However, in this case, bandwidth scarcity is a huge problem. Real-time computation and security and privacy of data are also very challenging issues. To handle this problem, Mobile Edge Computing (MEC) is used in IoT to perform the real-time computation locally. Combining state-of-the-art computer vision algorithms such as DL, especially Deep Convolutional Neural Network (CNN) based algorithms and MEC, can be a smart solution for onsite visual computing. This chapter scholarly discussed how deep CNN through MEC could be a potential technique for IoT based solutions. It also discuss how a deep transfer learning procedure can be applied in this method. This chapter proposes how different layers of deep CNN can be split up among Edge devices, Fog gateway, and Cloud servers to do visual computing at IoTs. Relevant technical backgrounds, current state-of-the-art, and future scopes are also emphasized in this chapter.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://fas.org/irp/nic/disruptive.pdf accessed on 11/02/2020.
References
J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of things (iot): A vision, architectural elements, and future directions,” Future generation computer systems, vol. 29, no. 7, pp. 1645–1660, 2013.
C. Granell, A. Kamilaris, A. Kotsev, F. O. Ostermann, and S. Trilles, “Internet of things,” in Manual of Digital Earth. Springer, 2020, pp. 387–423.
P. Schulz, M. Matthe, H. Klessig, M. Simsek, G. Fettweis, J. Ansari, S. A. Ashraf, B. Almeroth, J. Voigt, I. Riedel et al., “Latency critical iot applications in 5g: Perspective on the design of radio interface and network architecture,” IEEE Communications Magazine, vol. 55, no. 2, pp. 70–78, 2017.
M. Youssef and M. Hassan, “Next generation iot: Toward ubiquitous autonomous cost-efficient iot devices,” IEEE Pervasive Computing, vol. 18, no. 4, pp. 8–11, 2019.
B. Afzal, M. Umair, G. A. Shah, and E. Ahmed, “Enabling iot platforms for social iot applications: vision, feature mapping, and challenges,” Future Generation Computer Systems, vol. 92, pp. 718–731, 2019.
L. Atzori, A. Iera, and G. Morabito, “The internet of things: A survey,” Computer networks, vol. 54, no. 15, pp. 2787–2805, 2010.
M. Mohammadi, A. Al-Fuqaha, S. Sorour, and M. Guizani, “Deep learning for iot big data and streaming analytics: A survey,” IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 2923–2960, 2018.
H. Li, K. Ota, and M. Dong, “Learning iot in edge: Deep learning for the internet of things with edge computing,” IEEE Network, vol. 32, no. 1, pp. 96–101, 2018.
R. F. Molanes, K. Amarasinghe, J. Rodriguez-Andina, and M. Manic, “Deep learning and reconfigurable platforms in the internet of things: Challenges and opportunities in algorithms and hardware,” IEEE industrial electronics magazine, vol. 12, no. 2, pp. 36–49, 2018.
F. Xia, L. T. Yang, L. Wang, and A. Vinel, “Internet of things,” International Journal of Communication Systems, vol. 25, no. 9, p. 1101, 2012.
A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of things: A survey on enabling technologies, protocols, and applications,” IEEE communications surveys & tutorials, vol. 17, no. 4, pp. 2347–2376, 2015.
L. Atzori, A. Iera, and G. Morabito, “Understanding the internet of things: definition, potentials, and societal role of a fast evolving paradigm,” Ad Hoc Networks, vol. 56, pp. 122–140, 2017.
R. Khan, S. U. Khan, R. Zaheer, and S. Khan, “Future internet: the internet of things architecture, possible applications and key challenges,” in 2012 10th international conference on frontiers of information technology. IEEE, 2012, pp. 257–260.
J. Lin, W. Yu, N. Zhang, X. Yang, H. Zhang, and W. Zhao, “A survey on internet of things: Architecture, enabling technologies, security and privacy, and applications,” IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1125–1142, 2017.
X. Liu, K. Lam, K. Zhu, C. Zheng, X. Li, Y. Du, C. Liu, and P. W. Pong, “Overview of spintronic sensors, internet of things, and smart living,” arXiv preprint arXiv:1611.00317, 2016.
M. Wu, T.-J. Lu, F.-Y. Ling, J. Sun, and H.-Y. Du, “Research on the architecture of internet of things,” in 2010 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), vol. 5. IEEE, 2010, pp. V5–484.
A. Botta, W. De Donato, V. Persico, and A. Pescapé, “Integration of cloud computing and internet of things: a survey,” Future generation computer systems, vol. 56, pp. 684–700, 2016.
P. Bellavista, J. Berrocal, A. Corradi, S. K. Das, L. Foschini, and A. Zanni, “A survey on fog computing for the internet of things,” Pervasive and mobile computing, vol. 52, pp. 71–99, 2019.
W. Yu, F. Liang, X. He, W. G. Hatcher, C. Lu, J. Lin, and X. Yang, “A survey on the edge computing for the internet of things,” IEEE access, vol. 6, pp. 6900–6919, 2017.
R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, “Cloud computing and emerging it platforms: Vision, hype, and reality for delivering computing as the 5th utility,” Future Generation computer systems, vol. 25, no. 6, pp. 599–616, 2009.
Y. Zhao, W. Wang, Y. Li, C. C. Meixner, M. Tornatore, and J. Zhang, “Edge computing and networking: A survey on infrastructures and applications,” IEEE Access, vol. 7, pp. 101 213–101 230, 2019.
K. Jain and S. Mohapatra, “Taxonomy of edge computing: Challenges, opportunities, and data reduction methods,” in Edge Computing. Springer, 2019, pp. 51–69.
F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, “Fog computing and its role in the internet of things,” in Proceedings of the first edition of the MCC workshop on Mobile cloud computing, 2012, pp. 13–16.
P. Escamilla-Ambrosio, A. Rodríguez-Mota, E. Aguirre-Anaya, R. Acosta-Bermejo, and M. Salinas-Rosales, “Distributing computing in the internet of things: cloud, fog and edge computing overview,” in NEO 2016. Springer, 2018, pp. 87–115.
P. Garcia Lopez, A. Montresor, D. Epema, A. Datta, T. Higashino, A. Iamnitchi, M. Barcellos, P. Felber, and E. Riviere, “Edge-centric computing: Vision and challenges,” 2015.
Y. C. Hu, M. Patel, D. Sabella, N. Sprecher, and V. Young, “Mobile edge computing-a key technology towards 5g,” ETSI white paper, vol. 11, no. 11, pp. 1–16, 2015.
E. Ahmed and M. H. Rehmani, “Mobile edge computing: opportunities, solutions, and challenges,” 2017.
Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Communications Surveys & Tutorials, vol. 19, no. 4, pp. 2322–2358, 2017.
H. Elazhary, “Internet of things (iot), mobile cloud, cloudlet, mobile iot, iot cloud, fog, mobile edge, and edge emerging computing paradigms: Disambiguation and research directions,” Journal of Network and Computer Applications, vol. 128, pp. 105–140, 2019.
S. Li, L. Da Xu, and S. Zhao, “The internet of things: a survey,” Information Systems Frontiers, vol. 17, no. 2, pp. 243–259, 2015.
X. Sun and N. Ansari, “Edgeiot: Mobile edge computing for the internet of things,” IEEE Communications Magazine, vol. 54, no. 12, pp. 22–29, 2016.
D. Sabella, A. Vaillant, P. Kuure, U. Rauschenbach, and F. Giust, “Mobile-edge computing architecture: The role of mec in the internet of things,” IEEE Consumer Electronics Magazine, vol. 5, no. 4, pp. 84–91, 2016.
N. Ansari and X. Sun, “Mobile edge computing empowers internet of things,” IEICE Transactions on Communications, vol. 101, no. 3, pp. 604–619, 2018.
H. Khelifi, S. Luo, B. Nour, A. Sellami, H. Moungla, S. H. Ahmed, and M. Guizani, “Bringing deep learning at the edge of information-centric internet of things,” IEEE Communications Letters, vol. 23, no. 1, pp. 52–55, 2018.
B. Blanco-Filgueira, D. García-Lesta, M. Fernández-Sanjurjo, V. M. Brea, and P. López, “Deep learning-based multiple object visual tracking on embedded system for iot and mobile edge computing applications,” IEEE Internet of Things Journal, vol. 6, no. 3, pp. 5423–5431, 2019.
A. Sufian, A. Ghosh, A. S. Sadiq, and F. Smarandache, “A survey on deep transfer learning to edge computing for mitigating the covid-19 pandemic,” Journal of Systems Architecture, vol. 108, p. 101830, 2020.
R. Szeliski, Computer vision: algorithms and applications. Springer Science & Business Media, 2010.
M. Nixon and A. Aguado, Feature extraction and image processing for computer vision. Academic Press, 2019.
Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, and M. S. Lew, “Deep learning for visual understanding: A review,” Neurocomputing, vol. 187, pp. 27–48, 2016.
A. Voulodimos, N. Doulamis, A. Doulamis, and E. Protopapadakis, “Deep learning for computer vision: A brief review,” Computational intelligence and neuroscience, vol. 2018, 2018.
Y. Shen, T. Han, Q. Yang, X. Yang, Y. Wang, F. Li, and H. Wen, “Cs-cnn: Enabling robust and efficient convolutional neural networks inference for internet-of-things applications,” IEEE Access, vol. 6, pp. 13 439–13 448, 2018.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press, 2016.
A. Ghosh, A. Sufian, F. Sultana, A. Chakrabarti, and D. De, “Fundamental concepts of convolutional neural network,” in Recent Trends and Advances in Artificial Intelligence and Internet of Things. Springer, 2020, pp. 519–567.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012.
F. Sultana, A. Sufian, and P. Dutta, “A review of object detection models based on convolutional neural network,” in Intelligent Computing: Image Processing Based Applications. Springer, 2020, pp. 1–16.
F. Sultana, A. Sufian, and P. Dutta, “Evolution of image segmentation using deep convolutional neural network: A survey,” Knowledge-Based Systems, vol. 201–202, p. 106062, 2020.
N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, and F. Kawsar, “An early resource characterization of deep learning on wearables, smartphones and internet-of-things devices,” in Proceedings of the 2015 international workshop on internet of things towards applications, 2015, pp. 7–12.
F. Alam, R. Mehmood, I. Katib, and A. Albeshri, “Analysis of eight data mining algorithms for smarter internet of things (iot),” Procedia Computer Science, vol. 98, pp. 437–442, 2016.
N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, L. Jiao, L. Qendro, and F. Kawsar, “Deepx: A software accelerator for low-power deep learning inference on mobile devices,” in 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 2016, pp. 1–12.
D. Ravi, C. Wong, B. Lo, and G.-Z. Yang, “A deep learning approach to on-node sensor data analytics for mobile or wearable devices,” IEEE journal of biomedical and health informatics, vol. 21, no. 1, pp. 56–64, 2016.
S. Bhattacharya and N. D. Lane, “Sparsification and separation of deep learning layers for constrained resource inference on wearables,” in Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, 2016, pp. 176–189.
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
J. Tang, D. Sun, S. Liu, and J.-L. Gaudiot, “Enabling deep learning on iot devices,” Computer, vol. 50, no. 10, pp. 92–96, 2017.
H.-Y. Kim and J.-M. Kim, “A load balancing scheme based on deep-learning in iot,” Cluster Computing, vol. 20, no. 1, pp. 873–878, 2017.
S. Teerapittayanon, B. McDanel, and H.-T. Kung, “Distributed deep neural networks over the cloud, the edge and end devices,” in 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2017, pp. 328–339.
Z. Zhao, K. M. Barijough, and A. Gerstlauer, “Deepthings: Distributed adaptive deep learning inference on resource-constrained iot edge clusters,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37, no. 11, pp. 2348–2359, 2018.
M. Song, K. Zhong, J. Zhang, Y. Hu, D. Liu, W. Zhang, J. Wang, and T. Li, “In-situ ai: Towards autonomous and incremental deep learning for iot systems,” in 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 2018, pp. 92–103.
M. Ali, A. Anjum, M. U. Yaseen, A. R. Zamani, D. Balouek-Thomert, O. Rana, and M. Parashar, “Edge enhanced deep learning system for large-scale video stream analytics,” in 2018 IEEE 2nd International Conference on Fog and Edge Computing (ICFEC). IEEE, 2018, pp. 1–10.
Z. Tao and Q. Li, “esgd: Communication efficient distributed deep learning on the edge,” in {USENIX} Workshop on Hot Topics in Edge Computing (HotEdge 18), 2018.
K. Bhardwaj, C.-Y. Lin, A. Sartor, and R. Marculescu, “Memory-and communication-aware model compression for distributed deep learning inference on iot,” ACM Transactions on Embedded Computing Systems (TECS), vol. 18, no. 5s, pp. 1–22, 2019.
A. Ashiquzzaman, L. Van Ma, S. Kim, D. Lee, T.-W. Um, and J. Kim, “Compacting deep neural networks for light weight iot & scada based applications with node pruning,” in 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). IEEE, 2019, pp. 082–085.
J. Zhou, Y. Wang, K. Ota, and M. Dong, “Aaiot: Accelerating artificial intelligence in iot systems,” IEEE Wireless Communications Letters, vol. 8, no. 3, pp. 825–828, 2019.
M. Min, L. Xiao, Y. Chen, P. Cheng, D. Wu, and W. Zhuang, “Learning-based computation offloading for iot devices with energy harvesting,” IEEE Transactions on Vehicular Technology, vol. 68, no. 2, pp. 1930–1941, 2019.
J. Kang and D.-S. Eom, “Offloading and transmission strategies for iot edge devices and networks,” Sensors, vol. 19, no. 4, p. 835, 2019.
S. Tuli, N. Basumatary, and R. Buyya, “Edgelens: Deep learning based object detection in integrated iot, fog and cloud computing environments,” arXiv preprint arXiv:1906.11056, 2019.
J. Chen, K. Li, Q. Deng, K. Li, and S. Y. Philip, “Distributed deep learning model for intelligent video surveillance systems with edge computing,” IEEE Transactions on Industrial Informatics, 2019.
X. Wang, Y. Han, C. Wang, Q. Zhao, X. Chen, and M. Chen, “In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning,” IEEE Network, vol. 33, no. 5, pp. 156–165, 2019.
F. Liang, W. Yu, X. Liu, D. Griffith, and N. Golmie, “Towards edge-based deep learning in industrial internet of things,” IEEE Internet of Things Journal, 2020.
E. Kristiani, C.-T. Yang, and C.-Y. Huang, “isec: An optimized deep learning model for image classification on edge computing,” IEEE Access, vol. 8, pp. 27 267–27 276, 2020.
J. Azar, A. Makhoul, R. Couturier, and J. Demerjian, “Robust iot time series classification with data compression and deep learning,” Neurocomputing, 2020.
X. Wang, Y. Han, V. C. Leung, D. Niyato, X. Yan, and X. Chen, “Convergence of edge computing and deep learning: A comprehensive survey,” IEEE Communications Surveys & Tutorials, vol. 22, no. 2, pp. 869–904, 2020.
M. Aazam, S. Zeadally, and K. A. Harras, “Offloading in fog computing for iot: Review, enabling technologies, and research opportunities,” Future Generation Computer Systems, vol. 87, pp. 278–289, 2018.
M. Verhelst and B. Moons, “Embedded deep neural network processing: Algorithmic and processor techniques bring deep learning to iot and edge devices,” IEEE Solid-State Circuits Magazine, vol. 9, no. 4, pp. 55–65, 2017.
S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on knowledge and data engineering, vol. 22, no. 10, pp. 1345–1359, 2009.
K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” Journal of Big data, vol. 3, no. 1, p. 9, 2016.
F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He, “A comprehensive survey on transfer learning,” Proceedings of the IEEE, 2020.
C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, “A survey on deep transfer learning,” in International conference on artificial neural networks. Springer, 2018, pp. 270–279.
M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” Neurocomputing, vol. 312, pp. 135–153, 2018.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255.
I. Krasin, T. Duerig, N. Alldrin, V. Ferrari, S. Abu-El-Haija, A. Kuznetsova, H. Rom, J. Uijlings, S. Popov, A. Veit et al., “Openimages: A public dataset for large-scale multi-label and multi-class image classification,” Dataset available from https://github.com/openimages , vol. 2, no. 3, p. 18, 2017.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Sufian, A., Alam, E., Ghosh, A., Sultana, F., De, D., Dong, M. (2021). Deep Learning in Computer Vision Through Mobile Edge Computing for IoT. In: Mukherjee, A., De, D., Ghosh, S.K., Buyya, R. (eds) Mobile Edge Computing. Springer, Cham. https://doi.org/10.1007/978-3-030-69893-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-69893-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-69892-8
Online ISBN: 978-3-030-69893-5
eBook Packages: Computer ScienceComputer Science (R0)