Skip to main content

Advertisement

Springer Nature Link
Log in

Toward a mixed reality domain model for time-Sensitive applications using IoE infrastructure and edge computing (MRIoEF)

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Mixed reality (MR) is one of the technologies with many challenges in the design and implementation phases, especially the problems associated with time-sensitive applications. The main objective of this paper is to introduce a conceptual model for MR application that gives MR application a new layer of interactivity by using Internet of things/Internet of everything models, which provide an improved quality of experience for end-users. The model supports the cloud and fog compute layers to give more functionalities that need more processing resources and reduce the latency problems for time-sensitive applications. Validation of the proposed model is performed via demonstrating a prototype of the model applied to a real-time case study and discussing how to enable standard technologies of the various components in the model. Moreover, it shows the applicability of the model, the ease of defining the roles, and the coherence of data or processes found in the most common applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

References

  1. Miraz MH, Ali M, Excell PS, Picking R (2015) A review on internet of things (IoT), internet of everything (IoE) and internet of nano things (IoNT). Int Technol Appl (ITA) 2015:219–224. https://doi.org/10.1109/ITechA.2015.7317398

    Article  Google Scholar 

  2. Langley DJ, van Doorn J, Ng ICL, Stieglitz S, Lazovik A, Boonstra A (2021) The internet of everything: smart things and their impact on business models. J Bus Res 122:853–863. https://doi.org/10.1016/j.jbusres.2019.12.035

    Article  Google Scholar 

  3. Di Martino B, Li KC, Yang LT, Esposito A (2018) Trends and Strategic Researches in Internet of Everything. In: Di Martino B, Li KC, Yang L, Esposito A (eds) Internet of Everything. Internet of Things(Technology, Communications, and Computing), Springer, Singapore. https://doi.org/10.1007/978-981-10-5861-5_1

    Chapter  Google Scholar 

  4. Čolaković A, Hadžialić M (2018) Internet of things (IoT): a review of enabling technologies, challenges, and open research issues. Comp Net. 144:17–39. https://doi.org/10.1016/j.comnet.2018.07.017

    Article  Google Scholar 

  5. A. Feldmann et al. (2021) Implications of the COVID-19 Pandemic on the Internet Traffic, Broadband Coverage in Germany; 15th ITG-Symposium, p. 1–5.

  6. Khan WZ, Ahmed E, Hakak S, Yaqoob I, Ahmed A (2019) Edge computing: a survey. Fut Gen Comp Syst. 97:219–235. https://doi.org/10.1016/j.future.2019.02.050

    Article  Google Scholar 

  7. Nicolescu R, Huth M, Radanliev P et al (2018) Mapping the values of IoT. J Inf Technol 33:345–360. https://doi.org/10.1057/s41265-018-0054-1

    Article  Google Scholar 

  8. Kazmi A, Jan Z, Zappa, A, Serrano M (2016). Overcoming the heterogeneity in the Internet of things for smart cities. In International workshop on interoperability and open-source solutions. Springer, Cham, p. 20–35

  9. Bilal K, Khalid O, Erbad A, Khan SU (2018) Potentials, trends, and prospects in edge technologies: fog, cloudlet, mobile edge, and micro data centers. Comp Netw. 130:94–120. https://doi.org/10.1016/j.comnet.2017.10.002

    Article  Google Scholar 

  10. Dizdarevic J, Carpio F, Jukan A, Masip X (2018) A survey of communication protocols for internet of things and related challenges of fog and cloud computing integration. ACM Comp Surv. https://doi.org/10.1145/3292674

    Article  Google Scholar 

  11. Khorsand R, Ghobaei-Arani M, Ramezanpour M (2018) FAHP approach for autonomic resource provisioning of multitier applications in cloud computing environments. Softw Pract Exper 48:2147–2173. https://doi.org/10.1002/spe.2627

    Article  Google Scholar 

  12. Shahidinejad A, Ghobaei Arani M (2020) Joint computation offloading and resource provisioning for edge cloud computing environment: a machine learning based approach. Softw Pract Exper. 50:2212–2230

    Article  Google Scholar 

  13. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans. Inform Syst. E77-D(12):1321–1329

    Google Scholar 

  14. https://www.google.com/glass/start/. Accessed 1 Jan 2022

  15. https://www.microsoft.com/en-us/hololens. Accessed 1 Jan 2022

  16. Cao A, Dhanaliwala A, Shi J, Gade T, Park B (2019) Image based marker tracking and registration for intraoperative image guided interventions using augmented reality. Int Soc Opt Phot 11318:1131802

    Google Scholar 

  17. Luyang Liu, Hongyu L, Marco G. (2019). Edge assisted real-time object detection for mobile augmented reality. In: The 25th Annual International Conference on Mobile Computing and Networking (MobiCom '19). Association for Computing Machinery, New York, NY, USA, Article 25, 1–16. https://doi.org/10.1145/3300061.3300116

  18. Tsunezaki S, Nomura R, Komuro T, Yamamoto S, Tsumura N (2018) Reproducing material appearance of real objects using mobile augmented reality. In: IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) Munich, Germany 196 197 https://doi.org/10.1109/ISMAR-Adjunct.2018.00065

  19. A. Meka et al., "LIME: Live Intrinsic Material Estimation (2018) IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, p. 6315–6324. https://doi.org/10.1109/CVPR.2018.00661.

  20. Shannigrahi S, Mastorakis S, Ortega FR (2020) Next-generation networking and edge computing for mixed reality real-time interactive systems. In: IEEE International Conference on Communications Workshops (ICC Workshops) 1 6 https://doi.org/10.1109/ICCWorkshops49005.2020.9145075

  21. Fu L, Landay J, Nebeling M, Xu Y-Q, Zhao C. (2018). Redefining Natural User Interface. 1–3. https://doi.org/10.1145/3170427.3190649

  22. Zietsch J, Büth L, Juraschek M, Weinert N, Thiede S, Herrmann C (2019) Identifying the potential of edge computing in factories through mixed reality. Procedia CIRP. 81:1095–1100. https://doi.org/10.1016/j.procir.2019.03.259

    Article  Google Scholar 

  23. Fernández Caramés TM, Fraga Lamas P, Suárez Albela M, Vilar Montesinos M. A fog computing and cloudlet based augmented reality system for the industry shipyard. Sensors. 18, 1798. https://doi.org/10.3390/s18061798

  24. Phupattanasilp P, Tong S-R (2019) Augmented reality in the integrative internet of things (AR-IoT): application for precision farming. Sustainability 11:2658. https://doi.org/10.3390/su11092658

    Article  Google Scholar 

  25. Blanco-Novoa Ó, Fraga-Lamas PA, Vilar-Montesinos M, Fernández-Caramés TM (2020) Creating the internet of augmented things: an open-source framework to make IoT devices and augmented and mixed reality systems talk to each other. Sensors 20:3328. https://doi.org/10.3390/s20113328

    Article  Google Scholar 

  26. Bucsai S, Kučera E, Haffner O, Drahoš P (2020) Control and monitoring of IoT devices using mixed reality developed by unity engine. Cybern Inform (K&I) Velke Karlovice, Czech Rep. https://doi.org/10.1109/KI48306.2020.9039901

    Article  Google Scholar 

  27. Morris A, Guan J, Lessio N, Shao Y (2020) Toward mixed reality hybrid objects with IoT avatar agents. In: 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Toronto, ON, Canada p. 766–773. https://doi.org/10.1109/SMC42975.2020.9282914

  28. Vidal-Balea A, Blanco-Novoa O, Picallo-Guembe I, Celaya-Echarri M, Fraga-Lamas P, Lopez-Iturri P, Azpilicueta L, Falcone F, Fernández-Caramés TM (2020) Analysis, design and practical validation of an augmented reality teaching system based on microsoft hololens 2 and edge computing. Eng Proc 2:52. https://doi.org/10.3390/ecsa-7-08210

    Article  Google Scholar 

  29. Miraz MH, Ali M, Excell PS, Picking R (2015) A review on Internet of things (IoT), internet of everything (IoE) and internet of nano things (IoNT. ) Int Technol Appl (ITA) Wrexham. https://doi.org/10.1109/ITechA.2015.7317398

    Article  Google Scholar 

  30. https://tej.ie/diors-instagram-ar-virtual-makeup-experience/. Accessed 1 Jan 2022

  31. Bainomugisha Engineer L, Carreton C, Van Andoni S, Tom W, de olfgang M (2012) A survey on reactive programming. ACM Comput Surv. https://doi.org/10.1145/25016542501666

    Article  Google Scholar 

  32. https://github.com/dotnet/reactive. Accessed 1 Jan 2022

  33. https://rxjs-dev.firebaseapp.com/api. Accessed 1 Jan 2022

  34. Weisshaar BP, Kolnick FC, Kun AI, Mansfield BM. (1987) Computer X Inc,. Method of inter-process communication in a distributed data processing system. U.S. Patent. 4 694–396

  35. Baskaran S, Nagabushanam HK (2018) Relational localization based Augmented reality Interface for IoT applications. In: 2018 International Conference on Information and Communication Technology Convergence (ICTC), Jeju p. 103–106. https://doi.org/10.1109/ICTC.2018.8539469.

  36. Smith NM, Heldt-Sheller N (2020) Intel Corp. Internet of things (iot) network domain resource model. U.S. Patent Application 16/609,711

  37. https://gampltd.com/. Accessed 1 Jan 2022

  38. https://myhives.eu/. Accessed 1 Jan 2022

  39. https://ec.europa.eu/easme/en/life. Accessed 1 Jan 2022

  40. https://pycom.io/product/lopy4/. Accessed 1 Jan 2022

  41. https://pycom.io/. Accessed 1 Jan 2022

  42. https://pycom.io/product/fipy/. Accessed 1 Jan 2022

  43. https://developer.nvidia.com/embedded/jetson-nano-developer-kit. Accessed 1 Jan 2022

  44. https://www.nvidia.com/. Accessed 1 Jan 2022

  45. https://www.magicleap.com/en-us. Accessed 1 Jan 2022

  46. http://zugara.com/. Accessed 1 Jan 2022

  47. http://openbiometrics.org/. Accessed 1 Jan 2022

  48. https://github.com/facebookresearch/detectron2. Accessed 1 Jan 2022

  49. Toczé K, Lindqvist J, Nadjm-Tehrani S (2020) Characterization and modeling of an edge computing mixed reality workload. J Cloud Comp 9(1):1–24

    Article  Google Scholar 

  50. Toczé K, Lindqvist J, Nadjm Tehrani S. (2019). Performance study of mixed reality for edge computing. In: Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing p. 285–294

Download references

Author information

Authors and Affiliations

  1. Computer and Control Department, Faculty of Engineering, Tanta University, Tanta, Egypt

    Mohamed Elawady, Amany Sarhan & Mahmoud A. M. Alshewimy

Authors
  1. Mohamed Elawady
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Amany Sarhan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mahmoud A. M. Alshewimy
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Mohamed Elawady.

Ethics declarations

Conflict of interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elawady, M., Sarhan, A. & Alshewimy, M.A.M. Toward a mixed reality domain model for time-Sensitive applications using IoE infrastructure and edge computing (MRIoEF). J Supercomput 78, 10656–10689 (2022). https://doi.org/10.1007/s11227-022-04307-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04307-8

Keywords