Skip to main content
SpringerLink
Log in

Analytic Study of Containerizing Stateful Stream Processing as Microservice to Support Digital Twins in Fog Computing

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

Digital twins of processes and devices use information from sensors to synchronize their state with the entities of the physical world. The concept of stream computing enables effective processing of events generated by such sensors. However, the need to track the state of an instance of the object leads to the impossibility of organizing instances of digital twins as stateless services. Another feature of digital twins is that several tasks implemented on their basis require the ability to respond to incoming events at near-real-time speed. In this case, the use of cloud computing becomes unacceptable due to high latency. Fog computing manages this problem by moving some computational tasks closer to the data sources. One of the recent solutions providing the development of loosely coupled distributed systems is a Microservice approach, which implies the organization of the distributed system as a set of coherent and independent services interacting with each other using messages. The microservice is most often isolated by utilizing containers to overcome the high overheads of using virtual machines. The main problem is that microservices and containers together are stateless by nature. The container technology still does not fully support live container migration between physical hosts without data loss. It causes challenges in ensuring the uninterrupted operation of services in fog computing environments. Thus, an essential challenge is to create a containerized stateful stream processing based microservice to support digital twins in the fog computing environment. Within the scope of this article, we study live stateful stream processing migration and how to redistribute computational activity across cloud and fog nodes using Kafka middleware and its Stream DSL API.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.

Similar content being viewed by others

Notes

  1. https://www.docker.com/

  2. https://criu.org

  3. https://www.vmware.com/products/vcenter-server.html

  4. https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/manage/live-migration-overview

  5. https://ubuntu.com/blog/lxd-2-0-your-first-lxd-container

  6. https://kubernetes.io/

  7. https://openvz.org/

  8. https://github.com/checkpoint-restore/p.haul

  9. https://wiki.openvz.org/Virtuozzo_Storage

  10. http://spark.apache.org/docs/latest/streaming-programming-guide.html#checkpointing

  11. https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/checkpointing.html

  12. https://redis.io/

  13. https://storm.apache.org/

  14. http://activemq.apache.org/

  15. https://kafka.apache.org

  16. https://cloud.yandex.com/

  17. http://debs.org/grand-challenges/

  18. https://lenses.io/

REFERENCES

  1. Sensor Market Size, Share, Trends and Industry Analysis by 2025 | AMR. https://www.alliedmarketresearch.com/sensor-market. Accessed Nov. 30, 2019.

  2. Ericsson mobility report: on the pulse of the networked society, White Paper, Nov., 2015, pp. 1–36. https://doi.org/10.3103/S0005105510050031

  3. US Census Bureau world population, Population Clock World, 2019. https://www.census.gov/popclock/world. Accessed Nov. 30, 2019.

  4. Tchernykh, A., Babenko, M., Chervyakov, N., Miranda-Lopez, V., Avetisyan, A., Drozdov, A.Y., Rivera-Rodriguez, R., Radchenko, G., and Du, Z., Scalable data storage design for non-stationary IoT environment with adaptive security and reliability, IEEE Internet Things J., 2010, no. 61363019, p. 1. https://doi.org/10.1109/JIOT.2020.2981276

  5. Modoni, G.E., Sacco, M., and Terkaj, W., A telemetry-driven approach to simulate dataintensive manufacturing processes, Proc. CIRP, 2016, vol. 57, pp. 281–285. https://doi.org/10.1016/j.procir.2016.11.049

  6. Glaessgen, E.H. and Stargel, D.D.S., The digital twin paradigm for future NASA and U.S. air force vehicles, Proc. 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conf. – Special Session on the Digital Twin, Honolulu, Apr. 2012, pp. 1–14. https://doi.org/10.2514/6.2012-1818

  7. Grieves, M. and Vickers, J., Digital twin: mitigating unpredictable, undesirable emergent behavior in complex systems, in Transdisciplinary Perspectives on Complex Systems, Cham: Springer Int. Publ., 2017, pp. 85–113.

    Google Scholar 

  8. Radchenko, G., Alaasam, A., and Tchernykh, A., Micro-workflows: Kafka and Kepler fusion to support digital twins of industrial processes, Proc. 2018 IEEE/ACM Int. Conf. on Utility and Cloud Computing Companion (UCC Companion), Zurich, Dec. 2018, pp. 83–88. https://doi.org/10.1109/UCC-Companion.2018.00039

  9. Alaasam, A.B.A., Radchenko, G., and Tchernykh, A., Micro-workflows: a combination of workflows and data streaming to support digital twins of production processes, Bull. South Ural State Univ., Ser.: Comput. Math. Software Eng., 2019, vol. 8, no. 4, pp. 100–116. https://doi.org/10.14529/cmse190407

    Article  Google Scholar 

  10. Zhang, Q., Cheng, L., and Boutaba, R., Cloud computing: state-of-the-art and research challenges, J. Internet Serv. Appl., 2010, vol. 1, no. 1, pp. 7–18. https://doi.org/10.1007/s13174-010-0007-6

    Article  Google Scholar 

  11. Shabanov, B.M. and Samovarov, O.I., Building the software-defined data center, Program. Comput. Software, 2019, vol. 45, no. 8, pp. 458–466. https://doi.org/10.1134/S0361768819080048

    Article  Google Scholar 

  12. Kudryavtsev, A.O., Koshelev, V.K., and Avetisyan, A.I., Prospects for virtualization of highperformance x64 systems, Program. Comput. Software, 2013, vol. 39, no. 6, pp. 285–294. https://doi.org/10.1134/S0361768813060042

    Article  Google Scholar 

  13. Radchenko, G.I., Alaasam, A.B.A., and Tchernykh, A.N., Comparative analysis of virtualization methods in big data processing, Supercomput. Front. Innovations, 2019, vol. 6, no. 1, pp. 48–79. https://doi.org/10.14529/jsfi190107

    Article  Google Scholar 

  14. Scheepers, M.J., Virtualization and containerization of application infrastructure: a comparison, Proc. 21st Twente Student Conf. on IT, Twente, 2014, pp. 1–7.

  15. Luo, J., Yin, L., Hu, J., Wang, C., Liu, X., Fan, X., and Luo, H., Container-based fog computing architecture and energy-balancing scheduling algorithm for energy IoT, Future Gener. Comput. Syst., 2019, vol. 97, pp. 50–60. https://doi.org/10.1016/j.future.2018.12.063

    Article  Google Scholar 

  16. Hoque, S., De Brito, M.S., Willner, A., Keil, O., and Magedanz, T., Towards container orchestration in fog computing infrastructures, Proc. Int. Computer Software and Applications Conf., Torino, 2017, vol. 2, pp. 294–299. https://doi.org/10.1109/COMPSAC.2017.248

  17. Aazam, M., Zeadally, S., and Harras, K.A., Deploying fog computing in industrial internet of things and industry 4.0, IEEE Trans. Ind. Inf., 2018, vol. 14, no. 10, pp. 4674–4682. https://doi.org/10.1109/TII.2018.2855198

    Article  Google Scholar 

  18. Singh, S., Angrish, A., Barkley, J., Starly, B., Lee, Y.-S., and Cohen, P., Streaming machine generated data to enable a third-party ecosystem of digital manufacturing apps, Proc. Manuf., 2017, vol. 10, pp. 1020–1030. https://doi.org/10.1016/j.promfg.2017.07.093

    Article  Google Scholar 

  19. Qamsane, Y., Chen, C., Balta, E.C., Kao, B., Mohan, S., Moyne, J., Tilbury, D., and Barton, K., A unified digital twin framework for real-time monitoring and evaluation of smart manufacturing systems, Proc. 15th IEEE Int. Conf. on Automation Science and Engineering (CASE), Vancouver, Aug. 2019, pp. 1394–1401. https://doi.org/10.1109/COASE.2019.8843269

  20. Burgess, M., Locality, statefulness, and causality in distributed information systems concerning the scale dependence of system promises, Sep. 2019. http://arxiv.org/abs/1909.09357.

  21. Peiffer, C. and L’Heureux, I., US Patent US8346848B2, 2013.

  22. Naseri, M. and Towhidi, A., Stateful web services: a missing point in web service standards, Proc. Int. MultiConf. of Engineers and Computer Scientists, IMECS 2007, Hong Kong, 2007, pp. 993–997.

  23. Lichtenthaler, R., Model-driven software migration towards fine-grained cloud architectures, Proc. CEUR Workshop, Otzenhausen, Feb. 2019, vol. 2339, pp. 35–38.

  24. Newman, S., Building Microservices: Designing Fine-Grained System, O’Reilly Media, 2015.

    Google Scholar 

  25. Lewis, J. and Fowler, M., Microservices, 2014. https://martinfowler.com/articles/microservices.html. Accessed Jan. 11, 2019.

  26. Kratzke, N., About microservices, containers and their underestimated impact on network performance, Proc. 6th Int. Conf. on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING), Nice, 2015. http://arxiv.org/abs/1710.04049.

  27. The Microservice Revolution: Containerized Applications, Data and All. https://www.infoq.com/articles/microservicesrevolution/. Accessed Dec. 10, 2019.

  28. Balalaie, A., Heydarnoori, A., and Jamshidi, P., Microservices architecture enables DevOps: migration to a cloud-native architecture, IEEE Software, 2016, vol. 33, no. 3, pp. 42–52. https://doi.org/10.1109/MS.2016.64

    Article  Google Scholar 

  29. Mauro, T., Adopting Microservices at Netflix: Lessons for Architectural Design, 2015. https://www.nginx.com/blog/microservices-at-netflixarchitectural-best-practices/. Accessed Jan. 11, 2019.

  30. Fehling, C., Leymann, F., Retter, R., Schupeck, W., and Arbitter, P., Cloud Computing Patterns: Fundamentals to Design, Build, and Manage Cloud Applications, Vienna: Springer Vienna, 2014.

    Book  Google Scholar 

  31. Li, W. and Kanso, A., Comparing containers versus virtual machines for achieving high availability, Proc. IEEE Int. Conf. on Cloud Engineering, IC2E 2015, Tempe, AZ, 2015, pp. 353–358. https://doi.org/10.1109/IC2E.2015.79

  32. Gholami, M.F., Daneshgar, F., Low, G., and Beydoun, G., Cloud migration process – a survey, evaluation framework, and open challenges, J. Syst. Software, 2016, vol. 120, pp. 31–69. https://doi.org/10.1016/j.jss.2016.06.068

    Article  Google Scholar 

  33. Tilak, S., Hubbard, P., Miller, M., and Fountain, T., The ring buffer network bus (RBNB) data turbine streaming data middleware for environmental observing systems, Proc. 3rd IEEE Int. Conf. on e-Science and Grid Computing (e-Science 2007), Bangalore, 2007, pp. 12–13. https://doi.org/10.1109/ESCIENCE.2007.49

  34. Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., and Warfield, A., Live migration of virtual machines, Proc. 2nd Symp. Networked Systems Design & Implementation NSDI’05, Boston, MA, 2005, vol. 2, pp. 273–286.

  35. About storage drivers | Docker Documentation. https://docs.docker.com/storage/storagedriver/. Accessed Dec. 15, 2019.

  36. Docker checkpoint | Docker Documentation. https://docs.docker.com/engine/reference/commandline/checkpoint/. Accessed Dec. 11, 2019.

  37. Reber, A., CRIU and the PID dance, Proc. Linux Plumbers Conf., Lisbon, 2019, pp. 1–4.

  38. StatefulSets – Kubernetes. https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#limitations. Accessed Dec. 18, 2019.

  39. Abdollahi Vayghan, L., Saied, M.A., Toeroe, M., and Khendek, F., Microservice based architecture: towards high-availability for stateful applications with kubernetes, Proc. 19th IEEE Int. Conf. on Software Quality, Reliability and Security, QRS 2019, Sofia, 2019, pp. 176–185. https://doi.org/10.1109/QRS.2019.00034

  40. Loo, H., Yeo, A., Yip, K., and Liu, T., Live Pod Migration in Kubernetes, Vancouver. https://www.cs.ubc.ca/~bestchai/teaching/ cs416_2017w2/project2/project_m6r8_s8u8_v5v8_ y6x8_proposal.pdf.

  41. Ohtsuji, H. and Tatebe, O., Network-based data processing architecture for reliable and high-performance distributed storage system, Proc. European Conf. on Parallel Processing, Vienna 2015, pp. 16–26.

  42. Al-Osta, M., Bali, A., and Gherbi, A., Event driven and semantic based approach for data processing on IoT gateway devices, J. Ambient Intell. Human. Comput., 2019, vol. 10, no. 12, pp. 4663–4678. https://doi.org/10.1007/s12652-018-0843-y

    Article  Google Scholar 

  43. Alaasam, A.B.A., Radchenko, G., and Tchernykh, A., Stateful stream processing for digital twins: microservice-based Kafka stream DSL, Proc. 2019 Int. Multi-Conf. on Engineering, Computer and Information Sciences (SIBIRCON), Novosibirsk, Oct. 2019, pp. 0804–0809. https://doi.org/10.1109/SIBIRCON48586.2019.8958367

  44. AWS, What is Streaming Data?, 2019. https://aws.amazon.com/streaming-data/. Accessed Feb. 07, 2020.

  45. Raddaoui, A., Settle, A., Garbutt, J., and Singh, S., High availability of live migration, 2017. http://superuser.openstack.org/wp-content/uploads/2017/06/ha-livemigrate-whitepaper.pdf.

  46. Kuplyakov, D.A., Shalnov, E.V., Konushin, V.S., and Konushin, A.S., A distributed tracking algorithm for counting people in video, Program. Comput. Software, 2019, vol. 45, no. 4, pp. 163–170. https://doi.org/10.1134/S0361768819040042

    Article  Google Scholar 

  47. Carvalho, O., Roloff, E., and Navaux, P.O.A., A distributed stream processing based architecture for IoT smart grids monitoring, Proc. 10th Int. Conf. on Utility and Cloud Computing – UCC’17 Companion, Austin, TX, 2017, pp. 9–14. https://doi.org/10.1145/3147234.3148105

  48. Sunderrajan, A., Aydt, H., and Knoll, A., DEBS grand challenge: real time load prediction and outliers detection using STORM, Proc. 8th ACM Int. Conf. on Distributed Event-Based Systems DEBS’14, Montreal, 2014, pp. 294–297. https://dl.acm.org/citation.cfm?id=2611327.

  49. Trilles, S., Schade, S., Belmonte, O., and Huerta, J., Real-Time Anomaly Detection from Environmental Data Streams, Springer Int. Publ., 2015, pp. 125–144.

    Book  Google Scholar 

  50. Antonic, A., Pripuzic, K., Marjanovic, M., Skocir, P., Jezic, G., and Zarko, I.P., A high throughput processing engine for taxi-generated data streams, Proc. 9th ACM Int. Conf. on Distributed Event-Based Systems – DEBS’15, Oslo, 2015, pp. 309–315. https://doi.org/10.1145/2675743.2772588

  51. Reynold Xin, Apache spark officially sets a new record in large-scale sorting, 2014. https://databricks.com/blog/2014/11/05/sparkofficially-sets-a-new-record-in-large-scalesorting.html. Accessed May 09, 2020.

  52. Apache Kafka’s MirrorMaker – Confluent Platform. https://docs.confluent.io/4.0.0/multi-dc/mirrormaker.html. Accessed Apr. 15, 2020.

  53. Alaasam, A.B.A., Radchenko, G., Tchernykh, A., Borodulin, K., and Podkorytov, A., Scientific micro-workflows: where event-driven approach meets workflows to support digital twins, Proc. Int. Conf. RuSCDays’18 – Russian Supercomputing Days, Moscow, Sept. 24–25, 2018, vol. 1, pp. 489–495.

Download references

ACKNOWLEDGMENTS

The reported study was funded by RFBR, project no. 19-37-90073.

Author information

Authors and Affiliations

  1. South Ural State University, 454080, Chelyabinsk, Russia

    Ameer B. A. Alaasam, G. Radchenko & A. Tchernykh

  2. CICESE Research Center, Carr Tijuana–Ensenada 3918, Zona Playitas, 22860, Ensenada, B.C., Mexico

    A. Tchernykh

  3. Ivannikov Institute for System Programming, 109004, Moscow, Russia

    A. Tchernykh

  4. CINVESTAV Tamaulipas, Carretera Victoria- Soto la Marina Kilómetro 5.5, Ciudad Victoria – Soto la Marina, 87130, Cd Victoria, Tamps., Mexico

    J. L. González Compeán

Authors
  1. Ameer B. A. Alaasam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Radchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Tchernykh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. L. González Compeán
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ameer B. A. Alaasam, G. Radchenko, A. Tchernykh or J. L. González Compeán.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alaasam, A.B., Radchenko, G., Tchernykh, A. et al. Analytic Study of Containerizing Stateful Stream Processing as Microservice to Support Digital Twins in Fog Computing. Program Comput Soft 46, 511–525 (2020). https://doi.org/10.1134/S0361768820080083

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768820080083

Search

Navigation