Skip to main content

Advertisement

SpringerLink
Log in

Latency and Energy-Awareness in Data Stream Processing for Edge Based IoT Systems

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

LE-STREAM is a framework for IoT data stream processing. Data processing in IoT is challenging due to its dynamic and heterogeneous nature, and the massive amount of generated data. Sensor data suffers from uncertainty and inconsistency issues, that can affect its accuracy. Several IoT applications are time sensitive, requiring fast data processing. Finally, as IoT devices are often battery powered, processing tasks must be performed in an energy-efficient way. Therefore, challenges in IoT data stream processing span three dimensions: accuracy, latency and energy; and LE-STREAM jointly addresses them. It leverages edge computing to bring the data processing closer to the data sources, thus minimizing latency. Adaptive sampling combined with data prediction model reduce the energy consumption of devices without compromising data accuracy. An active node selection schema improves the workload distribution among devices, also tackling the energy dimension by promoting a graceful degradation of device resources.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ababneh, N.: Evaluation of on/off scheduling protocols for ad hoc and sensor networks. In: 2010 IEEE International Conference on Wireless Communications, Networking and Information Security, pp. 419–423. https://doi.org/10.1109/WCINS.2010.5544122 (2010)

  2. Aggarwal, C.C.: Mining sensor data streams. In: Managing and mining sensor data, pp. 143–171. Springer US, Boston (2013)

  3. Al-Hoqani, N., Yang, S.H.: Adaptive sampling for wireless household water consumption monitoring. Procedia Eng. 119, 1356–1365 (2015). https://doi.org/10.1016/j.proeng.2015.08.980

    Article  Google Scholar 

  4. Anastasi, G., Conti, M., Di Francesco, M., Passarella, A.: Energy conservation in wireless sensor networks: a survey. Ad Hoc Netw. 7, 537–568 (2009). https://doi.org/10.1016/j.adhoc.2008.06.003

    Article  Google Scholar 

  5. Campello, R.J.G.B., Moulavi, D., Sander, J.: Density-based clustering based on hierarchical density estimates. In: Pei, J., Tseng, V.S., Cao, L., Motoda, H., Xu, G. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 160–172. Springer, Berlin (2013)

  6. Carlson, J.L.: Redis in Action. Manning publications co., USA (2013)

    Google Scholar 

  7. Catarinucci, L., de Donno, D., Mainetti, L., Palano, L., Patrono, L., Stefanizzi, M.L., Tarricone, L.: An iot-aware architecture for smart healthcare systems. IEEE Int. Things J. 2, 515–526 (2015). https://doi.org/10.1109/JIOT.2015.2417684

    Article  Google Scholar 

  8. Chai, T., Draxler, R.R.: Root mean square error (rmse) or mean absolute error (mae)? Geosci. Model Dev. Discuss. 7, 1525–1534 (2014). https://doi.org/10.5194/gmdd-7-1525-2014

    Article  Google Scholar 

  9. Chauhan, R., Kaur, H., Chang, V.: An optimized integrated framework of big data analytics managing security and privacy in healthcare data. Wirel. Pers. Commun. 117, 87–108 (2021). https://doi.org/10.1007/s11277-020-07040-8

    Article  Google Scholar 

  10. Dautov, R., Distefano, S.: Stream processing on clustered edge devices. IEEE Trans. Cloud Comput.:1–1, https://doi.org/10.1109/TCC.2020.2983402 (2020)

  11. Dautov, R., Distefano, S., Bruneo, D., Longo, F., Merlino, G., Puliafito, A.: Pushing intelligence to the edge with a stream processing architecture. In: 2017 IEEE International Conference on Internet of Things (Ithings) and IEEE Green Computing And Communications (Greencom) and IEEE Cyber, physical and social computing (CPSCom) and IEEE lta (Smartdata), pp. 792–799. https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData.2017.121 (2017)

  12. Dias de Assunção, M., Da Silva Veith, A, Buyya, R.: Distributed data stream processing and edge computing: a survey on resource elasticity and future directions. J. Netw. Comput. Appl. 103, 1–17 (2018). https://doi.org/10.1016/j.jnca.2017.12.001

    Article  Google Scholar 

  13. Dwyer, G., Aggarwal, S., Stouffer, J.: Flask: building python web services packt publishing (2017)

  14. Elmazi, D., Cuka, M., Ikeda, M., Barolli, L.: A fuzzy-based system for actor node selection in wsans considering load balancing of actors. In: Barolli, L., Leu, F.Y., Enokido, T., Chen, H.C. (eds.) Advances on broadband and wireless computing, communication and applications, pp. 97–109. Springer International Publishing, Cham (2019)

  15. Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.M.: The many faces of publish/subscribe. ACM Comput. Surv. 35, 114–131 (2003). https://doi.org/10.1145/857076.857078

    Article  Google Scholar 

  16. Gama, J.A., žliobaitundefined, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation, vol. 46 (2014)

  17. Garg, S., Singh, A., Kaur, K., Aujla, G.S., Batra, S., Kumar, N., Obaidat, M.S.: Edge computing-based security framework for big data analytics in vanets. IEEE Netw. 33, 72–81 (2019). https://doi.org/10.1109/MNET.2019.1800239

    Article  Google Scholar 

  18. Ghobaei-Arani, M., Souri, A., Rahmanian, A.A.: Resource management approaches in fog computing: a comprehensive review. J. Grid Comput.:1–42 (2019)

  19. Giouroukis, D., Dadiani, A., Traub, J., Zeuch, S., Markl, V.: A survey of adaptive sampling and filtering algorithms for the internet of things. In: Proceedings of the 14th ACM International Conference on Distributed and Event-based Systems, DEBS ’20, pp. 27–38. Association for computing machinery, New York. https://doi.org/10.1145/3401025.3403777 (2020)

  20. Gupta, M., Shum, L.V., Bodanese, E., Hailes, S.: Design and evaluation of an adaptive sampling strategy for a wireless air pollution sensor network. In: 2011 IEEE 36th Conference on Local Computer Networks, pp. 1003–1010, https://doi.org/10.1109/LCN.2011.6115154 (2011)

  21. Ha, S., Rhee, I., Xu, L.: Cubic: a new tcp-friendly high-speed tcp variant. SIGOPS Oper. Syst. Rev. 42, 64–74 (2008). https://doi.org/10.1145/1400097.1400105

    Article  Google Scholar 

  22. Henning, S., Hasselbring, W.: Theodolite: scalability benchmarking of distributed stream processing engines in microservice architectures. Big Data Res. 25, 100209 (2021). https://doi.org/10.1016/j.bdr.2021.100209

    Article  Google Scholar 

  23. Huedo, E., Montero, R.S., Moreno-Vozmediano, R., Vázquez, C., Holer, V., Llorente, I.M.: Opportunistic deployment of distributed edge clouds for latency-critical applications. J. Grid Comput. 19(1), 1–16 (2021)

    Article  Google Scholar 

  24. Hyndman, R., Athanasopoulos, G.: Forecasting: Principles and Practice. OTexts, Australia (2014)

    Google Scholar 

  25. Isah, H., Abughofa, T., Mahfuz, S., Ajerla, D., Zulkernine, F., Khan, S.: A survey of distributed data stream processing frameworks, vol. 7 (2019)

  26. Janjua, Z.H., Vecchio, M., Antonini, M., Antonelli, F.: Irese: an intelligent rare-event detection system using unsupervised learning on the iot edge. Eng. Appl. Artif. Intel. 84, 41–50 (2019). https://doi.org/10.1016/j.engappai.2019.05.011

    Article  Google Scholar 

  27. Karkouch, A., Mousannif, H., Al Moatassime, H., Noel, T.: Data quality in internet of things: a state-of-the-art survey. J. Netw. Comput. Appl. 73, 57–81 (2016). https://doi.org/10.1016/j.jnca.2016.08.002

    Article  Google Scholar 

  28. Kaup, F., Gottschling, P., Hausheer, D.: Powerpi: measuring and Modeling the Power Consumption of the Raspberry Pi. In: 39Th Annual IEEE conference on local computer networks, pp. 236–243, https://doi.org/10.1109/LCN.2014.6925777 (2014)

  29. Komisarek, M., Choraundefined, M., Kozik, R., Pawlicki, M.: Real-time stream processing tool for detecting suspicious network patterns using machine learning. In: Proceedings of the 15th international conference on availability, reliability and security, ARES ’20. ACM, New York, https://doi.org/10.1145/3407023.3409189 (2020)

  30. Kotb, Y., Al Ridhawi, I., Aloqaily, M., Baker, T., Jararweh, Y., Tawfik, H.: Cloud-based multi-agent cooperation for iot devices using workflow-nets. J. Grid Comput. 17(4), 625–650 (2019)

    Article  Google Scholar 

  31. Le Borgne, Y.A., Santini, S., Bontempi, G.: Adaptive model selection for time series prediction in wireless sensor networks. Signal Process. 87, 3010–3020 (2007). https://doi.org/10.1016/j.sigpro.2007.05.015

    Article  MATH  Google Scholar 

  32. Li, W., Santos, I., Delicato, F.C., Pires, P.F., Pirmez, L., Wei, W., Song, H., Zomaya, A., Khan, S.: System modelling and performance evaluation of a three-tier cloud of things. Futur. Gener. Comput. Syst. 70, 104–125 (2017). https://doi.org/10.1016/j.future.2016.06.019

    Article  Google Scholar 

  33. Liu, D., Yan, Z., Ding, W., Atiquzzaman, M.: A survey on secure data analytics in edge computing. IEEE Int. Things J. 6, 4946–4967 (2019). https://doi.org/10.1109/JIOT.2019.2897619

    Article  Google Scholar 

  34. Loria, M.P., Toja, M., Carchiolo, V., Malgeri, M.: An efficient real-time architecture for collecting Iot data. In: 2017 Federated conference on computer science and information systems (FedCSIS), pp. 1157–1166 (2017), https://doi.org/10.15439/2017F381

  35. Loukopoulos, T., Tziritas, N., Koziri, M., Stamoulis, G., Khan, S.U.: A pareto-efficient algorithm for data stream processing at network edges. In: 2018 IEEE international conference on cloud computing technology and science (Cloudcom), pp. 159–162 (2018)

  36. Mehmood, E., Anees, T.: Challenges and solutions for processing real-time big data stream: a systematic literature review, vol. 8 (2020)

  37. Monteiro, L.C., Delicato, F.C., Pirmez, L., Pires, P.F., Miceli, C.: Dpcas: data prediction with cubic adaptive sampling for wireless sensor networks. In: Au, M.H.A., Castiglione, A., Choo, K.K.R., Palmieri, F., Li, K.C. (eds.) Green, pervasive, and cloud computing, pp. 353–368. Springer International Publishing, Cham (2017)

  38. Nah, F. F. H.: A study on tolerable waiting time: how long are web users willing to wait? Behav. Inf. Technol. 23, 153–163 (2004)

    Article  Google Scholar 

  39. Oliveira, E., Delicato, F.C., da Rocha, A.R., Mattoso, M.: A real-time and energy-aware framework for data stream processing in the internet of things. In: Proceedings of the 6th international conference on internet of things, big data and security - vol. 1: IoTBDS, pp. 17–28. INSTICC, SciTePress, https://doi.org/10.5220/0010370100170028 (2021)

  40. Ounacer, S., TALHAOUI, M.A., Ardchir, S., Daif, A., Azouazi, M.: A new architecture for real time data stream processing. Int. J. Adv. Comput. Sci. Appl., vol. 8, https://doi.org/10.14569/IJACSA.2017.081106 (2017)

  41. Rahman, A., Jadoon, W., Khan, F.: Energy efficiency techniques in cloud computing. Int. J. Comput. Sci. Inf. Secur. 14, 317–323 (2016)

    Google Scholar 

  42. Richards, R.: Representational state transfer (Rest). In: Pro PHP XML and Web services, pp. 633–672. Apress, Berkeley, CA, https://doi.org/10.1007/978-1-4302-0139-7_17 (2006)

  43. Salkind, N.: Encyclopaedia of Research Design, vol. 1. Sage Publications, Oaks, CA (2010)

    Book  Google Scholar 

  44. Samizadeh Nikoui, T., Rahmani, A.M., Balador, A., Haj Seyyed Javadi, H.: Internet of things architecture challenges: a systematic review. Int. J. Commun. Syst. e4678, 34 (2021). https://doi.org/10.1002/dac.4678

    Article  Google Scholar 

  45. Sarkar, C., Rao, V.S., Venkatesha Prasad, R., Das, S.N., Misra, S., Vasilakos, A.: Vsf: an energy-efficient sensing framework using virtual sensors. IEEE Sensors J. 16, 5046–5059 (2016). https://doi.org/10.1109/JSEN.2016.2546839

    Article  Google Scholar 

  46. Savaglio, C., Fortino, G.: A simulation-driven methodology for iot data mining based on edge computing. ACM Trans. Internet Technol. (TOIT) 21(2), 1–22 (2021)

    Article  Google Scholar 

  47. Schubert, E., Sander, J., Ester, M., Kriegel, H.P., Xu, X.: Dbscan revisited, revisited: why and how you should (still) use dbscan. ACM Trans. Database Syst. 42, 1–21 (2017)

    Article  MathSciNet  Google Scholar 

  48. Shelby, Z., Hartke, K., Bormann, C.: The constrained application protocol (coap). RFC, p. 7252, https://doi.org/10.17487/RFC7252 (2014)

  49. Tanganelli, G., Vallati, C., Mingozzi, E.: Coapthon: easy development of coap-based iot applications with python. In: 2015 IEEE 2nd world forum on internet of things (WF-Iot), pp. 63–68, https://doi.org/10.1109/WF-IoT.2015.7389028 (2015)

  50. Trinh, H., Chemodanov, D., Yao, S., Lei, Q., Zhang, B., Gao, F., Calyam, P., Palaniappan, K.: Energy-Aware Mobile Edge Computing for Low-Latency Visual Data Processing. In: 2017 IEEE 5th International Conference on Future Internet of Things and Cloud (Ficloud), pp. 128–133 (2017)

  51. Tsai, C.W., Lai, C.F., Chiang, M.C., Yang, L.T.: Data mining for internet of things: a survey. IEEE Commun. Surv. Tutor. 16, 77–97 (2014)

    Article  Google Scholar 

  52. Tusa, F., Clayman, S.: The impact of encoding and transport for massive real-time iot data on edge resource consumption. J. Grid Comput. 19(3), 1–20 (2021)

    Article  Google Scholar 

  53. Vikash, M.L., Varma, S.: Performance evaluation of real-time stream processing systems for internet of things applications. Futur. Gener. Comput. Syst. 113, 207–217 (2020). https://doi.org/10.1016/j.future.2020.07.012

    Article  Google Scholar 

  54. Wang, T., Ke, H., Zheng, X., Wang, K., Sangaiah, A.K., Liu, A.: Big data cleaning based on mobile edge computing in industrial sensor-cloud. IEEE Trans. Industr. Inform. 16, 1321–1329 (2020). https://doi.org/10.1109/TII.2019.2938861

    Article  Google Scholar 

  55. Xhafa, F., Kilic, B., Krause, P.: Evaluation of iot stream processing at edge computing layer for semantic data enrichment. Future Gener. Comput. Syst. 105, 730–736 (2020). https://doi.org/10.1016/j.future.2019.12.031

    Article  Google Scholar 

  56. Xu, Y., Helal, A.: Scalable cloud–sensor architecture for the internet of things. IEEE Internet Things J. 3, 285–298 (2016). https://doi.org/10.1109/JIOT.2015.2455555

    Article  Google Scholar 

  57. Zhang, H., Chen, G., Ooi, B.C., Tan, K.L., Zhang, M.: In-memory big data management and processing: a survey. IEEE Trans. Knowl. Data Eng. 27, 1920–1948 (2015). https://doi.org/10.1109/TKDE.2015.2427795

    Article  Google Scholar 

Download references

Acknowledgements

This work has been partially funded by Fundação de Amparo à Pesquisa do Estado de São Paulo - FAPESP (grant 2015/24144-7), Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro - FAPERJ (grant 2017/233868) and CNPq (grant 434874/2018-3). Marta Mattoso and Flavia Delicato are CNPq Fellows.

Author information

Authors and Affiliations

  1. PESC/COPPE, Universidade Federal do Rio de Janeiro, Av. Horacio Macedo 2030, Cidade Universitária, Centro de Tecnologia, block H, room 319 Post office box: 68511, Rio de Janeiro, RJ, 21941-972, Brazil

    Egberto Oliveira

  2. Universidade Federal do Ceará, Fortaleza - CE, Brazil

    Atslands R da Rocha

  3. PESC/COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, RJ, Brazil

    Marta Mattoso

  4. Instituto de Computação, Universidade Federal Fluminense, Niterói, RJ, Brazil

    Flavia C Delicato

Authors
  1. Egberto Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Atslands R da Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marta Mattoso
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Flavia C Delicato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Egberto Oliveira.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oliveira, E., Rocha, A.R.d., Mattoso, M. et al. Latency and Energy-Awareness in Data Stream Processing for Edge Based IoT Systems. J Grid Computing 20, 27 (2022). https://doi.org/10.1007/s10723-022-09611-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-022-09611-4

Keywords

Search

Navigation