Skip to main content
SpringerLink
Log in

Decision Based Model for Real-Time IoT Analysis Using Big Data and Machine Learning

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Use of internet of things (IoT) in different fields including smart cities, health care, manufacturing, and surveillance is growing rapidly, which results in massive amount of data generated by IoT devices. Real-time processing of large-scale data streams is one of the main challenges of IoT systems. Analyzing IoT data can help in providing better services, predicting trends and timely decision making for industries. The systematic structure of IoT data follows the pattern of big data. In this paper, a novel approach is proposed in which big data tools are used to perform real-time stream processing and analysis on IoT data. We have also applied Spark’s built-in support of the machine learning library in order to make real-time predictions. The efficiency of the proposed system is evaluated by conducting experiments and reporting results on the case scenario of IoT based weather station.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data and Code Availability

This manuscript is a presentation of my original research work and has not been published anywhere. Data and code can be available on request.

References

  1. Watson, H. J. (2014). Tutorial: Big data analytics: Concepts, technologies, and applications. CAIS, 34, 65.

    Article  Google Scholar 

  2. Laney, D. (2001). 3d Data management: Controlling data volume, velocity and variety. META Group Research Note, 6(70), 1.

    Google Scholar 

  3. Malek, Y. N., Kharbouch, A., El Khoukhi, H., Bakhouya, M., De Florio, V., El Ouadghiri, D., Latre, S., & Blondia, C. (2017). On the use of iot and big data technologies for real-time monitoring and data processing. Procedia Computer Science, 113, 429–434.

    Article  Google Scholar 

  4. Mukherjee, S., & Biswas, G. (2018). Networking for iot and applications using existing communication technology. Egyptian Informatics Journal, 19(2), 107–127.

    Article  Google Scholar 

  5. Yassine, A., Singh, S., Hossain, M. S., & Muhammad, G. (2019). Iot big data analytics for smart homes with fog and cloud computing. Future Generation Computer Systems, 91, 563–573.

    Article  Google Scholar 

  6. Yassine, A., Singh, S., & Alamri, A. (2017). Mining human activity patterns from smart home big data for health care applications. IEEE Access, 5, 13131–13141.

    Article  Google Scholar 

  7. Muhammad, G., Alhamid, M. F., Alsulaiman, M., & Gupta, B. (2018). Edge computing with cloud for voice disorder assessment and treatment. IEEE Communications Magazine, 56(4), 60–65.

    Article  Google Scholar 

  8. Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Hashem, I. A. T., Siddiqa, A., & Yaqoob, I. (2017). Big iot data analytics: Architecture, opportunities, and open research challenges. IEEE Access, 5, 5247–5261.

    Article  Google Scholar 

  9. Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018). High-order possibilistic c-means algorithms based on tensor decompositions for big data in iot. Information Fusion, 39, 72–80.

    Article  Google Scholar 

  10. Shadroo, S., & Rahmani, A. M. (2018). Systematic survey of big data and data mining in internet of things. Computer Networks, 139, 19–47.

    Article  Google Scholar 

  11. Mahdavinejad, M. S., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P., & Sheth, A. P. (2018). Machine learning for internet of things data analysis: A survey. Digital Communications and Networks, 4(3), 161–175.

    Article  Google Scholar 

  12. Kreps, J., Narkhede, N., Rao, J. et al. (2011). Kafka: A distributed messaging system for log processing. In Proceedings of the NetDB (pp. 1–7).

  13. Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2010). Spark: Cluster computing with working sets. HotCloud, 10(10–10), 95.

    Google Scholar 

  14. Lakshman, A., & Malik, P. (2010). Cassandra: A decentralized structured storage system. ACM SIGOPS Operating Systems Review, 44(2), 35–40.

    Article  Google Scholar 

  15. White, T. (2012). Hadoop: The definitive guide. New York: O’Reilly Media Inc.

    Google Scholar 

  16. Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., & Tzoumas, K. Apache ink: Stream and batch processing in a single engine. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 36(4), 28–38.

  17. Iqbal, M. H., & Soomro, T. R. (2015). Big data analysis: Apache storm perspective. International Journal of Computer Trends and Technology, 19(1), 9–14.

    Article  Google Scholar 

  18. Noghabi, S. A., Paramasivam, K., Pan, Y., Ramesh, N., Bringhurst, J., Gupta, I., & Campbell, R. H. (2017). Samza: Stateful scalable stream processing at linkedin. Proceedings of the VLDB Endowment, 10(12), 1634–1645.

    Article  Google Scholar 

  19. Buddhika, T., Pallickara, S. (2016). Neptune: Real time stream processing for internet of things and sensing environments. In 2016 IEEE international parallel and distributed processing symposium (IPDPS) (pp. 1143–1152). IEEE.

  20. Tonjes, R., Barnaghi, P., Ali, M., Mileo, A., Hauswirth, M., Ganz, F., Ganea, S., Kjrgaard, B., Kuemper, D., Nechifor, S., et al. (2014) Real time iot stream processing and large-scale data analytics for smart city applications. In Poster session, European conference on networks and communications, sn.

  21. Yasumoto, K., Yamaguchi, H., & Shigeno, H. (2016). Survey of real-time processing technologies of iot data streams. Journal of Information Processing, 24(2), 195–202.

    Article  Google Scholar 

  22. Nakamura, Y., Suwa, H., Arakawa, Y., Yamaguchi, H., Yasumoto, K., (2016). Design and implementation of middleware for iot devices toward real-time flow processing. In 2016 IEEE 36th international conference on distributed computing systems workshops (ICDCSW) (pp. 162–167). IEEE.

  23. Zamam, M. (2017). A unified framework for real-time streaming and processing of iot data. Master’s Thesis, Linnaeus University.

  24. Kho, D. D., Lee, S., & Zhong, R. Y. (2018). Big data analytics for processing time analysis in an iot-enabled manufacturing shop floor. Procedia Manufacturing, 26, 1411–1420.

    Article  Google Scholar 

  25. Rathore, M. M., Paul, A., Hong, W.-H., Seo, H., Awan, I., & Saeed, S. (2018). exploiting iot and big data analytics: Defining smart digital city using real-time urban data. Sustainable cities and society, 40, 600–610.

    Article  Google Scholar 

  26. Shah, S. K., Tariq, Z., & Lee, Y. (2018). Audio iot analytics for home automation safety. In 2018 IEEE international conference on big data (big data) (pp. 5181–5186). IEEE.

  27. Terroso-Saenz, F., Gonzalez-Vidal, A., Ramallo-Gonzalez, A. P., & Skarmeta, A. F. (2019). An open iot platform for the management and analysis of energy data. Future Generation Computer Systems, 92, 1066–1079.

    Article  Google Scholar 

  28. Nair, L. R., Shetty, S. D., & Shetty, S. D. (2018). applying spark-based machine learning model on streaming big data for health status prediction. Computers & Electrical Engineering, 65, 393–399.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Comsats University Islamabad, Wah Campus, Rawalpindi, Pakistan

    Hina Jamil

  2. Department of Computer Science, Comsats University Islamabad, Lahore Campus, Lahore, Pakistan

    Tariq Umer

  3. Sakarya University, Serdivan, Turkey

    Celal Ceken

  4. Antalya Bilim University, Antalya, Turkey

    Fadi Al-Turjman

Authors
  1. Hina Jamil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tariq Umer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Celal Ceken
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fadi Al-Turjman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hina Jamil.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jamil, H., Umer, T., Ceken, C. et al. Decision Based Model for Real-Time IoT Analysis Using Big Data and Machine Learning. Wireless Pers Commun 121, 2947–2959 (2021). https://doi.org/10.1007/s11277-021-08857-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-08857-7

Keywords

Search

Navigation