Skip to main content

Advertisement

Springer Nature Link
Log in

SZ4IoT: an adaptive lightweight lossy compression algorithm for diverse IoT devices and data types

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The Internet of Things (IoT) is an essential platform for industrial applications since it enables massive systems connecting many IoT devices for analytical data collection. This attribute is responsible for the exponential development in the amount of data created by IoT devices. IoT devices can generate voluminous amounts of data, which may place extraordinary demands on their limited resources, data transfer bandwidths, and cloud storage. Using lightweight IoT data compression techniques is a practical way to deal with these problems. This paper presents adaptable lightweight SZ lossy compression algorithm for IoT devices (SZ4IoT), a lightweight and adjusted version of the SZ lossy compression method. The SZ4IoT is a local (non-distributed) and interpolation-based compressor that can accommodate any sensor data type and can be implemented on microcontrollers with low resources. It operates on univariate and multivariate time series. It was implemented and tested on various devices, including the ESP32, Teensy 4.0, and RP2040, and evaluated on multiple datasets. The experiments of this paper focus on the compression ratio, compression and decompression time, normalized root mean square error (NRMSE), and energy consumption and prove the effectiveness of the proposed approach. The compression ratio outperforms LTC, WQT RLE, and K RLE by two, three, and two times, respectively. The proposed SZ4IoT decreased the consumed energy for the data size 40 KB by 31.4, 29.4, and 27.3% compared with K RLE, LTC, and WQT RLE, respectively. In addition, this paper investigates the impact of stationary versus non-stationary time series datasets on the compression ratio.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

Not applicable.

Notes

  1. https://szcompressor.org/

References

  1. Biljana L, Stojkoska R, Trivodaliev KV (2017) A review of internet of things for smart home: challenges and solutions. J Clean Product 140:1454–1464

    Article  Google Scholar 

  2. Stojkoska BR, Nikolovski Z (2017) Data compression for energy efficient iot solutions. In: 2017 25th telecommunication forum (TELFOR), pp 1–4. IEEE

  3. Pattnaik SK, Samal SR, Bandopadhaya S, Swain K, Choudhury S, Das JK, Mihovska A, Poulkov V (2022) Future wireless communication technology towards 6g iot: An application-based analysis of iot in real-time location monitoring of employees inside underground mines by using ble. Sensors 22(9):3438

    Article  Google Scholar 

  4. Idrees SK, Idrees AK (2022) New fog computing enabled lossless eeg data compression scheme in iot networks. J Ambient Intell Humaniz Comput 13(6):3257–3270

    Article  MATH  Google Scholar 

  5. Idrees AK, Idrees SK, Couturier R, Ali-Yahiya T (2022) An edge-fog computing-enabled lossless eeg data compression with epileptic seizure detection in iomt networks. IEEE Internet Things J 9(15):13327–13337

    Article  MATH  Google Scholar 

  6. Alhussein Duaa A, Idrees Ali K, Harb H (2021) Energy-saving adaptive sampling mechanism for patient health monitoring based iot networks. In: New Trends in Information and Communications Technology Applications: 5th International Conference, NTICT 2021, Baghdad, Iraq, Nov 17–18, 2021, Proceedings 5, pp 163–175. Springer

  7. Idrees Ali K, Ali-Yahiya T, Idrees Sara K, Couturier R (2022) Energy-efficient fog computing-enabled data transmission protocol in tactile internet-based applications. In: Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing, pp 206–209

  8. Hussein AM, Idrees AK, Couturier R (2022) Distributed energy-efficient data reduction approach based on prediction and compression to reduce data transmission in iot networks. Int J Commun Syst 35(15):e5282

    Article  Google Scholar 

  9. Idrees Ali K, Jaoude CA, Al-Qurabat AKM (2020) Data Reduction and Cleaning Approach for Energy-Saving in Wireless Sensors Networks of iot. In: 2020 16th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp 1–6. IEEE

  10. Al-Nassrawy KK, Idrees AK, Al-Shammary D (2022) A novel lossless eeg compression model using fractal combined with fixed-length encoding technique. In: AI and IoT for Sustainable Development in Emerging Countries, pp 439–454. Springer

  11. Azar J, Makhoul A, Barhamgi M, Couturier R (2019) An energy efficient iot data compression approach for edge machine learning. Futur Gener Comput Syst 96:168–175

    Article  Google Scholar 

  12. Idrees Sara K, Azar J, Couturier R, Idrees AK, Gechter F (2022) Lightweight sz4iot library. https://github.com/saraidrees/SZ4IoT

  13. Del Testa D, Rossi M (2015) Lightweight lossy compression of biometric patterns via denoising autoencoders. IEEE Signal Process Lett 22(12):2304–2308

    Article  MATH  Google Scholar 

  14. Liu T, Wang J, Liu Q, Alibhai S, Lu T, He X (2021) High-ratio lossy compression: exploring the autoencoder to compress scientific data. IEEE Trans Big Data

  15. Lindstrom P (2014) Fixed-rate compressed floating-point arrays. IEEE Trans Visual Comput Graphics 20(12):2674–2683

    Article  MATH  Google Scholar 

  16. da Silva, Marcus de VD, Rocha A, Gomes RL, Nogueira M (2021) Lightweight Data Compression for Low Energy Consumption in Industrial Internet of Things. In: 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC), pp 1–2. IEEE

  17. Capo-Chichi EP, Guyennet H, Friedt J-M (2009) K-rle: A New Data Compression Algorithm for Wireless Sensor Network. In: 2009 third international conference on sensor technologies and applications, pp 502–507. IEEE

  18. Lu T, Xia W, Zou X, Xia Q (2020) Adaptively compressing iot data on the resource-constrained edge. In HotEdge

  19. Moon A, Kim J, Zhang J, Son SW (2017) Lossy Compression on iot Big Data by Exploiting Spatiotemporal Correlation. In: 2017 IEEE High Performance Extreme Computing Conference (HPEC), pp 1–7. IEEE

  20. Correa JDA, Pinto ASR, Montez C (2022) Lossy data compression for iot sensors: a review. Internet of Things 19:100516

    Article  MATH  Google Scholar 

  21. Wang C-M, Yen C-C, Yang W-Y, Wang J-S (2016) Tree-Structured Linear Approximation for Data Compression Over wsns. In: 2016 International Conference on Distributed Computing in Sensor Systems (DCOSS), pp 43–51. IEEE

  22. Al Fallah S, Arioua M, El Oualkadi A, El Asri J (2018) On the Performance of Piecewise Linear Approximation Techniques in wsns. In: 2018 International Conference on Advanced Communication Technologies and Networking (CommNet), pp 1–6. IEEE

  23. Pham Ngoc D, Le TD, Choo H (2008) Enhance exploring temporal correlation for data collection in wsns. In: 2008 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies, pp 204–208. IEEE

  24. Schoellhammer T, Greenstein B, Osterweil E, Wimbrow M, Estrin D (2004) Lightweight Temporal Compression of Microclimate Datasets. In: 29th Annual IEEE Conference on Local Computer Networks( LCN), pp 516-524

  25. Parker D, Stojanovic M, Yu C (2013) Exploiting Temporal and Spatial Correlation in Wireless Sensor Networks. In: 2013 Asilomar Conference on Signals, Systems and Computers, pp 442–446. IEEE

  26. Sharma R (2015) A Data Compression Application for Wireless Sensor Networks Using ltc Algorithm. In: 2015 IEEE International Conference on Electro/Information Technology (EIT), pp 598–604. IEEE

  27. Klus L, Klus R, Lohan ES, Granell C, Talvitie J, Valkama M, Nurmi J (2021) Direct lightweight temporal compression for wearable sensor data. IEEE Sens Lett 5(2):1–4

    Article  MATH  Google Scholar 

  28. Di S, Cappello F (2016) Fast Error-Bounded Lossy hpc Data Compression with sz. In: 2016 IEEE international parallel and distributed processing symposium (ipdps), pp 730–739. IEEE

  29. Azar J, Makhoul A, Couturier R, Demerjian J (2020) Robust iot time series classification with data compression and deep learning. Neurocomputing 398:222–234

    Article  MATH  Google Scholar 

  30. Azar J, Tayeh GB, Makhoul A (2022) Raphaël Couturier. Efficient lossy compression for iot using sz and reconstruction with 1d u-net. Mobile Networks and Applications, pp 1–13

  31. Hussein AM, Idrees AK, Couturier R (2023) A distributed prediction-compression-based mechanism for energy saving in iot networks. J Supercomput 79(15):16963–16999

    Article  MATH  Google Scholar 

  32. Idrees AK, Jawad LW (2023) Energy-efficient data processing protocol in edge-based iot networks. Ann Telecommun 78(5):347–362

    Article  MATH  Google Scholar 

  33. Khlief MS, Idrees AK (2022) Efficient EEG Data Compression Technique for Internet of Health Things Networks. In: 2022 IEEE world conference on applied intelligence and computing (AIC), pp 403–409. IEEE

  34. Lin S, Lin W, Keyi W, Wang S, Minxian X, Wang JZ (2024) Cocv: a compression algorithm for time-series data with continuous constant values in iot-based monitoring systems. Internet Things 25:101049

    Article  MATH  Google Scholar 

  35. Hasan BT, Idrees AK (2023) Edge computing for iot. In: Learning Techniques for the Internet of Things, pp 1–20. Springer

  36. Kant K, Jolfaei A, Moessner K (2024) Iot systems for extreme environments. IEEE Internet Things J 11(3):3671–3675

    Article  Google Scholar 

  37. Idrees AK, Ali-Yahiya T, Idrees SK, Couturier R (2024) Edatad: energy-aware data transmission approach with decision-making for fog computing-based iot applications. J Netw Syst Manage 32(3):54

    Article  MATH  Google Scholar 

  38. Sadri AA, Rahmani AM, Saberikamarposhti M, Hosseinzadeh M (2022) Data reduction in fog computing and internet of things: a systematic literature survey. Internet Things 20:100629

    Article  MATH  Google Scholar 

  39. Idrees AK, Khlief MS (2023) Efficient compression technique for reducing transmitted eeg data without loss in iomt networks based on fog computing. J Supercomput 79(8):9047–9072

    Article  MATH  Google Scholar 

  40. Espressif Systems (2023) Esp32-wroom-32 datasheet. https://www.espressif.com/sites/default/files/documentation/esp32-wroom-32_datasheet_en.pdf

  41. PJRC. Teensy 4.0 development board. https://www.pjrc.com/store/teensy40.html

  42. Raspberry Pi. Raspberry pi pico series documentation. https://www.raspberrypi.com/documentation/microcontrollers/pico-series.html

  43. Andrzejak RG, Lehnertz K, Mormann F, Rieke C, David P, Elger CE (2001) Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: dependence on recording region and brain state. Phys Rev E 64(6):061907

    Article  MATH  Google Scholar 

  44. Cesarelli M, Ruffo M, Romano M, Bifulco P (2012) Simulation of foetal phonocardiographic recordings for testing of fhr extraction algorithms. Comput Methods Programs Biomed 107(3):513–523

    Article  MATH  Google Scholar 

  45. Saeed M, Villarroel M, Reisner AT, Clifford G, Lehman L-W, Moody G, Heldt T, Kyaw TH, Moody B, Mark RG (2011) Multiparameter intelligent monitoring in intensive care ii (mimic-ii): a public-access intensive care unit database. Crit Care Med 39(5):952

    Article  Google Scholar 

  46. Dau HA, Bagnall A, Kamgar K, Yeh C-CM, Zhu Y, Gharghabi S, Ratanamahatana CA, Keogh E (2019) The ucr time series archive. IEEE/CAA J Autom Sinica 6(6):1293–1305

    Article  MATH  Google Scholar 

  47. Mitzenmacher M (2024) Introduction to data compression, 2024. Accessed: 2024-09-15

Download references

Acknowledgements

This work has been supported by the EIPHI Graduate school (contract "ANR-17-EURE-0002").

Funding

Not applicable.

Author information

Authors and Affiliations

  1. CIAD (UMR 7533), Université de Technologie de Belfort Montbéliard, UTBM, Belfort, France

    Sara Kadhum Idrees & Franck Gechter

  2. FEMTO-ST Institute, UMR 6174 CNRS, University of Franche-Comté, Besançon, France

    Joseph Azar & Raphaël Couturier

  3. Department of Information Networks, College of Information Technology, University of Babylon, Babylon, Iraq

    Sara Kadhum Idrees & Ali Kadhum Idrees

  4. Smart Edge Lab, Faculty of Computer Science, University of Applied Science and Arts, Dortmund, Germany

    Ali Kadhum Idrees

Authors
  1. Sara Kadhum Idrees
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Joseph Azar
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Raphaël Couturier
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ali Kadhum Idrees
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Franck Gechter
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Joseph Azar.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Ethical approval

Not applicable.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the authors used ChatGPT in order to improve readability and language. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kadhum Idrees, S., Azar, J., Couturier, R. et al. SZ4IoT: an adaptive lightweight lossy compression algorithm for diverse IoT devices and data types. J Supercomput 81, 392 (2025). https://doi.org/10.1007/s11227-024-06667-9

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06667-9

Keywords