Skip to main content
SpringerLink
Log in

EDA-78: A Novel Error Detection Algorithm for Lempel-Ziv-78 Compressed Data

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

This paper presents an error detection algorithm for Lempel-Ziv-78 (LZ78) compressed data. LZ78 data compression involves dictionary coding and aims to compress the original data without loss. For detecting bit errors in compressed data, methods such as parity bit and Hamming code approaches have been applied. However, in these conventional methods, the insertion of additional bits is required for error detection, increasing the data redundancy. For error detection of LZ78 compressed data, we introduced four unique properties of LZ78 compressed data and developed an algorithm that detects bit errors in LZ78 compressed data according to these properties, without the insertion of additional bits. The proposed algorithm, which is called EDA-78 (Error Detection Algorithm for LZ78 compressed data), achieved an error detection rate of nearly 100% in the case of six or more bit errors. However, when the number of bit errors was smaller than six, the error detection rate was degraded. To overcome this drawback, we employed parity bits, significantly improving the error detection rate for a small number of bits.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. i is encoded using 4 bytes and c is encoded using 7 bits, according to the American Standard Code for Information Interchange (ASCII).

References

  1. Kwon, B., Gong, M., Huh, J., & Lee, S. (2018). Identification and restoration of LZ77 compressed data using a machine learning approach. In Proceeding Asia-Pacific signal and information processing association annual summit and conference (APSIPA ASC), pp. 1787–1790.

  2. Song, H., Kwon, B., Lee, S., & Lee, S. (2019). Dictionary based compression type classification using a CNN architecture. In Proceeding Asia-Pacific signal and information processing association annual summit and conference (APSIPA ASC), pp. 1–5.

  3. Kostina, V. (2017). Data compression with low distortion and finite blocklength. IEEE Transactions on Information Theory, 63(7), 4268–4285.

    Article  MathSciNet  Google Scholar 

  4. Movassagh, M., & Kabal, P. (2016). Scalable audio coding using trellis-based optimized joint entropy coding and quantization. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24(12), 2288–2300.

    Article  Google Scholar 

  5. Lee, H., & Lee, S. (2006). Visual entropy gain for wavelet image coding. IEEE Signal Processing Letters, 13(9), 553–556.

    Article  Google Scholar 

  6. Ha, H., Oh, T., & Lee, S. (2009). Macroblock-based frequency selective weighting for visual scalable video coding of H. 264/AVC. IEEE Transactions on Broadcasting, 55(3), 559–568.

    Article  Google Scholar 

  7. Schmidhuber, J., & Heil, S. (1996). Sequential neural text compression. IEEE Transactions on Neural Networks, 7(1), 142–146.

    Article  Google Scholar 

  8. Moffat, A., Zobel, J., & Sharman, N. (1997). Text compression for dynamic document databases. IEEE Transactions on Knowledge and Data Engineering, 9(2), 302–313.

    Article  Google Scholar 

  9. Abel, J., & Teahan, W. (2005). Universal text preprocessing for data compression. IEEE Transactions on Computers, 54(5), 497–507.

    Article  Google Scholar 

  10. Ziv, J., & Lempel, A. (1978). Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory, 24(5), 530–536.

    Article  MathSciNet  Google Scholar 

  11. Chi, C. H. (1998). Study on multi-lingual LZ77 and LZ78 text compression. In Proceeding IEEE data compression conference (DCC), p. 533.

  12. Bannai, H., Inenaga, S., & Takeda, M. (2012). Efficient LZ78 factorization of grammar compressed text. In Proceeding international symposium on string processing and information retrieval (SPIRE), pp. 86–98.

  13. Li, M., & Sleep, R. (2005). Genre classification via an LZ78-based string kernel. In Proceeding International Society for Music Information Retrieval (ISMIR), pp. 252–259.

  14. Bannai, H., Inenaga, S., & Takeda, M. (2006). Image classification via LZ78 based string kernel: A comparative study. In Proceeding Pacific-Asia conference on knowledge discovery and data mining (PAKDD), pp. 704–712.

  15. Moghaddam, A., & Kabir, E. (2009). Dynamic and memory efficient web page prediction model using LZ78 and LZW algorithms. In Proceeding IEEE international CSI computer conference (CSICC), pp. 676–681.

  16. Alam, M. R., Reaz, M. B. I., & Ali, M. M. (2012). SPEED: An inhabitant activity prediction algorithm for smart homes. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 42(4), 985–990.

    Article  Google Scholar 

  17. Das, S. K., & Cook, D. J. (2004). Health monitoring in an agent-based smart home by activity prediction. In Proceeding international conference on smart homes and health telematics (ICOST), pp. 3–14.

  18. Zhang, H., & Dong, Y. N. (2006). Mobility prediction model based link stability metric for wireless ad hoc networks. In Proceeding IEEE international conference on wireless communications, networking and mobile computing (WiCOM), pp. 1–4.

  19. Kwon, B., Park, J., & Lee, S. (2015). Virtual MIMO broadcasting transceiver design for multi-hop relay networks. Digital Signal Processing, 46, 97–107.

    Article  MathSciNet  Google Scholar 

  20. Kwon, B., Park, J., & Lee, S. (2015). A target position decision algorithm based on analysis of path departure for an autonomous path keeping system. Wireless Personal Communications, 83(3), 1843–1865.

    Article  Google Scholar 

  21. Kwon, B., Kim, S., Lee, H., & Lee, S. (2015). A downlink power control algorithm for long-term energy efficiency of small cell network. Wireless Networks, 21(7), 2223–223.

    Article  Google Scholar 

  22. Kwon, B., Kim, S., Jeon, D., & Lee, S. (2016). Iterative interference cancellation and channel estimation in evolved multimedia broadcast multicast system using filter-bank multicarrier-quadrature amplitude modulation. IEEE Transactions on Broadcasting, 62(4), 864–875.

    Article  Google Scholar 

  23. Kwon, B., Kim, S., & Lee, S. (2017). Scattered reference symbol-based channel estimation and equalization for FBMC-QAM systems. IEEE Transactions on Communications, 65(8), 3522–3537.

    Google Scholar 

  24. Kwon, B., & Lee, S. (2017). Effective interference nulling virtual MIMO broadcasting transceiver for multiple relaying. IEEE Access, 5, 20695–20706.

    Article  Google Scholar 

  25. Kwon, B., Kim, J., Lee, K., Lee, Y., Park, S., & Lee, S. (2017). Implementation of a virtual training simulator based on 360° multi-view human action recognition. IEEE Access, 5, 12496–12511.

    Article  Google Scholar 

  26. Kwon, B., & Lee, S. (2018). Cross-antenna interference cancellation and channel estimation for MISO-FBMC/QAM-based eMBMS. Wireless Networks, 24(8), 3281–3293.

    Article  Google Scholar 

  27. Hamming, R. W. (1950). Error detecting and error correcting codes. Bell Labs Technical Journal, 29(2), 147–160.

    Article  MathSciNet  Google Scholar 

  28. Ziv, J., & Lempel, A. (1997). A universal algorithm for sequential data compression. IEEE Transactions on Information Theory, 23(3), 337–343.

    Article  MathSciNet  Google Scholar 

  29. Welch, T. A. (1984). A technique for high-performance data compression. Computer, 6(17), 8–19.

    Article  Google Scholar 

  30. Storer, J. A., & Szymanski, T. G. (1982). Data compression via textual substitution. Journal of the ACM, 29(4), 928–951.

    Article  MathSciNet  Google Scholar 

  31. Lonardi, S., Szpankowski, W., & Ward, M. D. (2007). Error resilient LZ’77 data compression: Algorithms, analysis, and experiments. IEEE Transactions on Information Theory, 53(5), 1799–1813.

    Article  MathSciNet  Google Scholar 

  32. Korosec, T., & Tomazic, S. (2012). An adaptive-parity error-resilient LZ’77 compression algorithm. Informacije MIDEM—Journal of Microelectronics, Electronic Components and Materials, 42(1), 29–35.

    Google Scholar 

  33. Wu, Y., Lonardi, S., & Szpankowski, W. (2006). Error-resilient LZW data compression. In Proceeding IEEE data compression conference (DCC), pp. 193–202.

  34. Kitakami, M., & Kawasaki, T. (2009). Burst error recovery method for LZSS coding. IEICE Transactions on Information and Systems, 92(12), 2439–2444.

    Article  Google Scholar 

  35. Kwon, B., Gong, M., & Lee, S. (2017). Novel error detection algorithm for LZSS compressed data. IEEE Access, 5, 8940–8947.

    Article  Google Scholar 

  36. Bell, T., Witten, I. H., & Cleary, J. G. (1989). Modeling for text compression. ACM Computing Surveys, 21(4), 557–591.

    Article  Google Scholar 

  37. Arnold, R., & Bell, T. (1997). A corpus for the evaluation of lossless compression algorithms. In Proceeding IEEE data compression conference (DCC), pp. 201–210.

  38. Kwon, B., & Lee, S. (2019). Error detection algorithm for Lempel-Ziv-77 compressed data. Jornal of Communications and Networks, 21(2), 100–112.

    Article  Google Scholar 

  39. Mstafa, R. J., & Elleithy, K. M. (2014). A highly secure video steganography using Hamming code (7, 4). In Proceeding IEEE long island systems. Applications and technology conference (LISAT), pp. 1–6.

Download references

Acknowledgements

This work was supported by the research fund of Signal Intelligence Research Center supervised by Defense Acquisition Program Administration and Agency for Defense Development of Korea.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, Yonsei University, Seoul, South Korea

    Beom Kwon, Myongsik Gong & Sanghoon Lee

Authors
  1. Beom Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Myongsik Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sanghoon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanghoon Lee.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kwon, B., Gong, M. & Lee, S. EDA-78: A Novel Error Detection Algorithm for Lempel-Ziv-78 Compressed Data. Wireless Pers Commun 111, 2177–2189 (2020). https://doi.org/10.1007/s11277-019-06979-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-019-06979-7

Keywords

Search

Navigation