Skip to main content
Springer Nature Link
Log in

An outlier detection method based on the hidden Markov model and copula for wireless sensor networks

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Some of the collected data by wireless sensor networks (WSNs) often digress from the normal pattern of the set and are called the ‘outliers’ or ‘anomalies’. These outlier data are usually derived from an error that can be due to the constraints of sensor nodes such as problems in the processing unit, power supply, and component failure or an event occurring at the development site of sensor nodes, including the earthquake or flood, affecting the quality of the collected data and their reliability for decision making. In this work, a method is proposed for detecting outlier data in networks based on the hidden Markov model and the copula theory. Copula was used because of its ability to deal with multivariate sets and its unlimitedness in such a way that it required no assumption about data distribution, and Markov has been applied due to its good performance in forecasting. The results of evaluations on the actual data from the Intel laboratory represent the efficiency of the proposed method. Bases on the performance evaluations of the proposed method, in comparison with a relatively new work leading to good results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Zhang, Y., Meratnia, N., & Havinga, P. J. M. (2010). Outlier detection techniques for wireless sensor networks: A survey. IEEE Communications Surveys and Tutorials, 12(2), 159–170.

    Article  Google Scholar 

  2. Ayadi, A., et al. (2017). Outlier detection approaches for wireless sensor networks: A survey. Computer Networks, 129, 319–333.

    Article  Google Scholar 

  3. Rajasegarar, S., Leckie, C., & Palaniswami, M. (2008). Anomaly detection in wireless sensor networks. IEEE Wireless Communications, 15(4), 34–40.

    Article  Google Scholar 

  4. Hawkins, D. M. (1980). Identification of outliers (Vol. 11). London: Chapman and Hall.

    Book  Google Scholar 

  5. Liu, X., et al. (2020). Reinforcement learning-based multislot double-threshold spectrum sensing with Bayesian fusion for industrial big spectrum data. IEEE Transactions on Industrial Informatics, 17(5), 3391–3400.

    Article  Google Scholar 

  6. Liu, X., et al. (2020). Big-data-based intelligent spectrum sensing for heterogeneous spectrum communications in 5G. IEEE Wireless Communications, 27(5), 67–73.

    Article  Google Scholar 

  7. Warriach, E. U., et al. (2012). Notice of violation of IEEE publication principles: a hybrid fault detection approach for context-aware wireless sensor networks. In 2012 IEEE 9th international conference on mobile ad-hoc and sensor systems (MASS 2012). IEEE.

  8. Chandore, P. R., & Chatur, D. P. N. (2013). Hybrid approach for outlier detection over wireless sensor network real time data. International Journal of Computer Science and Applications, 6(2), 76–81.

    Google Scholar 

  9. Kannan, K., Manoj, K., & Sakthivel, E. (2015). A comparative study on nearest-neighbor based outlier detection in data mining. A J Manag. NISMA Noorul Islam Strateg. Manag. Ambience, 1, 203–204.

    Google Scholar 

  10. Ghorbel O., Abid, M., & Snoussi, H. (2014). Improved KPCA for outlier detection in Wireless Sensor Networks. In 2014 1st international conference on advanced technologies for signal and image processing (ATSIP). IEEE.

  11. Ghorbel, O., et al. (2015). Fast and efficient outlier detection method in wireless sensor networks. IEEE Sensors Journal, 15(6), 3403–3411.

    Article  Google Scholar 

  12. Abid, A., Kachouri, A., & Mahfoudhi, A. (2016). Anomaly detection through outlier and neighborhood data in Wireless Sensor Networks. In 2016 2nd international conference on advanced technologies for signal and image processing (ATSIP). IEEE.

  13. Zhang, Y., et al. (2012). Statistics-based outlier detection for wireless sensor networks. International Journal of Geographical Information Science, 26(8), 1373–1392.

    Article  Google Scholar 

  14. Shuai, M., et al. (2008). A Kalman filter based approach for outlier detection in sensor networks. In 2008 international conference on computer science and software engineering. Vol. 4. IEEE.‏

  15. Samparthi, V. S. K., & Verma, H. K. (2010). Outlier detection of data in wireless sensor networks using kernel density estimation. International Journal of Computer Applications, 5(7), 28–32.

    Article  Google Scholar 

  16. Fawzy, A., Mokhtar, H. M. O., & Hegazy, O. (2013). Outliers detection and classification in wireless sensor networks. Egyptian Informatics Journal, 14(2), 157–164.

    Article  Google Scholar 

  17. Abid, A., et al. (2017). Outlier detection in wireless sensor networks based on OPTICS method for events and errors identification. Wireless Personal Communications, 97(1), 1503–1515.

    Article  Google Scholar 

  18. Chatzigiannakis, V., et al. (2006). Hierarchical anomaly detection in distributed large-scale sensor networks. In 11th IEEE symposium on computers and communications (ISCC'06). IEEE, 2006.

  19. Chen, J., Kher, S., & Somani, A. (2006). Distributed fault detection of wireless sensor networks. In Proceedings of the 2006 workshop on Dependability issues in wireless ad hoc networks and sensor networks‏.

  20. Wu, W., et al. (2007). Localized outlying and boundary data detection in sensor networks. IEEE Transactions on Knowledge and Data Engineering, 19(8), 1145–1157.

    Article  Google Scholar 

  21. Khan, S. A., Daachi, B., & Djouani, K. (2012). Application of fuzzy inference systems to detection of faults in wireless sensor networks. Neurocomputing, 94, 111–120.

    Article  Google Scholar 

  22. Saihi, M., et al. (2018). Hidden Gaussian Markov model for distributed fault detection in wireless sensor networks. Transactions of the Institute of Measurement and Control, 40(6), 1788–1798.

    Article  Google Scholar 

  23. Ayadi, A., et al. (2019). Kernelized technique for outliers detection to monitoring water pipeline based on WSNs. Computer Networks, 150, 179–189.

    Article  Google Scholar 

  24. Bharti, S., Pattanaik, K. K., & Pandey, A. (2019). Contextual outlier detection for wireless sensor networks. Journal of Ambient Intelligence and Humanized Computing, 11, 1511–1530.

    Article  Google Scholar 

  25. Ghalem, S. K., et al. (2019). A probabilistic multivariate Copula-based technique for faulty node diagnosis in Wireless Sensor Networks. Journal of Network and Computer Applications, 127, 9–25.

    Article  Google Scholar 

  26. Dai, T., & Ding, Z. (2020). Online distributed distance-based outlier clearance approaches for wireless sensor networks. Pervasive and Mobile Computing, 63, 101130.

    Article  Google Scholar 

  27. Lapuyade-Lahorgue, J., Xue, J.-H., & Ruan, Su. (2017). Segmenting multi-source images using hidden Markov fields with copula-based multivariate statistical distributions. IEEE Transactions on Image Processing, 26(7), 3187–3195.

    Article  MathSciNet  Google Scholar 

  28. Derrode, S., & Pieczynski, W. (2013). Unsupervised data classification using pairwise Markov chains with automatic copulas selection. Computational Statistics & Data Analysis, 63, 81–98.

    Article  MathSciNet  Google Scholar 

  29. Flitti, F., Collet, C., & Joannic-Chardin, A. (2005). Unsupervised multiband image segmentation using hidden Markov quadtree and copulas. In IEEE international conference on image processing 2005. VOL. 2. IEEE.

  30. Ötting, M., Langrock, R., & Maruotti, A. (2021). A copula-based multivariate hidden Markov model for modelling momentum in football. AStA Advances in Statistical Analysis. https://doi.org/10.1007/s10182-021-00395-8

    Article  Google Scholar 

  31. Härdle, W. K., Okhrin, O., & Wang, W. (2015). Hidden Markov structures for dynamic copulae. Econometric Theory, 31(5), 981–1015.

    Article  MathSciNet  Google Scholar 

  32. Brunel, N., & Pieczynski, W. (2005). Unsupervised signal restoration using hidden Markov chains with copulas. Signal processing, 85(12), 2304–2315.

    Article  Google Scholar 

  33. Lanchantin, P., Lapuyade-Lahorgue, J., & Pieczynski, W. (2011). Unsupervised segmentation of randomly switching data hidden with non-Gaussian correlated noise. Signal Processing, 91(2), 163–175.

    Article  Google Scholar 

  34. Martino, A., Guatteri, G., & Paganoni, A. M. (2020). Multivariate hidden Markov models for disease progression. Statistical Analysis and Data Mining: The ASA Data Science Journal, 13(5), 499–507.

    Article  MathSciNet  Google Scholar 

  35. Zhu, S., et al. (2020). Improved hidden Markov model incorporated with copula for probabilistic seasonal drought forecasting. Journal of Hydrologic Engineering, 25(6), 04020019.

    Article  Google Scholar 

  36. Nelsen, R. B., et al. (2013). QUASI-COPULAS. Distributions with Given Marginals and Statistical Modelling, 179–185.

  37. Aas, K., et al. (2009). Pair-copula constructions of multiple dependence. Insurance Mathematics and economics, 44(2), 182–198.

    Article  MathSciNet  Google Scholar 

  38. Czado, C., Brechmann, E. C., & Gruber, L. (2013). Selection of vine copulas Copulae in mathematical and quantitative finance (pp. 17–37). Berlin: Springer.

    Book  Google Scholar 

  39. Griffing, A. (2006). Solving the running key cipher with the Viterbi algorithm. Cryptologia, 30(4), 361–367.

    Article  Google Scholar 

  40. Alhaidari, S., & Zohdy, M. (2019). Network anomaly detection using two-dimensional hidden markov model based viterbi algorithm. In 2019 IEEE International Conference on Artificial Intelligence Testing (AITest). IEEE.

  41. http://db.csail.mit.edu/labdata/labdata.html

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Engineering, Shahid Rajaee Teacher Training University, Tehran, Iran

    Sina Dogmechi, Zeinab Torabi & Negin Daneshpour

Authors
  1. Sina Dogmechi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zeinab Torabi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Negin Daneshpour
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Zeinab Torabi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dogmechi, S., Torabi, Z. & Daneshpour, N. An outlier detection method based on the hidden Markov model and copula for wireless sensor networks. Wireless Netw 30, 4797–4810 (2024). https://doi.org/10.1007/s11276-022-03131-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-022-03131-5

Keywords