Skip to main content

Assessing and Improving Sensors Data Quality in Streaming Context

  • Conference paper
  • First Online:
Computational Collective Intelligence (ICCCI 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10449))

Included in the following conference series:

Abstract

An environmental monitoring process consists of a regular collection and analysis of sensors data streams. It aims to infer new knowledge about the environment, enabling the explorer to supervise the network and to take right decisions. Different data mining techniques are then applied to the collected data in order to infer aggregated statistics useful for anomalies detection and forecasting. The obtained results are closely dependent on the collected data quality. In fact, the data are often dirty, they contain noisy, erroneous and missing values. Poor data quality leads to defective and faulty results. One solution to overcome this problem will be presented in this paper. It consists of evaluating and improving the data quality, to be able to obtain reliable results. In this paper, we first introduce the data quality concept. Then, we discuss the existing related research studies. Finally, we propose a complete sensors data quality management system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.waves-rsp.org.

References

  1. El Sibai, R., Chabchoub, Y., Demerjian, J., Kazi-Aoul, Z., Barbar, K.: A performance study of the chain sampling algorithm. In: Proceedings of the IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS), pp. 487–494. IEEE (2015)

    Google Scholar 

  2. El Sibai, R., Chabchoub, Y., Demerjian, J., Kazi-Aoul, Z., Barbar, K.: Sampling algorithms in data stream environments. In: Proceedings of the IEEE First International Conference on Digital Economy Emerging Technologies and Business Innovation (ICDEc), pp. 29–36. IEEE (2016)

    Google Scholar 

  3. Fan, W.: Data quality: from theory to practice. ACM SIGMOD Rec. 44(3), 7–18 (2015)

    Article  Google Scholar 

  4. Strong, D.M., Lee, Y.W., Wang, R.Y.: Data quality in context. Commun. ACM 40, 103–110 (1997)

    Article  Google Scholar 

  5. Wang, R.Y., Strong, D.M.: Beyond accuracy: what data quality means to data consumers. J. Manag. Inf. Syst. 12, 5–33 (1996)

    Article  Google Scholar 

  6. Jeffery, S.R., Alonso, G., Franklin, M.J., Hong, W., Widom, J.: Declarative support for sensor data cleaning. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) Pervasive 2006. LNCS, vol. 3968, pp. 83–100. Springer, Heidelberg (2006). doi:10.1007/11748625_6

    Chapter  Google Scholar 

  7. Lim, H.S., Moon, Y.S., Bertino, E.: Research issues in data provenance for streaming environments. In: Proceedings of the 2nd SIGSPATIAL ACM GIS 2009 International Workshop on Security and Privacy in GIS and LBS, pp. 58–62 (2009)

    Google Scholar 

  8. Rodriguez, C.G.: Qualité des données capteurs pour les systèmes de surveillance de phénomènes environnementaux. Ph.D. thesis, Villeurbanne, INSA (2010)

    Google Scholar 

  9. Rodriguez, C.G., Servigne, S.: Sensor data quality for geospatial monitoring applications. In: AGILE, 15th Internationale Conference on Geographic Information Science, pp. 1–6 (2012)

    Google Scholar 

  10. Ramirez, G., Fuentes, O., Tweedie, C.E.: Assessing data quality in a sensor network for environmental monitoring. In: Fuzzy Information Processing Society (NAFIPS), pp. 1–6. IEEE (2011)

    Google Scholar 

  11. Klein, A., Do, H.H., Hackenbroich, G., Karnstedt, M., Lehner, W.: Representing data quality for streaming and static data. In: 23rd International Conference on Data Engineering Workshop, pp. 3–10. IEEE (2007)

    Google Scholar 

  12. Klein, A., Lehner, W.: Representing data quality in sensor data streaming environments. J. Data Inf. Qual. (JDIQ) 1, 10–28 (2009)

    Google Scholar 

  13. Olbrich, S.: Warehousing and analyzing streaming data quality information. In: AMCIS, p. 159 (2010)

    Google Scholar 

  14. Smith, S.: Digital Signal Processing: A Practical Guide for Engineers and Scientists. Newnes, Oxford (2013)

    Google Scholar 

  15. Sharma, A.B., Golubchik, L., Govindan, R.: Sensor faults: detection methods and prevalence in real-world datasets. ACM Trans. Sens. Netw. (TOSN) 6(3), 23 (2010)

    Google Scholar 

  16. Abuaitah, G.R., Wang, B.: A taxonomy of sensor network anomalies and their detection approaches. In: Technological Breakthroughs in Modern Wireless Sensor Applications, pp. 172–206. IGI Global (2015)

    Google Scholar 

  17. Pang, Q., Wong, V.W.S.: Reliable data transport and congestion control in wireless sensor networks. Int. J. Sens. Netw. 3(1), 16–24 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rayane El Sibai .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

El Sibai, R., Chabchoub, Y., Chiky, R., Demerjian, J., Barbar, K. (2017). Assessing and Improving Sensors Data Quality in Streaming Context. In: Nguyen, N., Papadopoulos, G., Jędrzejowicz, P., Trawiński, B., Vossen, G. (eds) Computational Collective Intelligence. ICCCI 2017. Lecture Notes in Computer Science(), vol 10449. Springer, Cham. https://doi.org/10.1007/978-3-319-67077-5_57

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67077-5_57

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67076-8

  • Online ISBN: 978-3-319-67077-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics