Skip to main content
SpringerLink
Log in

Data Science Challenges to Improve Quality Assurance of Internet of Things Applications

  • Conference paper
  • First Online:
Book cover Leveraging Applications of Formal Methods, Verification and Validation: Discussion, Dissemination, Applications (ISoLA 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9953))

Included in the following conference series:

Abstract

With the increasing importance and complexity of Internet of Things (IoT) applications, also the development of adequate quality assurance techniques becomes essential. Due to the massive amount of data generated in workflows of IoT applications, data science plays a key role in their quality assurance. In this paper, we present respective data science challenges to improve quality assurance of Internet of Things applications. Based on an informal literature review, we first outline quality assurance requirements evolving with the IoT grouped into six categories (Environment, User, Compliance/Service Level Agreement, Organizational, Security and Data Management). Finally, we present data science challenges to improve the quality assurance of Internet of Things applications sub-divided into four categories (Defect prevention, Defect analysis, User incorporation and Organizational) derived from the six quality assurance requirement categories.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Santucci, G.: The Internet of Things: between the revolution of the internet and the metamorphosis of objects. In: Sundmaeker, H., Guillemin, P., Friess, P., Woelfflé, S. (eds.) Vision and Challenges for Realising the Internet of Things, pp. 11–24. CERP-IoT – Cluster of European Research Projects on the Internet of Things, Luxembourg (2010)

    Google Scholar 

  2. Lee, I., Lee, K.: The Internet of Things (IoT): applications, investments, and challenges for enterprises. Bus. Horiz. 58, 431–440 (2015)

    Article  Google Scholar 

  3. Marwah, Q.M., Sirshar, M.: Software quality assurance in Internet of Things. Int. J. Comput. Appl. 109, 16–24 (2015)

    Google Scholar 

  4. Gubbi, J., Buyya, R., Marusic, S., Palaniswami, M.: Internet of Things (IoT): a vision, architectural elements, and future directions. Future Gener. Comput. Syst. 29, 1645–1660 (2013)

    Article  Google Scholar 

  5. Xia, F., Yang, L.T., Wang, L., Vinel, A.: Internet of Things. Int. J. Commun Syst 25, 1101–1102 (2012)

    Article  Google Scholar 

  6. Prasad, N.R., Eisenhauer, M., Ahlsén, M., Badii, A., Brinkmann, A., Hansen, K.M., Rosengren, P.: Open source middleware for networked embedded systems towards future Internet of Things. In: Sundmaeker, H., Guillemin, P., Friess, P., Woelfflé, S. (eds.) Vision and Challenges for Realising the Internet of Things, pp. 153–163. CERP-IoT – Cluster of European Research Projects on the Internet of Things, Luxembourg (2010)

    Google Scholar 

  7. Cai, L., Zhu, Y.: The challenges of data quality and data quality assessment in the Big Data Era. Data Sci. J. 14, 1–10 (2015)

    Article  Google Scholar 

  8. Zhu, Y., Xiong, Y.: Towards data science. Data Sci. J. 14, 1–7 (2015)

    Article  Google Scholar 

  9. Katal, A., Wazid, M., Goudar, R.H.: Big Data: issues, challenges, tools and good practices. In: Sixth International Conference on Contemporary Computing (IC3 2013), pp. 404–409. IEEE (2013)

    Google Scholar 

  10. IEEE: IEEE Standard for Software Quality Assurance Processes, vol. 730™-2014 (Revision of IEEE Std. 730-2002). IEEE Computer Society, New York (2014)

    Google Scholar 

  11. May, T.: The New Know: Innovation Powered by Analytics. Wiley, New Jersey (2009)

    Google Scholar 

  12. Provost, F., Fawcett, T.: Data science and its relationship to Big Data and data-driven decision making. Big Data 1, 51–59 (2013)

    Article  Google Scholar 

  13. Taylor, Q., Giraud-Carrier, C.: Applications of data mining in software engineering. Int. J. Data Anal. Tech. Strat. 2, 243–257 (2010)

    Article  Google Scholar 

  14. Xie, T., Pei, J., Hassan, A.E.: Mining software engineering data. In: 29th International Conference on Software Engineering (ICSE 2007 Companion), pp. 172–173. IEEE, Minneapolis (2007)

    Google Scholar 

  15. Hassan, A.E., Xie, T.: Software intelligence: the future of mining software engineering data. In: Proceedings of the FSE/SDP Workshop on Future of Software Engineering Research, pp. 161–165. ACM, Santa Fe (2010)

    Google Scholar 

  16. Bird, C., Menzies, T., Zimmermann, T.: The Art and Science of Analyzing Software Data. Morgan Kaufmann, Waltham (2015)

    Google Scholar 

  17. Santucci, G., Lange, S.: Internet of Things in 2020 - a roadmap for the future. INFSO D.4 Networked Enterprise & RFID INFSO G.2 Micro & Nanosystems (2008)

    Google Scholar 

  18. Agrawal, S., Vieira, D.: A survey on Internet of Things. Abakós 1, 78–95 (2013)

    Article  Google Scholar 

  19. Atzori, L., Iera, A., Morabito, G.: The Internet of Things: a survey. Comput. Netw. 54, 2787–2805 (2010)

    Article  MATH  Google Scholar 

  20. Foidl, H., Felderer, M.: Research challenges of industry 4.0 for quality management. In: Felderer, M., Piazolo, F., Ortner, W., Brehm, L., Hof, H.-J. (eds.) ERP Future 2015 - Research. LNBIP, vol. 245, pp. 121–137. Springer, Heidelberg (2016). doi:10.1007/978-3-319-32799-0_10

    Chapter  Google Scholar 

  21. Vermesan, O., Harrison, M., Vogt, H., Kalaboukas, K., Tomasella, M., Wouters, K., Gusmeroli, S., Haller, S.: Strategic research agenda. In: Sundmaeker, H., Guillemin, P., Friess, P., Woelfflé, S. (eds.) Vision and Challenges for Realising the Internet of Things, pp. 39–82. CERP-IoT – Cluster of European Research Projects on the Internet of Things, Luxembourg (2010)

    Google Scholar 

  22. Miorandi, D., Sicari, S., De Pellegrini, F., Chlamtac, I.: Internet of Things: vision, applications and research challenges. Ad Hoc Netw. 10, 1497–1516 (2012)

    Article  Google Scholar 

  23. Zhang, D., Yang, L.T., Huang, H.: Searching in Internet of Things: vision and challenges. In: Ninth IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA 2011), pp. 201–206. IEEE (2011)

    Google Scholar 

  24. Liu, Y., Zhou, G.: Key technologies and applications of Internet of Things. In: Fifth International Conference on Intelligent Computation Technology and Automation (ICICTA 2012), pp. 197–200. IEEE, Zhangjiajie (2012)

    Google Scholar 

  25. van der Aalst, W.M.P.: Extracting event data from databases to unleash process mining. In: Vom Brocke, J., Schmiedel, T. (eds.) BPM - Driving Innovation in a Digital World. Management for Professionals, pp. 105–128. Springer, Switzerland (2015)

    Google Scholar 

  26. Naur, P.: The Science of datalogy. Commun. ACM 9, 485 (1966)

    Article  Google Scholar 

  27. Smith, J.F.: Data Science as an academic discipline. Data Sci. J. 5, 163–164 (2006)

    Article  Google Scholar 

  28. Data Science Journal. http://datascience.codata.org/

  29. Journal of Data Science. http://www.jds-online.com/about

  30. Hayashi, E.C.: What is data science? Fundamental concepts and a heuristic example. In: Hayashi, C., et al. (eds.) Data Science, Classification, and Related Methods. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 40–51. Springer, Japan (1996)

    Google Scholar 

  31. Liu, L., Zhang, H., Li, J., Wang, R., Yu, L., Yu, J., Li, P.: Building a community of data scientists: an explorative analysis. Data Sci. J. 8, 201–208 (2009)

    Article  Google Scholar 

  32. Dhar, V.: Data science and prediction (2012)

    Google Scholar 

  33. Waller, M.A., Fawcett, S.E.: Data science, predictive analytics, and big data: a revolution that will transform supply chain design and management. J. Bus. Logistics 34, 77–84 (2013)

    Article  Google Scholar 

  34. van der Aalst, W.M.P.: Data scientist: the engineer of the future. In: Mertins, K., Bénaben, F., Poler, R., Bourrières, J.-P. (eds.) Enterprise Interoperability VI - Interoperability for Agility, Resilience and Plasticity of Collaborations. Proceedings of the I-ESA Conferences, vol. 7, pp. 13–26. Springer, Switzerland (2014)

    Chapter  Google Scholar 

  35. Menzies, T., Zimmermann, T.: Software analytics: so what? IEEE Softw. 30, 31–37 (2013)

    Article  Google Scholar 

  36. Kim, M., Zimmermann, T., DeLine, R., Begel, A.: The emerging role of data scientists on software development teams - Technical report. MSR-TR-2015-30. Microsoft Research (2015)

    Google Scholar 

  37. Begel, A., Zimmermann, T.: Analyze this! 145 questions for data scientists in software engineering. In: Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), pp. 12–23. ACM (2014)

    Google Scholar 

  38. Mockus, A., Weiss, D.M., Zhang, P.: Understanding and predicting effort in software projects. In: Proceedings of the 25th International Conference on Software Engineering (ICSE 2003), pp. 274–284. IEEE Computer Society (2003)

    Google Scholar 

  39. Zimmermann, T., Weißgerber, P., Diehl, S., Zeller, A.: Mining version histories to guide software changes. In: Proceedings of the 26th International Conference on Software Engineering (ICSE 2004), pp. 563–572. IEEE (2004)

    Google Scholar 

  40. Cheatham, T.J.: Software testing: a machine learning experiment. In: 23rd Annual Conference on Computer Science (CSC 1995), pp. 135–141. ACM (1995)

    Google Scholar 

  41. Gorla, A., Tavecchia, I., Gross, F., Zeller, A.: Checking app behavior against app descriptions. In: 36th International Conference on Software Engineering (ICSE 2014), pp. 1025–1035. ACM (2014)

    Google Scholar 

  42. Chaturvedi, K.K., Singh, V.B., Singh, P.: Tools in mining software repositories. In: 13th International Conference on Computational Science and Its Applications (ICCSA 2013), pp. 89–98. IEEE (2013)

    Google Scholar 

  43. Fuggetta, A., Di Nitto, E.: Software process. In: Proceedings of the Future of Software Engineering, FOSE 2014, pp. 1–12. ACM (2014)

    Google Scholar 

  44. Trendowicz, A., Kopczynska, S.: Adapting multi-criteria decision analysis for assessing the quality of software products. Current approaches and future perspectives. In: Hurson, A., Memon, A. (eds.) Advances in Computers, vol. 93, pp. 153–226. Academic Press, Waltham (2014)

    Google Scholar 

  45. ISO/IEC/IEEE: ISO/IEC/IEEE 24765:2010 - Systems and software engineering — Vocabulary. ISO (2010)

    Google Scholar 

  46. Wagner, S.: Software Product Quality Control. Springer, Heidelberg (2013)

    Book  Google Scholar 

  47. Garousi, V., Felderer, M., Mäntylä, M.V.: The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In: 20th International Conference on Evaluation and Assessment in Software Engineering (EASE 2016). ACM, Limerick (2016)

    Google Scholar 

  48. Cognizant. http://www.cognizant.com/InsightsWhitepapers/the-internet-of-things-qa-unleashed-codex1233.pdf

  49. TechArcis Solutions. http://techarcis.com/whitepaper/testing-for-internet-of-things/

  50. Polarion Software. https://www.polarion.com/resources/download/testing-the-internet-of-things?utm_campaign=Blog-2016-Embedded-Q1&utm_medium=Blog&utm_source=Blog

  51. Ayla Networks. http://theinternetofthings.report/Resources/Whitepapers/7f4b81fe-25c3-4fa1-a1fb-a12fa6d42f44_Ayla_Whitepaper_Art-of-IoT-QA.pdf

  52. Testbirds. https://www.testbirds.com/fileadmin/Whitepaper-Studies/Whitepaper-Internet-of-Things-EN.pdf

  53. Gerrard Consulting. http://gerrardconsulting.com/sites/default/files/IoETestStrategy.pdf

  54. Henning, B.: http://de.slideshare.net/HenningBoeger/german-testing-nightiothenningboeger20130228enexport

  55. Test and Verification Solutions. http://www.testandverification.com/wp-content/uploads/Testing%20the%20Internet%20of%20Things.pdf

  56. Lau, M.: Testing the Internet of Things. Printed Circuit Design and Fab, 43, April 2014

    Google Scholar 

  57. DevOps. http://devops.com/2015/02/24/functional-testing-iot/

  58. LeanTesting. https://leantesting.com/resources/how-do-we-test-the-internet-of-things/

  59. Neotys. http://www.neotys.com/blog/performance-testing-101-how-to-approach-the-internet-of-things/

  60. Semiconductor Engineering. http://semiengineering.com/how-to-cut-verification-costs-for-iot/

  61. SmartBear. http://blog.smartbear.com/user-experience/testing-the-internet-of-things/

  62. IoT-Now. http://www.iot-now.com/2015/05/25/33241-testing-the-internet-of-things-its-time-to-plan-your-test-strategy/

  63. Beyond Security. http://www.beyondsecurity.com/security_testing_iot_internet_of_things.html

  64. CenturyLink Cloud. https://www.ctl.io/blog/post/qa-with-the-iot/

  65. Embedded. http://www.embedded.com/electronics-news/4437315/The-testing-challenges-ahead-for-the-Internet-of-things

  66. LogiGear. http://www.logigear.com/magazine/issue/past-articles/testing-strategy-for-the-iot/

  67. TestPlant. http://www.testplant.com/explore/testing-use-cases/testing-the-internet-of-things-set-top-boxes/

  68. Nilsson, D.K., Larson, U.E.: Secure firmware updates over the air in intelligent vehicles. In: ICC Workshops - 2008 IEEE International Conference on Communications Workshops, pp. 380–384. IEEE (2008)

    Google Scholar 

  69. Zumel, N., Mount, J.: Practical Data Science with R. Manning Publications, New York (2014)

    Google Scholar 

  70. Schutt, R., O’Neil, C.: Doing Data Science. O’Reilly, Sebastopol (2014)

    Google Scholar 

  71. Buse, R.P.L., Zimmermann, T.: Information needs for software development analytics. In: Proceedings of the 34th International Conference on Software Engineering (ICSE 2012), pp. 987–996. IEEE (2012)

    Google Scholar 

  72. Suma, V., Nair Gopalakrishnan, T.R.: Effective defect prevention approach in software process for achieving better quality levels. Proc. World Acad. Sci. Eng. Technol. 42, 258–262 (2008)

    Google Scholar 

  73. Kumaresh, S., Baskaran, R.: Defect analysis and prevention for software process quality improvement. Int. J. Comput. Appl. 8, 42–47 (2010)

    Google Scholar 

  74. Han, S., Dang, Y., Ge, S., Zhang, D., Xie, T.: Performance debugging in the large via mining millions of stack traces. In: Proceedings of the 34th International Conference on Software Engineering (ICSE 2012), pp. 145–155. IEEE (2012)

    Google Scholar 

  75. Jiang, Z.M., Hassan, A.E., Hamann, G., Flora, P.: Automated performance analysis of load tests. In: International Conference on Software Maintenance (ICSM 2009), pp. 125–134. IEEE (2009)

    Google Scholar 

  76. Hindle, A.: Green mining: a methodology of relating software change to power consumption. In: 9th IEEE Working Conference on Mining Software Repositories (MSR), pp. 78–87. IEEE (2012)

    Google Scholar 

  77. Rubin, V., Lomazova, I., van der Aalst, W.M.P.: Agile development with software process mining. In: Proceedings of the 2014 International Conference on Software and System Process (ICSSP 2014), pp. 70–74. ACM (2014)

    Google Scholar 

  78. van der Aalst, W.M.P.: Process Mining – Discovery, Conformance and Enhancement of Business Processes. Springer, Heidelberg (2011)

    MATH  Google Scholar 

  79. Rubin, V.A., Mitsyuk, A.A., Lomazova, I.A., van der Aalst, W.M.P.: Process mining can be applied to software too! In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2014). ACM, Torino (2014)

    Google Scholar 

  80. Shershakov, S.A., Rubin, V.A.: System runs analysis with process mining. Model. Anal. Inf. Syst. 22, 813–833 (2015)

    MathSciNet  Google Scholar 

  81. Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. Softw. Eng. 32, 2–13 (2007)

    Article  Google Scholar 

  82. Ostrand, T.J., Weyuker, E.J., Bell, R.M.: Where the bugs are. In: International Symposium on Software Testing and Analysis (ISSTA 2004), pp. 86–96. ACM (2004)

    Google Scholar 

  83. Lazic, L., Velasevic, D.: Applying simulation and design of experiments to the embedded software testing process. Softw. Test. Verification Reliab. 14, 257–282 (2004)

    Article  Google Scholar 

  84. Wagner, S.: Defect classification and defect types revisited. In: International Symposium on Software Testing and Analysis (ISSTA 2008) – Workshop on Defects in Large Software Systems (DEFECTS 2008), pp. 39–40. ACM (2008)

    Google Scholar 

  85. Wong, E.W., Qi, Y.: BP neural network-based effective fault localization. Int. J. Softw. Eng. Knowl. Eng. 19, 573–597 (2009)

    Article  Google Scholar 

  86. Kannadhasan, N., Maheswari, U.B.: Machine learning based methodology for testing object oriented applications. J. Eng. Appl. Sci. 10, 7400–7405 (2015)

    Google Scholar 

  87. ISO/IEC: ISO/IEC 25010:2011 - Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models. ISO/IEC (2011)

    Google Scholar 

  88. Cao, H., Bao, T., Yang, Q., Chen, E., Tian, J.: An effective approach for mining mobile user habits. In: 19th ACM International Conference on Information and Knowledge Management (CIKM 2010), pp. 1677–1680. ACM (2010)

    Google Scholar 

  89. Gruska, N., Wasylkowski, A., Zeller, A.: Learning from 6,000 projects: lightweight cross-project anomaly detection. In: Proceedings of the 19th International Symposium on Software Testing and Analysis (ISSTA 2010), pp. 119–130. ACM (2010)

    Google Scholar 

  90. Santos, R.M.S., Oliveira, T.C., Brito e Abreu, F.: Mining software development process variations. In: 30th Annual ACM Symposium on Applied Computing (SAC 2015), pp. 1657–1660. ACM (2015)

    Google Scholar 

  91. Bacchelli, A., Dal Sasso, T., DʼAmbros, M., Lanza, M.: Content classification of development emails. In: 34th International Conference on Software Engineering (ICSE 2012), pp. 375–385. IEEE (2012)

    Google Scholar 

  92. Bird, C., Gourley, A., Devanbu, P., Gertz, M., Swaminathan, A.: Mining email social networks. In: International Workshop on Mining Software Repositories (MSR 2006), pp. 137–143. ACM (2006)

    Google Scholar 

  93. Noorian, M., Bagheri, E., Du, W.: Machine learning-based software testing: towards a classification framework. In: 23rd International Conference on Software Engineering & Knowledge Engineering (SEKE 2011), pp. 225–229 (2011)

    Google Scholar 

  94. Lenz, A.R., Pozo, A., Vergilio, S.R.: Linking software testing results with a machine learning approach. Eng. Appl. Artif. Intell. 26, 1631–1640 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Computer Science, University of Innsbruck, Innsbruck, Austria

    Harald Foidl & Michael Felderer

Authors
  1. Harald Foidl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Felderer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Felderer .

Editor information

Editors and Affiliations

  1. Lero, Limerick, Ireland

    Tiziana Margaria

  2. TU Dortmund, Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Foidl, H., Felderer, M. (2016). Data Science Challenges to Improve Quality Assurance of Internet of Things Applications. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation: Discussion, Dissemination, Applications. ISoLA 2016. Lecture Notes in Computer Science(), vol 9953. Springer, Cham. https://doi.org/10.1007/978-3-319-47169-3_54

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47169-3_54

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47168-6

  • Online ISBN: 978-3-319-47169-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation