Abstract
With the increasing importance and complexity of Internet of Things (IoT) applications, also the development of adequate quality assurance techniques becomes essential. Due to the massive amount of data generated in workflows of IoT applications, data science plays a key role in their quality assurance. In this paper, we present respective data science challenges to improve quality assurance of Internet of Things applications. Based on an informal literature review, we first outline quality assurance requirements evolving with the IoT grouped into six categories (Environment, User, Compliance/Service Level Agreement, Organizational, Security and Data Management). Finally, we present data science challenges to improve the quality assurance of Internet of Things applications sub-divided into four categories (Defect prevention, Defect analysis, User incorporation and Organizational) derived from the six quality assurance requirement categories.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Santucci, G.: The Internet of Things: between the revolution of the internet and the metamorphosis of objects. In: Sundmaeker, H., Guillemin, P., Friess, P., Woelfflé, S. (eds.) Vision and Challenges for Realising the Internet of Things, pp. 11–24. CERP-IoT – Cluster of European Research Projects on the Internet of Things, Luxembourg (2010)
Lee, I., Lee, K.: The Internet of Things (IoT): applications, investments, and challenges for enterprises. Bus. Horiz. 58, 431–440 (2015)
Marwah, Q.M., Sirshar, M.: Software quality assurance in Internet of Things. Int. J. Comput. Appl. 109, 16–24 (2015)
Gubbi, J., Buyya, R., Marusic, S., Palaniswami, M.: Internet of Things (IoT): a vision, architectural elements, and future directions. Future Gener. Comput. Syst. 29, 1645–1660 (2013)
Xia, F., Yang, L.T., Wang, L., Vinel, A.: Internet of Things. Int. J. Commun Syst 25, 1101–1102 (2012)
Prasad, N.R., Eisenhauer, M., Ahlsén, M., Badii, A., Brinkmann, A., Hansen, K.M., Rosengren, P.: Open source middleware for networked embedded systems towards future Internet of Things. In: Sundmaeker, H., Guillemin, P., Friess, P., Woelfflé, S. (eds.) Vision and Challenges for Realising the Internet of Things, pp. 153–163. CERP-IoT – Cluster of European Research Projects on the Internet of Things, Luxembourg (2010)
Cai, L., Zhu, Y.: The challenges of data quality and data quality assessment in the Big Data Era. Data Sci. J. 14, 1–10 (2015)
Zhu, Y., Xiong, Y.: Towards data science. Data Sci. J. 14, 1–7 (2015)
Katal, A., Wazid, M., Goudar, R.H.: Big Data: issues, challenges, tools and good practices. In: Sixth International Conference on Contemporary Computing (IC3 2013), pp. 404–409. IEEE (2013)
IEEE: IEEE Standard for Software Quality Assurance Processes, vol. 730™-2014 (Revision of IEEE Std. 730-2002). IEEE Computer Society, New York (2014)
May, T.: The New Know: Innovation Powered by Analytics. Wiley, New Jersey (2009)
Provost, F., Fawcett, T.: Data science and its relationship to Big Data and data-driven decision making. Big Data 1, 51–59 (2013)
Taylor, Q., Giraud-Carrier, C.: Applications of data mining in software engineering. Int. J. Data Anal. Tech. Strat. 2, 243–257 (2010)
Xie, T., Pei, J., Hassan, A.E.: Mining software engineering data. In: 29th International Conference on Software Engineering (ICSE 2007 Companion), pp. 172–173. IEEE, Minneapolis (2007)
Hassan, A.E., Xie, T.: Software intelligence: the future of mining software engineering data. In: Proceedings of the FSE/SDP Workshop on Future of Software Engineering Research, pp. 161–165. ACM, Santa Fe (2010)
Bird, C., Menzies, T., Zimmermann, T.: The Art and Science of Analyzing Software Data. Morgan Kaufmann, Waltham (2015)
Santucci, G., Lange, S.: Internet of Things in 2020 - a roadmap for the future. INFSO D.4 Networked Enterprise & RFID INFSO G.2 Micro & Nanosystems (2008)
Agrawal, S., Vieira, D.: A survey on Internet of Things. Abakós 1, 78–95 (2013)
Atzori, L., Iera, A., Morabito, G.: The Internet of Things: a survey. Comput. Netw. 54, 2787–2805 (2010)
Foidl, H., Felderer, M.: Research challenges of industry 4.0 for quality management. In: Felderer, M., Piazolo, F., Ortner, W., Brehm, L., Hof, H.-J. (eds.) ERP Future 2015 - Research. LNBIP, vol. 245, pp. 121–137. Springer, Heidelberg (2016). doi:10.1007/978-3-319-32799-0_10
Vermesan, O., Harrison, M., Vogt, H., Kalaboukas, K., Tomasella, M., Wouters, K., Gusmeroli, S., Haller, S.: Strategic research agenda. In: Sundmaeker, H., Guillemin, P., Friess, P., Woelfflé, S. (eds.) Vision and Challenges for Realising the Internet of Things, pp. 39–82. CERP-IoT – Cluster of European Research Projects on the Internet of Things, Luxembourg (2010)
Miorandi, D., Sicari, S., De Pellegrini, F., Chlamtac, I.: Internet of Things: vision, applications and research challenges. Ad Hoc Netw. 10, 1497–1516 (2012)
Zhang, D., Yang, L.T., Huang, H.: Searching in Internet of Things: vision and challenges. In: Ninth IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA 2011), pp. 201–206. IEEE (2011)
Liu, Y., Zhou, G.: Key technologies and applications of Internet of Things. In: Fifth International Conference on Intelligent Computation Technology and Automation (ICICTA 2012), pp. 197–200. IEEE, Zhangjiajie (2012)
van der Aalst, W.M.P.: Extracting event data from databases to unleash process mining. In: Vom Brocke, J., Schmiedel, T. (eds.) BPM - Driving Innovation in a Digital World. Management for Professionals, pp. 105–128. Springer, Switzerland (2015)
Naur, P.: The Science of datalogy. Commun. ACM 9, 485 (1966)
Smith, J.F.: Data Science as an academic discipline. Data Sci. J. 5, 163–164 (2006)
Data Science Journal. http://datascience.codata.org/
Journal of Data Science. http://www.jds-online.com/about
Hayashi, E.C.: What is data science? Fundamental concepts and a heuristic example. In: Hayashi, C., et al. (eds.) Data Science, Classification, and Related Methods. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 40–51. Springer, Japan (1996)
Liu, L., Zhang, H., Li, J., Wang, R., Yu, L., Yu, J., Li, P.: Building a community of data scientists: an explorative analysis. Data Sci. J. 8, 201–208 (2009)
Dhar, V.: Data science and prediction (2012)
Waller, M.A., Fawcett, S.E.: Data science, predictive analytics, and big data: a revolution that will transform supply chain design and management. J. Bus. Logistics 34, 77–84 (2013)
van der Aalst, W.M.P.: Data scientist: the engineer of the future. In: Mertins, K., Bénaben, F., Poler, R., Bourrières, J.-P. (eds.) Enterprise Interoperability VI - Interoperability for Agility, Resilience and Plasticity of Collaborations. Proceedings of the I-ESA Conferences, vol. 7, pp. 13–26. Springer, Switzerland (2014)
Menzies, T., Zimmermann, T.: Software analytics: so what? IEEE Softw. 30, 31–37 (2013)
Kim, M., Zimmermann, T., DeLine, R., Begel, A.: The emerging role of data scientists on software development teams - Technical report. MSR-TR-2015-30. Microsoft Research (2015)
Begel, A., Zimmermann, T.: Analyze this! 145 questions for data scientists in software engineering. In: Proceedings of the 36th International Conference on Software Engineering (ICSE 2014), pp. 12–23. ACM (2014)
Mockus, A., Weiss, D.M., Zhang, P.: Understanding and predicting effort in software projects. In: Proceedings of the 25th International Conference on Software Engineering (ICSE 2003), pp. 274–284. IEEE Computer Society (2003)
Zimmermann, T., Weißgerber, P., Diehl, S., Zeller, A.: Mining version histories to guide software changes. In: Proceedings of the 26th International Conference on Software Engineering (ICSE 2004), pp. 563–572. IEEE (2004)
Cheatham, T.J.: Software testing: a machine learning experiment. In: 23rd Annual Conference on Computer Science (CSC 1995), pp. 135–141. ACM (1995)
Gorla, A., Tavecchia, I., Gross, F., Zeller, A.: Checking app behavior against app descriptions. In: 36th International Conference on Software Engineering (ICSE 2014), pp. 1025–1035. ACM (2014)
Chaturvedi, K.K., Singh, V.B., Singh, P.: Tools in mining software repositories. In: 13th International Conference on Computational Science and Its Applications (ICCSA 2013), pp. 89–98. IEEE (2013)
Fuggetta, A., Di Nitto, E.: Software process. In: Proceedings of the Future of Software Engineering, FOSE 2014, pp. 1–12. ACM (2014)
Trendowicz, A., Kopczynska, S.: Adapting multi-criteria decision analysis for assessing the quality of software products. Current approaches and future perspectives. In: Hurson, A., Memon, A. (eds.) Advances in Computers, vol. 93, pp. 153–226. Academic Press, Waltham (2014)
ISO/IEC/IEEE: ISO/IEC/IEEE 24765:2010 - Systems and software engineering — Vocabulary. ISO (2010)
Wagner, S.: Software Product Quality Control. Springer, Heidelberg (2013)
Garousi, V., Felderer, M., Mäntylä, M.V.: The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In: 20th International Conference on Evaluation and Assessment in Software Engineering (EASE 2016). ACM, Limerick (2016)
Cognizant. http://www.cognizant.com/InsightsWhitepapers/the-internet-of-things-qa-unleashed-codex1233.pdf
TechArcis Solutions. http://techarcis.com/whitepaper/testing-for-internet-of-things/
Polarion Software. https://www.polarion.com/resources/download/testing-the-internet-of-things?utm_campaign=Blog-2016-Embedded-Q1&utm_medium=Blog&utm_source=Blog
Ayla Networks. http://theinternetofthings.report/Resources/Whitepapers/7f4b81fe-25c3-4fa1-a1fb-a12fa6d42f44_Ayla_Whitepaper_Art-of-IoT-QA.pdf
Testbirds. https://www.testbirds.com/fileadmin/Whitepaper-Studies/Whitepaper-Internet-of-Things-EN.pdf
Gerrard Consulting. http://gerrardconsulting.com/sites/default/files/IoETestStrategy.pdf
Henning, B.: http://de.slideshare.net/HenningBoeger/german-testing-nightiothenningboeger20130228enexport
Test and Verification Solutions. http://www.testandverification.com/wp-content/uploads/Testing%20the%20Internet%20of%20Things.pdf
Lau, M.: Testing the Internet of Things. Printed Circuit Design and Fab, 43, April 2014
DevOps. http://devops.com/2015/02/24/functional-testing-iot/
LeanTesting. https://leantesting.com/resources/how-do-we-test-the-internet-of-things/
Neotys. http://www.neotys.com/blog/performance-testing-101-how-to-approach-the-internet-of-things/
Semiconductor Engineering. http://semiengineering.com/how-to-cut-verification-costs-for-iot/
SmartBear. http://blog.smartbear.com/user-experience/testing-the-internet-of-things/
Beyond Security. http://www.beyondsecurity.com/security_testing_iot_internet_of_things.html
CenturyLink Cloud. https://www.ctl.io/blog/post/qa-with-the-iot/
LogiGear. http://www.logigear.com/magazine/issue/past-articles/testing-strategy-for-the-iot/
TestPlant. http://www.testplant.com/explore/testing-use-cases/testing-the-internet-of-things-set-top-boxes/
Nilsson, D.K., Larson, U.E.: Secure firmware updates over the air in intelligent vehicles. In: ICC Workshops - 2008 IEEE International Conference on Communications Workshops, pp. 380–384. IEEE (2008)
Zumel, N., Mount, J.: Practical Data Science with R. Manning Publications, New York (2014)
Schutt, R., O’Neil, C.: Doing Data Science. O’Reilly, Sebastopol (2014)
Buse, R.P.L., Zimmermann, T.: Information needs for software development analytics. In: Proceedings of the 34th International Conference on Software Engineering (ICSE 2012), pp. 987–996. IEEE (2012)
Suma, V., Nair Gopalakrishnan, T.R.: Effective defect prevention approach in software process for achieving better quality levels. Proc. World Acad. Sci. Eng. Technol. 42, 258–262 (2008)
Kumaresh, S., Baskaran, R.: Defect analysis and prevention for software process quality improvement. Int. J. Comput. Appl. 8, 42–47 (2010)
Han, S., Dang, Y., Ge, S., Zhang, D., Xie, T.: Performance debugging in the large via mining millions of stack traces. In: Proceedings of the 34th International Conference on Software Engineering (ICSE 2012), pp. 145–155. IEEE (2012)
Jiang, Z.M., Hassan, A.E., Hamann, G., Flora, P.: Automated performance analysis of load tests. In: International Conference on Software Maintenance (ICSM 2009), pp. 125–134. IEEE (2009)
Hindle, A.: Green mining: a methodology of relating software change to power consumption. In: 9th IEEE Working Conference on Mining Software Repositories (MSR), pp. 78–87. IEEE (2012)
Rubin, V., Lomazova, I., van der Aalst, W.M.P.: Agile development with software process mining. In: Proceedings of the 2014 International Conference on Software and System Process (ICSSP 2014), pp. 70–74. ACM (2014)
van der Aalst, W.M.P.: Process Mining – Discovery, Conformance and Enhancement of Business Processes. Springer, Heidelberg (2011)
Rubin, V.A., Mitsyuk, A.A., Lomazova, I.A., van der Aalst, W.M.P.: Process mining can be applied to software too! In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2014). ACM, Torino (2014)
Shershakov, S.A., Rubin, V.A.: System runs analysis with process mining. Model. Anal. Inf. Syst. 22, 813–833 (2015)
Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. Softw. Eng. 32, 2–13 (2007)
Ostrand, T.J., Weyuker, E.J., Bell, R.M.: Where the bugs are. In: International Symposium on Software Testing and Analysis (ISSTA 2004), pp. 86–96. ACM (2004)
Lazic, L., Velasevic, D.: Applying simulation and design of experiments to the embedded software testing process. Softw. Test. Verification Reliab. 14, 257–282 (2004)
Wagner, S.: Defect classification and defect types revisited. In: International Symposium on Software Testing and Analysis (ISSTA 2008) – Workshop on Defects in Large Software Systems (DEFECTS 2008), pp. 39–40. ACM (2008)
Wong, E.W., Qi, Y.: BP neural network-based effective fault localization. Int. J. Softw. Eng. Knowl. Eng. 19, 573–597 (2009)
Kannadhasan, N., Maheswari, U.B.: Machine learning based methodology for testing object oriented applications. J. Eng. Appl. Sci. 10, 7400–7405 (2015)
ISO/IEC: ISO/IEC 25010:2011 - Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models. ISO/IEC (2011)
Cao, H., Bao, T., Yang, Q., Chen, E., Tian, J.: An effective approach for mining mobile user habits. In: 19th ACM International Conference on Information and Knowledge Management (CIKM 2010), pp. 1677–1680. ACM (2010)
Gruska, N., Wasylkowski, A., Zeller, A.: Learning from 6,000 projects: lightweight cross-project anomaly detection. In: Proceedings of the 19th International Symposium on Software Testing and Analysis (ISSTA 2010), pp. 119–130. ACM (2010)
Santos, R.M.S., Oliveira, T.C., Brito e Abreu, F.: Mining software development process variations. In: 30th Annual ACM Symposium on Applied Computing (SAC 2015), pp. 1657–1660. ACM (2015)
Bacchelli, A., Dal Sasso, T., DʼAmbros, M., Lanza, M.: Content classification of development emails. In: 34th International Conference on Software Engineering (ICSE 2012), pp. 375–385. IEEE (2012)
Bird, C., Gourley, A., Devanbu, P., Gertz, M., Swaminathan, A.: Mining email social networks. In: International Workshop on Mining Software Repositories (MSR 2006), pp. 137–143. ACM (2006)
Noorian, M., Bagheri, E., Du, W.: Machine learning-based software testing: towards a classification framework. In: 23rd International Conference on Software Engineering & Knowledge Engineering (SEKE 2011), pp. 225–229 (2011)
Lenz, A.R., Pozo, A., Vergilio, S.R.: Linking software testing results with a machine learning approach. Eng. Appl. Artif. Intell. 26, 1631–1640 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Foidl, H., Felderer, M. (2016). Data Science Challenges to Improve Quality Assurance of Internet of Things Applications. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation: Discussion, Dissemination, Applications. ISoLA 2016. Lecture Notes in Computer Science(), vol 9953. Springer, Cham. https://doi.org/10.1007/978-3-319-47169-3_54
Download citation
DOI: https://doi.org/10.1007/978-3-319-47169-3_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47168-6
Online ISBN: 978-3-319-47169-3
eBook Packages: Computer ScienceComputer Science (R0)