Abstract
Dynamic data and continuously evolving sets of records are essential for a wide variety of today’s data management applications. Such applications range from large, social, content-driven Internet applications, to highly focused data processing verticals like data intensive science, telecommunications and intelligence applications. However, the dynamic and multimodal nature of data makes it challenging to transform it into machine-readable and machine-interpretable forms. In this paper, we report on an action research study that we conducted in collaboration with a multinational company in the embedded systems domain. In our study, and in the context of a real-world industrial application of dynamic data management, we provide insights to data science community and research to guide discussions and future research into dynamic data management in embedded systems. Our study identifies the key challenges in the phases of data collection, data storage and data cleaning that can significantly impact the overall performance of the system.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Darema, F.: Dynamic data driven applications systems: a new paradigm for application simulations and measurements. In: Bubak, M., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2004. LNCS, vol. 3038, pp. 662–669. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24688-6_86
Polyzotis, N., Roy, S., Whang, S.E., Zinkevich, M.: Data lifecycle challenges in production machine learning: a survey. SIGMOD Record 47, 17–28 (2018)
Kennedy, O., Ahmad, Y., Koch, C.: DBToaster: agile views for a dynamic data management system. In: CIDR 2011 - 5th Biennial Conference on Innovative Data Systems Research, Conference Proceedings, pp. 284–295 (2011)
Tegen, A., Davidsson, P., Mihailescu, R.C., Persson, J.A.: Collaborative sensing with interactive learning using dynamic intelligent virtual sensors. Sensors (Basel, Switzerland) 19(3), 477 (2019). https://doi.org/10.3390/s19030477
Charles, H., Bellarnyz, R., Ericksonz, T., Burnett, M.: Trials and tribulations of developers of intelligent systems: a field study, pp. 162–170 (2016). https://doi.org/10.1109/vlhcc.2016.7739680
Polyzotis, N., Roy, S., Whang, S.E., Zinkevich, M.: Data management challenges in production machine learning. pp. 1723–1726 (2017). https://doi.org/10.1145/3035918.3054782
Arpteg, A., Raj, A., Brinne, B., Crnkovic-Friis, L., Bosch, J.: Data Management Challenges of Deep Learning, pp. 50–59 (2019). https://doi.org/10.1109/seaa.2019.00018
Kumar, A., Boehm, M., Yang, J.: Data Management in Machine Learning: Challenges, Techniques, and Systems, pp. 1717–1722 (2017). https://doi.org/10.1145/3035918.3054775
Delaye, E., Sirasao, A., Dudha, C., Das, S.: Deep learning challenges and solutions with Xilinx FPGAs, pp. 908–913 (2017). https://doi.org/10.1109/iccad.2017.8203877
Lwakatare, L.E., Raj, A., Bosch, J., Olsson, H., Crnkovic, I.: A Hilltironomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation (2019). https://doi.org/10.1007/978-3-030-19034-7_14
Zhou, L., Pan, S., Wang, J., Vasilakos, A.: Machine learning on big data: opportunities and challenges. Neurocomputing 237, 350–361 (2017). https://doi.org/10.1016/j.neucom.2017.01.026
Chu, X., Ilyas, I., Krishnan, S., Wang, J.: Data Cleaning: Overview and Emerging Challenges, pp. 2201–2206 (2016). https://doi.org/10.1145/2882903.2912574
Krawczyk, B.: Learning from imbalanced data: open challenges and future directions. Progress Artif. Intell. 5, 221–232 (2016)
Ahuja, S., Angra, S.: Machine learning and its Applications: A Review (2017). https://doi.org/10.1109/icbdaci.2017.8070809
Hedgebeth, D.: Data-driven decision making for the enterprise: an overview of business intelligence applications. VINE 37(4), 414–420 (2007)
Maxwell, J.A.: Qualitative Research Design: An interactive approach, 2nd edn. SAGE Publications, Thousands Oaks (2005)
Berthold, M.R., Borgelt, C., Höppner, F., Klawonn, F.: Guide to Intelligent Data Analysis. TCS. Springer, London (2010). https://doi.org/10.1007/978-1-84882-260-3
Broy, M.: Challenges in automotive software engineering. In: Proceedings of the 28th International Conference on Software Engineering (ICSE 2006), ACM, New York, NY, USA, pp. 33–42 (2006)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ouhaichi, H., Olsson, H.H., Bosch, J. (2019). Dynamic Data Management for Machine Learning in Embedded Systems: A Case Study. In: Hyrynsalmi, S., Suoranta, M., Nguyen-Duc, A., Tyrväinen, P., Abrahamsson, P. (eds) Software Business. ICSOB 2019. Lecture Notes in Business Information Processing, vol 370. Springer, Cham. https://doi.org/10.1007/978-3-030-33742-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-33742-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33741-4
Online ISBN: 978-3-030-33742-1
eBook Packages: Computer ScienceComputer Science (R0)