Abstract
With the closer integration of database and machine learning, machine learning task in database can reduce the data transmission, thus dramatically boosting the runtime performance of the whole task. Moreover, if there is a chance of storing machine learning models involved in similar tasks in the system intelligently, the computation resource and time cost of repeated training will be greatly reduced. However, the intelligent storage system of machine learning model has not been developed yet. In order to achieve this goal, a method is proposed to measure the similarity of machine learning tasks. Second, the intelligent storage system of machine learning model was designed to manage models. Finally, it introduced the overall architecture and key technologies of intelligent storage system of machine learning model based on task similarity (ISSMLM), and describe three demonstration scenarios of the system. The results show the validity of the proposed method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Du, X.Y., Lu, W., Zhang, F.: History, present, and future of big data management systems. Ruan Jian Xue Bao/J. Softw. 30(1), 127–141 (2019). (in Chinese). http://www.jos.org.cn/1000-9825/5644.htm
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE, 20 June 2009
Lian, Z., Li, Y., Tao, J., Huang, J.: Improving speech emotion recognition via transformer-based predictive coding through transfer learning. arXiv. 2018 Nov:arXiv-1811
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 11 October 2018
Cui, J.W., Zhao, Z., Du, X.Y.: Data management technology for machine learning. Ruan Jian Xue Bao/J. Softw. (2021). (in Chinese)
Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243.2019, June 5
ONNX (2020). https://en.wikipedia.org/wiki/Open_Neural_Network_Exchange
Vartak, M., et al.: ModelDB: a system for machine learning model management. In: Proceedings of the Workshop on Human-In-the-Loop Data Analytics, pp. 1–3. 26 June 2016
Bhardwaj, A., et al.: Datahub: collaborative data science & dataset version management at scale. arXiv preprint arXiv:1409.0798. 2 September 2014
Miao, H., Li, A., Davis, L.S., Deshpande, A.: Modelhub: towards unified data and lifecycle management for deep learning. arXiv preprint arXiv:1611.06224. 18 November 2016
Smith, M.J., Sala, C., Kanter, J.M., Veeramachaneni, K.: The machine learning bazaar: harnessing the ML ecosystem for effective system development. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pp. pp. 785–800. 14 June 2020
Derakhshan, B., Rezaei Mahdiraji, A., Abedjan, Z., Rabl, T., Markl, V.: Optimizing machine learning workloads in collaborative environments. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pp. 1701–1716, 14 June 2020
Gu, B., Sung, Y.: enhanced reinforcement learning method combining one-hot encoding-based vectors for CNN-based alternative high-level decisions. Appl. Sci. 11(3), 1291 (2021)
Eaton, C.: Using a similarity measure to investigate factors in failure classification schemes (2021)
Phani, A.: LIMA: fine-grained lineage tracing and reuse in machine learning systems (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Cui, S., Wang, H., Xie, Y., Gu, H. (2021). Intelligent Storage System of Machine Learning Model Based on Task Similarity. In: Zeng, J., Qin, P., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2021. Communications in Computer and Information Science, vol 1451. Springer, Singapore. https://doi.org/10.1007/978-981-16-5940-9_9
Download citation
DOI: https://doi.org/10.1007/978-981-16-5940-9_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-5939-3
Online ISBN: 978-981-16-5940-9
eBook Packages: Computer ScienceComputer Science (R0)