Abstract
A service recommendation system is an information system that helps build mashups quickly to implement new features in response to environmental changes. With the development of deep learning (DL) in recent years, more researchers have started using DL-based methods to solve service recommendation problem and have achieved remarkable results. However, these works have some common deficiencies on non-unified dataset, pre-trained model, evaluation protocol, and experiment environment. These issues will disrupt evaluating the performance of models accurately and make reproducing them difficult. To solve these problems, we propose a service mashup recommendation benchmark (SMRB) that provides a standard environment to enhance comparability between models and credibility of results. We implement eight models (five from top service computing conferences and journals and three created by ourselves) based on SMRB and compare their performance, which proves the effectiveness of SMRB. After analyzing these results, we found that most DL-based models do not perform as well as they promise; instead, the simplest Multilayer perceptron (MLP) models perform better after tuning the parameters, which inspires us to re-examine whether the particular structure of the model can be helpful for the intended purpose and whether it can really improve the performance of the recommendation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bhatia, K., et al.: The extreme classification repository: multi-label datasets and code (2016)
Bianchini, D., De Antonellis, V., Melchiori, M.: A multi-perspective framework for web API search in enterprise mashup design. In: Salinesi, C., Norrie, M.C., Pastor, Ó. (eds.) Advanced Information Systems Engineering, pp. 353–368. Springer, Berlin Heidelberg, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38709-8_23
Fletcher, K.K.: A quality-aware web API recommender system for mashup development. In: Ferreira, J.E., Musaev, A., Zhang, L.J. (eds.) Services Computing - SCC 2019, pp. 1–15. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-23554-3_1
Gao, W., Chen, L., Wu, J., Gao, H.: Manifold-learning based API recommendation for mashup creation. In: ICWS, pp. 432–439 (2015)
Jain, A., Liu, X., Yu, Q.: Aggregating functionality, use history, and popularity of APIs to recommend mashup creation. In: Barros, A., Grigori, D., Narendra, N.C., Dam, H.K. (eds.) Service-Oriented Computing, pp. 188–202. Springer, Berlin Heidelberg, Berlin, Heidelberg (2015). https://doi.org/10.1007/978-3-662-48616-0_12
Kang, G., Liu, J., Cao, B., Cao, M.: NAFM: neural and attentional factorization machine for web API recommendation. In: ICWS, pp. 330–337 (2020)
Li, H., Liu, J., Cao, B., Tang, M., Liu, X., Li, B.: Integrating tag, topic, co-occurrence, and popularity to recommend web APIs for mashup creation. In: SCC, pp. 84–91 (2017). https://doi.org/10.1109/SCC.2017.19
Liu, M., Tu, Z., Zhu, Y., Xu, X., Wang, Z., Sheng, Q.Z.: Data correction and evolution analysis of the Programmableweb service ecosystem. J. Syst. Softw. 182, 111066 (2021)
Liu, M., Zhu, Y., Xu, H., Tu, Z., Wang, Z.: T2L2: a tiny three linear layers model for service mashup creation. In: Hacid, H., Kao, O., Mecella, M., Moha, N., Paik, H. (eds.) ICSOC 2021. LNCS, vol. 13121, pp. 317–331. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-91431-8_20
Liu, Y., Cao, J.: API-prefer: an API package recommender system based on composition feature learning. In: Kafeza, E., Benatallah, B., Martinelli, F., Hacid, H., Bouguettaya, A., Motahari, H. (eds.) Service-Oriented Computing, pp. 500–507. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-65310-1_36
Ma, Y., Geng, X., Wang, J.: A deep neural network with multiplex interactions for cold-start service recommendation. IEEE Trans. Eng. Manage. 68(1), 105–119 (2021). https://doi.org/10.1109/TEM.2019.2961376
Ma, Y., Geng, X., Wang, J., He, K., Athanasopoulos, D.: Deep learning framework for online interactive service recommendation in iterative mashup development. CoRR abs/2101.02836 (2021). arXiv:2101.02836
Qi, L., Song, H., Zhang, X., Srivastava, G., Xu, X., Yu, S.: Compatibility-aware web API recommendation for mashup creation via textual description mining. ACM Trans. Multimedia Comput. Commun. Appl. 17(1s), 1–19 (2021)
Rahman, M.M., Liu, X.F.: Integrated topic modeling and user interaction enhanced webAPI recommendation using regularized matrix factorization for mashup application development. In: SCC, pp. 124–131 (2020)
Shi, M., Tang, Y., Liu, J.: Functional and contextual attention-based LSTM for service recommendation in mashup creation. IEEE Trans. Parallel Distrib. Syst. 30(5), 1077–1090 (2019)
Shi, M., Tang, Y., Liu, J.: TA-BLSTM: tag attention-based bidirectional long short-term memory for service recommendation in mashup creation. In: IJCNN, pp. 1–8 (2019). https://doi.org/10.1109/IJCNN.2019.8852438
Wu, H., Duan, Y., Yue, K., Zhang, L.: Mashup-oriented web API recommendation via multi-model fusion and multi-task learning. TSC 15, 3330–3343 (2021)
Yan, R., Fan, Y., Zhang, J., Zhang, J., Lin, H.: Service recommendation for composition creation based on collaborative attention convolutional network. In: ICWS, pp. 397–405 (2021). https://doi.org/10.1109/ICWS53863.2021.00059
Yao, L., Wang, X., Sheng, Q.Z., Benatallah, B., Huang, C.: Mashup recommendation by regularizing matrix factorization with API co-invocations. TSC 14(2), 502–515 (2021). https://doi.org/10.1109/TSC.2018.2803171
Yu, J., Benatallah, B., Casati, F., Daniel, F.: Understanding mashup development. IEEE Internet Comput. 12(5), 44–52 (2008)
Zheng, Z., Ma, H., Lyu, M.R., King, I.: QoS-aware web service recommendation by collaborative filtering. TSC 4(2), 140–152 (2011)
Acknowledgments
The research in this paper is partially supported by the National Key Research and Development Program of China (No 2021YFB3300700) and the National Natural Science Foundation of China (61832014, 61832004).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, T., Liu, M., Tu, Z., Wang, Z. (2023). Identifying and Removing the Ghosts of Reproducibility in Service Recommendation Research. In: Indulska, M., Reinhartz-Berger, I., Cetina, C., Pastor, O. (eds) Advanced Information Systems Engineering. CAiSE 2023. Lecture Notes in Computer Science, vol 13901. Springer, Cham. https://doi.org/10.1007/978-3-031-34560-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-031-34560-9_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34559-3
Online ISBN: 978-3-031-34560-9
eBook Packages: Computer ScienceComputer Science (R0)