Abstract
Today’s enterprise software systems are much more complicated than the past. Increasing numbers of dependent applications, heterogeneous technologies, and wide usage of Service-Oriented Architectures (SOA), where numerous services communicate with each other, makes testing of such systems challenging. For testing these software systems, the concept of service virtualization is gaining popularity. Service virtualization is an automated technique to mimic the behavior of a given real service. Services can be classified as stateless or stateful services. Many services are stateful in nature, yet virtualization of stateful services is harder than virtualization of stateless services. In this work, we introduce two novel stateful service virtualization approaches. We employ classification-based and sequence-to-sequence-based machine learning algorithms in developing our solutions. Classification is a supervised learning method where the task is assigning given inputs to corresponding classes. A sequence-to-sequence model is a deep neural network architecture where the input and the output are sequences. We demonstrate the validity of our approaches on three datasets. Our evaluation shows that we obtain 75 % to 81 % accuracy on subject datasets with classification based method. Our deep neural network-based solution achieves even better accuracy results ranging from 89 to 97 % on subject datasets. Our evaluation on training times of the mentioned techniques show that classification based technique significantly outperforms other methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., et al. (2016). Tensorflow:, Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467.
Alpaydin, E. (2014). Introduction to machine learning. MIT press.
Beschastnikh, I., Brun, Y., Ernst, M.D., Krishnamurthy, A. (2014). Inferring models of concurrent systems from logs of their behavior with csight. In: International Conference on Software Engineering.
Beschastnikh, I., Brun, Y., Schneider, S., Sloan, M., Ernst, M.D. (2011). Leveraging existing instrumentation to automatically infer invariant-constrained models. In: ACM SIGSOFT Symposium and European Conference on Foundations of Software Engineering.
Bhagya, T., Dietrich, J., Guesgen, H., Versteeg, S. (2018). GHTraffic: a dataset for reproducible research in service-oriented computing. In: International Conference on Web Services (ICWS) (pp. 123–130).
Biermann, A.W., & Feldman, J.A. (1972). On the synthesis of finite-state machines from samples of their behavior. IEEE Transactions on Computers, 100(6), 592–597.
Boettiger, C. (2015). An introduction to docker for reproducible research. ACM SIGOPS Operating Systems Review 49(1).
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., et al. (2016). End to end learning for self-driving cars. arXiv:1604.07316.
Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., Grobler, J., Layton, R., VanderPlas, J., Joly, A., Holt, B., Varoquaux, G. (2013). API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning.
Casey, M. (2008). The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Dynamics 8(6).
Chollet, F., & et al. (2015). Keras. https://github.com/fchollet/keras.
Clarke, E., Grumberg, O., Jha, S., Lu, Y., Veith, H. (2000). Counterexample-guided abstraction refinement. In: Computer Aided Verification. Springer.
Cleeremans, A., Servan-Schreiber, D., McClelland, J.L. (1989). Finite state automata and simple recurrent networks. Neural Computation, 1(3), 372–381.
Cook, J.E., & Wolf, A.L. (1998). Discovering models of software processes from event-based data. ACM Transactions on Software Engineering and Methodology (TOSEM), 7(3), 215–249.
Dallmeier, V., Lindig, C., Wasylkowski, A., Zeller, A. (2006). Mining object behavior with ADABU. In: International Workshop on Dynamic Systems Analysis.
Du, M., Schneider, J.G., Hine, C., Grundy, J., Versteeg, S. (2013). Generating service models by trace subsequence substitution. In: International ACM Sigsoft Conference on Quality of Software Architectures.
Du, M., Versteeg, S., Schneider, J.G., Han, J., Grundy, J. (2015). Interaction traces mining for efficient system responses generation. SIGSOFT Software Engineering Notes 40(1).
Enişer, H.F., & Sen, A. (2018). Testing service oriented architectures using stateful service visualization via machine learning. In: Proceedings of the 13th International Workshop on Automation of Software Test (pp. 9–15). ACM.
Eniser, H.F., Sen, A., Polat, S.O. (2018). Fancymock: creating virtual services from transactions. In: Proceedings of the 33rd Annual ACM Symposium on Applied Computing (pp. 1576–1578). ACM.
Farahmandpour, Z., Versteeg, S., Han, J., Kameswaran, A. (2017). Service virtualisation of internet-of-things devices: techniques and challenges. In: Proceedings of the 3rd International Workshop on Rapid Continuous Software Engineering (pp. 32–35). IEEE Press.
Giles, C.L., Miller, C.B., Chen, D., Chen, H.H., Sun, G.Z., Lee, Y.C. (2008). Learning and extracting finite state automata with second-order recurrent neural networks. Learning 4(3).
Giudice, D.L. (2014). Service virtualization and testing solutions. Forrester Wave.
Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. MIT Press. http://www.deeplearningbook.org.
Graves, A., Mohamed, A.R., Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In: 2013 ieee International Conference on Acoustics, Speech and Signal Processing (icassp) (pp. 6645–6649). IEEE.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
Hossain, M.A., Versteeg, S., Han, J., Kabir, M.A., Jiang, J., Schneider, J.G. (2018). Mining accurate message formats for service APIs. In: 25Th International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 266–276).
IT-Central-Station: Service virtualization a peek into what real users think. https://goo.gl/oN23qH (2017). Accessed at December 2017.
Kingma, D.P., & Ba, J. (2014). Adam:, A method for stochastic optimization. arXiv:1412.6980.
Krka, I., Brun, Y., Medvidovic, N. (2014). Automatic mining of specifications from invocation traces and method invariants. In: ACM SIGSOFT International Symposium on Foundations of Software Engineering (pp. 178–189). ACM.
Littman, M.L., & Sutton, R.S. (2002). Predictive representations of state. In: Advances in Neural Information Processing Systems (pp. 1555–1561).
Lorenzoli, D., Mariani, L., Pezzè, M. (2008). Automatic generation of software behavioral models. In: International Conference on Software Engineering (pp. 501–510). ACM.
Michelsen, J., & English, J. (2012). What is service virtualization?. In: Service Virtualization (pp. 27–35). Springer.
Mohammad, S.S. (2010). A literature survey on algorithms for multi-label learning. Oregon State University, Corvallis 73.
Murphy, T.E., & Wilson, N. (2013). Magic quadrant for integrated software quality suites. Gartner Research.
Nizamic, F., Groenboom, R., Lazovik, A. (2011). Testing for highly distributed service-oriented systems using virtual environments. Dutch Testing Day.
Ohmann, T., Herzberg, M., Fiss, S., Halbert, A., Palyart, M., Beschastnikh, I., Brun, Y. (2014). Behavioral resource-aware model inference. In: International Conference on Automated Software Engineering.
Reiss, S.P., & Renieris, M. (2001). Encoding program executions. In: International Conference on Software Engineering.
Schneider, J.G., Mandile, P., Versteeg, S. (2015). Generalized suffix tree based multiple sequence alignment for service virtualization. In: Australasian Software Engineering Conference (ASWEC).
Singh, S., James, M.R., Rudary, M.R. (2004). Predictive state representations: a new theory for modeling dynamical systems. In: Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (pp. 512–519). AUAI Press.
Sutskever, I., Vinyals, O., Le, Q.V. (2014). Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems.
Tang, L., Rajan, S., Narayanan, V.K. (2009). Large scale multi-label classification via metalabeler. In: Proceedings of the 18th International Conference on World Wide Web (pp. 211–220). ACM.
Versteeg, S., Du, M., Bird, J., Schneider, J.G., Grundy, J., Han, J. (2016a). Enhanced playback of automated service emulation models using entropy analysis. In: International Workshop on Continuous Software Evolution and Delivery (CSED).
Versteeg, S., Du, M., Schneider, J.G., Grundy, J., Han, J., Goyal, M. (2016b). Opaque service virtualisation: a practical tool for emulating endpoint systems. In: International Conference on Software Engineering Companion.
Walkinshaw, N., & Bogdanov, K. (2008). Inferring finite-state models with temporal constraints. In: International Conference on Automated Software Engineering.
Walkinshaw, N., Taylor, R., Derrick, J. (2016). Inferring extended finite state machine models from software executions. Empirical Software Engineering, 21(3), 811–853.
Xu, A., Liu, Z., Guo, Y., Sinha, V., Akkiraju, R. (2017). A new chatbot for customer service on social media. In: Conference on Human Factors in Computing Systems.
Zhou, X., Hu, B., Chen, Q., Tang, B., Wang, X. (2015). Answer sequence learning with neural networks for answer selection in community question answering. arXiv:1506.06490.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Enişer, H.F., Sen, A. Virtualization of stateful services via machine learning. Software Qual J 28, 283–306 (2020). https://doi.org/10.1007/s11219-019-09468-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-019-09468-z