Abstract
Model inference is a form of systematic testing of black-box systems while learning at the same time a model of their behaviour. In this paper, we study the impact of W-set reduction in hW-inference, an inference algorithm for learning models from scratch. hW-inference relies on progressively extending a sequence h into a homing sequence for the system, and a set W of separating sequences into a fully characterizing set. Like most other inference algorithms, it elaborates intermediate conjectures which can be refined through counterexamples provided by an oracle. We observed that the size of the W-set could vary by an order of magnitude when using random counterexamples. Consequently, the length of the test suite is hugely impacted by the size variation of the W-set. Whereas the original hW-inference algorithm keeps increasing the W-set until it is characterizing, we propose reassessing the set and pruning it based on intermediate conjectures. This can lead to a shorter test suite to thoroughly learn a model. We assess the impact of reduction methods on a self-scanning system as used in supermarkets, where the model we get is a finite state machine with 121 states and over 1800 transitions, leading to an order of magnitude of around a million events for the trace length of the inference.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Actually, the 121 states of the FSM can be characterized by a W-set with as few as 4 sequences totalling 11 inputs or 3 sequences totalling 23 inputs (and it might even not be minimal).
- 2.
The SIMPA software can be downloaded from:
http://vasco.imag.fr/tools/SIMPA or directly from
- 3.
References
Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 2, 87–106 (1987)
Bennaceur, A., Hähnle, R., Meinke, K. (eds.): Machine Learning for Dynamic Software Analysis: Potentials and Limits – International Dagstuhl Seminar 16172, Dagstuhl Castle, Germany, April 24–27, 2016, Revised Papers, Volume 11026 of Lecture Notes in Computer Science. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-96562-8
Bremond, N., Groz, R.: Case studies in learning models and testing without reset. In: 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops, AMOST 2019, ICST Workshops 2019, Xi’an, China, 22 April 2019, pp. 40–45 (2019)
Fujiwara, S., von Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6), 591–603 (1991)
Gill, A.: Introduction to the Theory of Finite-State Machines. McGraw-Hill, New York (1962)
Groz, R., Bremond, N., Simao, A.: Using adaptive sequences for learning non-resettable FSMs. In: Unold, O., Dyrka, W., Wieczorek, W. (eds.) Proceedings of the 14th International Conference on Grammatical Inference 2018, Volume 93 of Proceedings of Machine Learning Research, pp. 30–43. PMLR, February 2019
Groz, R., Bremond, N., Simao, A., Oriat, C.: \(hW\)-inference: a heuristic approach to retrieve models through black box testing. J. Syst. Softw. 159, 110426 (2020)
Irfan, M.N., Oriat, C., Groz, R.: Angluin style finite state machine inference with non-optimal counterexamples. In: MIIT, pp. 11–19. ACM, New York (2010)
Isberner, M., Howar, F., Steffen, B.: The TTT algorithm: a redundancy-free approach to active automata learning. In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 307–322. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11164-3_26
Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - a survey. Proc. IEEE 84(8), 1090–1123 (1996)
Moore, E.F.: Gedanken-experiments on sequential machines. In: Shannon, C.E., McCarthy, J. (eds.) Automata Studies (AM-34), vol. 34, pp. 129–154. Princeton University Press (1956)
Niese, O.: An integrated approach to testing complex systems. Ph.D. thesis, University of Dortmund (2003)
Peled, D., Vardi, M.Y., Yannakakis, M.: Black box checking. In: Wu, J., Chanson, S.T., Gao, Q. (eds.) Formal Methods for Protocol Engineering and Distributed Systems. IAICT, vol. 28, pp. 225–240. Springer, Boston (1999). https://doi.org/10.1007/978-0-387-35578-8_13
Petrenko, A., Li, K., Groz, R., Hossen, K., Oriat, C.: Inferring approximated models for systems engineering. In: HASE 2014, Miami, Florida, USA, pp. 249–253 (2014)
Shahbaz, M., Groz, R.: Inferring mealy machines. In: Cavalcanti, A., Dams, D.R. (eds.) FM 2009. LNCS, vol. 5850, pp. 207–222. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-05089-3_14
Utting, M., Legeard, B., Dadeau, F., Tamagnan, F., Bouquet, F.: Identifying and generating missing tests using machine learning on execution traces. In: 2020 IEEE International Conference On Artificial Intelligence Testing (AITest), pp. 83–90 (2020)
Vaandrager, F.: Model learning. Commun. ACM 60(2), 86–95 (2017)
Vasilievskii, M.P.: Failure diagnosis of automata. Cybern. Syst. Anal. 9, 653–665 (1973). https://doi.org/10.1007/BF01068590
Acknowledgments
The internship of the first author was funded by the French National Research Agency: ANR PHILAE project (ANR-18-CE25-0013). The second and the fifth authors were funded by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP: 2013/07375-0).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Halm, M., Braz, R.S., Groz, R., Oriat, C., Simao, A. (2022). Improving Model Inference via W-Set Reduction. In: Clark, D., Menendez, H., Cavalli, A.R. (eds) Testing Software and Systems. ICTSS 2021. Lecture Notes in Computer Science, vol 13045. Springer, Cham. https://doi.org/10.1007/978-3-031-04673-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-04673-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04672-8
Online ISBN: 978-3-031-04673-5
eBook Packages: Computer ScienceComputer Science (R0)
