Skip to main content
SpringerLink
Log in

Model-based testing of autonomous robots using TestIt

  • Original Article
  • Published:
Journal of Reliable Intelligent Environments Aims and scope Submit manuscript

Abstract

This paper presents a testing toolkit named TestIt. Its main goal is to provide tools for automated model-based testing of autonomous multi-robot systems to verify long-term autonomy in various applications including those of smart city environments and smart buildings. The main novelty of the presented toolkit is the open and scalable multi-pipeline architecture that enables incorporation of test development and execution tools from various vendors. The second main contribution is adaptive test optimization technique that takes advantage of the proposed multi-pipeline architecture where testing threads can communicate and coordinate the test runs based on their cooperatively collected test performance data. The usability of TestIt for test generation, testing process performance improvement and optimization in multi-robot autonomous navigation context is demonstrated using a smart building robotic security system case study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. http://www.testributor.com/.

  2. https://github.com/floweisshardt/atf.

  3. http://rosin-project.eu/wp-content/uploads/D3.1-Quality-Assurance-Process-and-Community-Management-in-ROS.pdf.

  4. http://wiki.ros.org/roslint.

  5. https://github.com/GertKanter/testit.

  6. https://www.docker.com.

  7. https://yaml.org/.

  8. https://www.turtlebot.com/.

  9. https://gitlab.com/GertKanter/testit-patrol-sut.

  10. https://gitlab.com/GertKanter/testit-patrol-aws.

References

  1. Quigley M et al (2009) ROS: an open-source robot operating system. In: ICRA workshop on open source software, 2009(online). https://www.willowgarage.com/sites/default/files/icraoss09-ROS.pdf

  2. Kumar S, Vealey T, Srivastava H (2016) Security in internet of things: challenges, solutions and future directions. 49th Hawaii international conference on system sciences (HICSS). Koloa, HI, pp 5772–5781

    Chapter  Google Scholar 

  3. Deshmukh J, Horvat M, Jin X, Majumdar R, Prabhu V (2017) Testing cyber-physical systems through Bayesian optimization. ACM Trans Embed Comput Syst 16(5s):1–18. https://doi.org/10.1145/3126521 (article 170, Sep 2017)

    Article  Google Scholar 

  4. Bartocci E et al (2018) Specification-based monitoring of cyber-physical systems: a survey on theory, tools and applications. In: Bartocci E, Falcone Y (eds) Lectures on runtime verification. Lecture notes in computer science, vol 10457. Springer, Cham

  5. Larsen KG, Pettersson P, Yi W (1997) Uppaal in a nutshell. Int J Softw Tools Technol Transf 1(1–2):134–152. https://doi.org/10.1007/s100090050010

    Article  MATH  Google Scholar 

  6. Kapinski J, Deshmukh JV, Jin X, Ito H, Butts K (2016) Simulation-based approaches for verification of embedded control systems: an overview of traditional and advanced modeling, testing, and verification techniques. IEEE Control Syst Mag 36(6):45–64

    Article  MathSciNet  Google Scholar 

  7. Utting M, Pretschner A, Legeard B (2012) A taxonomy of model based testing approaches. Softw Test Verif Reliab 22(5):297–312. https://doi.org/10.1002/stvr.456. Accessed 18 May 2015

  8. Tretmans J (1996) Test generation with inputs, outputs, and quiescence. In: TACAS, vol 1055 of LNCS, p 127–146. Springer, New York

  9. Vain J, Kanter G, Srinivasan S (2017) Model based testing of distributed time critical systems. In: 2017 6th international conference on reliability, Infocom technologies and optimization (ICRITO) (trends and future directions), 20–22 Sep 2017, Noida

  10. Baier C, Katoen J-P (2008) Principles of model checking (representation and mind series). MIT Press, New York

  11. Anier A, Vain J, Tsiopolous L (2017) DTRON: a tool for distributed model-based testing of time critical applications. Est Acad Sci 66:75–88

    Article  Google Scholar 

  12. Larsen KG, Mikucionis M, Nielsen B, Skou A (2005) Testing real-time embedded software using UPPAAL-TRON: an industrial case study. In: EMSOFT

  13. Santos A, Cunha A, Macedo N, Lourenço C (2016) A framework for quality assessment of ROS repositories. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), Daejeon, pp 4491–4496

  14. Behrmann G, David A, Larsen KG (2004) A tutorial on UPPAAL. In: Bernardo M, Corradini F (eds) SFM-RT 2004. LNCS, vol 3185, pp 200–237. Springer, New York

  15. Štill V, Ročkai P, Barnat J (2016) DIVINE: explicit-state LTL model checker. Tools and algorithms for the construction and analysis of systems. Springer, Berlin

    Google Scholar 

  16. Ernits J, Roo R, Jacky J, Veanes M (2009) Model-based testing of web applications using NModel. TESTCOM/FATES 2009. Springer, New York

    Google Scholar 

  17. Hawes N et al (2017) The STRANDS project: long-term autonomy in everyday environments. IEEE Robot Autom Mag 24(3):146–156

    Article  MathSciNet  Google Scholar 

  18. Gummel A (2018) Modelbased testing with TestIt: the robot operating system case-study. MSc Thesis, Tallinn University of Technology, Tallinn, Estonia. https://digi.lib.ttu.ee/i/?10616

  19. Vaughan R (2008) Massively multiple robot simulations in stage. Swarm Intell 2(2–4):189–208 (Springer)

    Article  Google Scholar 

Download references

Acknowledgements

This research was partially supported by the Estonian Ministry of Education and Research institutional research Grant no. IUT33-13.

Author information

Authors and Affiliations

  1. Department of Software Science, Tallinn University of Technology, Akadeemia tee 15a, Tallinn, Estonia

    Gert Kanter & Jüri Vain

Authors
  1. Gert Kanter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jüri Vain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gert Kanter.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kanter, G., Vain, J. Model-based testing of autonomous robots using TestIt. J Reliable Intell Environ 6, 15–30 (2020). https://doi.org/10.1007/s40860-019-00095-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40860-019-00095-w

Keywords

Search

Navigation