Skip to main content
SpringerLink
Log in

Testing cockpit display systems of aircraft using a model-based approach

  • Special Section Paper
  • Published:
Software and Systems Modeling Aims and scope Submit manuscript

Abstract

Avionics are highly critical systems that require extensive testing to comply with international safety standards. Cockpit display systems (CDS) are a mandatory part of modern cockpits of both manned and unmanned aircraft. The information from various avionics components is displayed on CDS using a variety of flight instruments. An important part of testing avionics systems is to evaluate whether the displayed information on the CDS is correct or not. A common industrial practice is to manually test CDS, which is time-consuming, labor-intensive, and error-prone. In this paper, we propose a model-based approach to automate the CDS testing of aircraft. The proposed approach tests the CDS at two levels: (i) at the system level to verify that the CDS are working correctly and (ii) at system integration level of CDS when these are integrated with various avionics components. As a part of our approach, we develop a UML profile to model various elements of the CDS. The models are then used to support the automated testing process. We evaluate our approach on two industrial case studies, the first case study represents a primary flight display (PFD) of an aircraft and the second one is the CDS of the ground control station (GCS-CDS) of an unmanned aerial vehicle. The evaluation results show that three potential faults are identified in the PFD and four major faults are found in the GCS-CDS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. https://cockpitsonic.de/product/a320-table-trainer/.

  2. https://ardupilot.org/planner/docs/mission-planner-overview.html.

  3. https://github.com/hassansartaj/models19.

  4. https://www.eclipse.org/modeling/emf/.

  5. https://www.x-plane.com/.

  6. https://github.com/hassansartaj/cdst-toolkit.

  7. http://ardupilot.org/dev/docs/sitl-simulator-software-in-the-loop.html.

  8. http://ardupilot.org/plane/index.html.

  9. http://ardupilot.org/planner/index.html.

  10. https://github.com/hassansartaj/cdst-toolkit/tree/master/datasets.

References

  1. Alegroth, E., Nass, M., Olsson, H.H.: Jautomate: A tool for system-and acceptance-test automation. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, pp. 439–446. IEEE (2013)

  2. Amalfitano, D., Fasolino, A.R., Tramontana, P., De Carmine, S., Memon, A.M.: Using GUI ripping for automated testing of android applications. In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, pp. 258–261. ACM (2012)

  3. ANSYS: Scade suite (2014). https://www.ansys.com/products/embedded-software/ansys-scade-suite

  4. Antoniol, G., Briand, L.C., Di Penta, M., Labiche, Y.: A case study using the round-trip strategy for state-based class testing. In: 13th International Symposium on Software Reliability Engineering, 2002. Proceedings., p. 269. IEEE (2002)

  5. ARP4754, S.: Certification considerations for highly-integrated or complex aircraft systems. SAE, Warrendale (1996)

  6. Aviation, D.: List of arinc661 useful widgets (2017). http://j661.sourceforge.net/

  7. Baek, Y.M., Bae, D.H.: Automated model-based android GUI testing using multi-level GUI comparison criteria. In: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, pp. 238–249. ACM (2016)

  8. Behnken, D., Salgado, R.: Automated testing of cockpit display visual aspects. AUTOTESTCON IEEE 1(1), 551–551 (1992)

    Google Scholar 

  9. Berndt, J.: Jsbsim: An open source flight dynamics model in c++. In: AIAA Modeling and Simulation Technologies Conference and Exhibit, p. 4923 (2004)

  10. Binder, R.: Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Professional, Boston (2000)

    Google Scholar 

  11. Bradski, G., Kaehler, A.: Opencv. Dr Dobb’s journal of software tools 3 (2000)

  12. Campos, J.C., Fayollas, C., Martinie, C., Navarre, D., Palanque, P., Pinto, M.: Systematic automation of scenario-based testing of user interfaces. In: Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 138–148. ACM (2016)

  13. Campos, J.C., Fayollas, C., Gonçalves, M., Martinie, C., Navarre, D., Palanque, P., Pinto, M.: A more intelligent test case generation approach through task models manipulation. Proc. ACM Hum. Comput. Interact. 1(1), 9 (2017)

    Google Scholar 

  14. Catelani, M., Ciani, L., Venzi, M., Barile, G.: Environmental tests and optical measurements in the validation process of TFT-LCD for avionics applications. In: Metrology for Aerospace (MetroAeroSpace), 2015 IEEE, pp. 421–425. IEEE (2015)

  15. Chang, T.H., Yeh, T., Miller, R.C.: Gui testing using computer vision. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1535–1544. ACM (2010)

  16. Choi, Y.M., Lim, D.J.: Automatic feasible transition path generation from UML state chart diagrams using grouping genetic algorithms. Inf. Softw. Technol. 94, 38–58 (2018)

    Article  Google Scholar 

  17. Damilano, L., Guglieri, G., Quagliotti, F., Sale, I., Lunghi, A.: Ground control station embedded mission planning for UAS. J. Intell. Robot. Syst. 69(1–4), 241–256 (2013)

    Article  Google Scholar 

  18. Eclipse: Model development tools: Eclipse ocl (2019). https://projects.eclipse.org/projects/modeling.mdt.ocl

  19. El-Far, I.K., Whittaker, J.A.: Model-based software testing. Encyclopedia of Software Engineering (2002)

  20. Ferrell, T.K., Ferrell, U.D.: Rtca do-178c/eurocae ed-12c. In: Digital Avionics Handbook (2017)

  21. Garousi, V., Afzal, W., Caglar, A., Isik, I.B., Baydan, B., Caylak, S., Boyraz, A.Z., Yolacan, B., Herkiloglu, K.: Comparing automated visual GUI testing tools: an industrial case study. In: Proceedings of the 8th ACM SIGSOFT International Workshop on Automated Software Testing, pp. 21–28. ACM (2017)

  22. Group, I., et al.: Ieee standard classification for software anomalies. IEEE Std 1044-2009 (Revision of IEEE Std 1044-1993) 1044(2), 1–23 (2010)

  23. Guerra, M., Rhudy, M., Gu, Y., Seanor, B., Napolitano, M.: Mobile ground control station development for fault tolerant UAV research. In: AIAA Guidance, Navigation, and Control Conference, p. 4544 (2012)

  24. Iftikhar, S., Iqbal, M.Z., Khan, M.U., Mahmood, W.: An automated model based testing approach for platform games. In: Model Driven Engineering Languages and Systems (MODELS), 2015 ACM/IEEE 18th International Conference on, pp. 426–435. IEEE (2015)

  25. Iqbal, M.Z., Ali, S., Yue, T., Briand, L.: Applying UML/MARTE on industrial projects: challenges, experiences, and guidelines. Softw. Syst. Model. 14(4), 1367–1385 (2015)

    Article  Google Scholar 

  26. Iqbal, M.Z., Sartaj, H., Khan, M.U., Haq, F.U., Qaisar, I.: A model-based testing approach for cockpit display systems of avionics. In: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 67–77. IEEE (2019)

  27. Kayayurt, B., Yayla, I., Yapici, A., Küçükoğuz, C.: Ground control station avionics software development in ANKA UAV. In: 2011 IEEE/AIAA 30th Digital Avionics Systems Conference, pp. 5B6–1. IEEE (2011)

  28. Khan, M.U., Sartaj, H., Iqbal, M.Z., Usman, M., Arshad, N.: Aspectocl: using aspects to ease maintenance of evolving constraint specification. Empir. Softw. Eng. 24(4), 2674–2724 (2019)

    Article  Google Scholar 

  29. Lelli, V., Blouin, A., Baudry, B.: Classifying and qualifying GUI defects. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–10. IEEE (2015)

  30. Mor, N., Wolf, L.: Confidence prediction for lexicon-free OCR. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 218–225. IEEE (2018)

  31. Nguyen, B.N., Robbins, B., Banerjee, I., Memon, A.: Guitar: an innovative tool for automated testing of GUI-driven software. Autom. Softw. Eng. 21(1), 65–105 (2014)

    Article  Google Scholar 

  32. OMG: Object constraint language specification v2.4. Object Management Group Inc (2014). http://www.omg.org/spec/OCL/2.4/

  33. OMG: Uml. unified modeling language specification, version 2.5.1. Object Management Group Inc (2017). http://www.omg.org/spec/UML/2.5.1/

  34. Patel, C., Patel, A., Patel, D.: Optical character recognition by open source OCR tool tesseract: a case study. Int. J. Comput. Appl. 55(10), 50–56 (2012)

    Google Scholar 

  35. Presagis: Vaps xt (2018). https://www.presagis.com/en/product/vaps-xt/

  36. Salton, G., McGill, M.J.: Introduction to Modern Information, Retrieval. McGraw-Hill Inc, New York (1986)

    MATH  Google Scholar 

  37. Sartaj, H., Iqbal, M.Z., Jilani, A.A.A., Khan, M.U.: A search-based approach to generate mc/dc test data for OCL constraints. In: International Symposium on Search Based Software Engineering, pp. 105–120. Springer (2019)

  38. Sartaj, H., Iqbal, M.Z., Khan, M.U.: Cdst: A toolkit for testing cockpit display systems. In: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), pp. 436–441. IEEE (2020)

  39. Smith, R.: An overview of the tesseract OCR engine. In: Ninth International Conference on Document Analysis and Recognition, 2007. ICDAR 2007, vol. 2, pp. 629–633, IEEE (2007)

  40. Smith, R., Antonova, D., Lee, D.S.: Adapting the tesseract open source OCR engine for multilingual OCR. In: Proceedings of the International Workshop on Multilingual OCR, p. 1. ACM (2009)

  41. Specification, A.: 661–3 Cockpit Display System Interfaces to User Systems. Aeronautical Radio Inc, Annapolis (2007)

    Google Scholar 

  42. Ulbig, P., Müller, D., Torens, C., Insaurralde, C.C., Stripf, T., Durak, U.: Flight simulator-based verification for model-based avionics applications on multi-core targets. In: AIAA Scitech 2019 Forum, p. 1976 (2019)

  43. Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Elsevier, Amsterdam (2010)

    Google Scholar 

  44. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, Berlin (2012)

    Book  Google Scholar 

  45. Yang, W., Prasad, M.R., Xie, T.: A grey-box approach for automated GUI-model generation of mobile applications. In: International Conference on Fundamental Approaches to Software Engineering, pp 250–265. Springer (2013)

  46. Yeh, T., Chang, T.H., Miller, R.C.: Sikuli: using GUI screenshots for search and automation. In: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, pp 183–192. ACM (2009)

  47. Yin, Y., Liu, B., Ni, H.: Real-time embedded software testing method based on extended finite state machine. J. Syst. Eng. Electron. 23(2), 276–285 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

This research work is supported through a research grant titled ‘Establishment of National Centre of Robotics and Automation (NCRA)’ by Higher Education Commission (HEC), Pakistan.

Author information

Authors and Affiliations

  1. Quest Lab, Department of Computer Science, National University of Computer and Emerging Sciences, Islamabad, Pakistan

    Hassan Sartaj, Muhammad Zohaib Iqbal & Muhammad Uzair Khan

  2. UAV Dependability Lab, National Centre of Robotics and Automation (NCRA), Islamabad, Pakistan

    Hassan Sartaj, Muhammad Zohaib Iqbal & Muhammad Uzair Khan

Authors
  1. Hassan Sartaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Zohaib Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Uzair Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hassan Sartaj.

Additional information

Communicated by Tao Yue, Man Zhang, and Silvia Abrahao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sartaj, H., Iqbal, M.Z. & Khan, M.U. Testing cockpit display systems of aircraft using a model-based approach. Softw Syst Model 20, 1977–2002 (2021). https://doi.org/10.1007/s10270-020-00844-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-020-00844-z

Keywords

Search

Navigation