Abstract
Multicore processors have now become the norm. However, for many embedded real-time systems their use introduces challenges in verification as their shared components are potential channels for interference. Of particular interest is the determination for each task of its worst case (longest) execution time (WCET). In this paper, we investigate the effectiveness of a variety of metaheuristic search algorithms for dynamically finding extreme execution times of tasks executing on a multicore processor. Over finite search spaces, these are shown to perform considerably better than randomly generated test inputs and the work reveals significant performance differences between the various algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The GSL used in this paper is version 2.1, while the current one is version 2.4 (released on 19 June 2017 [30]).
- 2.
Single-state methods have recently been included in ECJ version 25.
- 3.
The time from another clock source, namely CLOCK_REALTIME, may leap forward or even backward after a time adjustment.
References
Burns, A., McDermid, J.A.: Real-time safety-critical systems: analysis and synthesis. Softw. Eng. J. 9(6), 267–281 (1994)
Engel, A.: Verification, Validation and Testing of Engineered Systems. Wiley Series in Systems Engineering and Management, 2nd edn. John Wiley & Sons, Hoboken (2010)
Pohlheim, H., Wegener, J.: Testing the temporal behavior of real-time software modules using extended evolutionary algorithms. In: Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation - vol. 2, GECCO 1999, p. 1795. Morgan Kaufmann Publishers Inc., San Francisco (1999)
Graydon, P., Bate, I.: Realistic safety cases for the timing of systems. Comput. J. 57(5), 759–774 (2014)
Wilhelm, R., Engblom, J., Ermedahl, A., Holsti, N., Thesing, S., Whalley, D., Bernat, G., Ferdinand, C., Heckmann, R., Mitra, T., Mueller, F., Puaut, I., Puschner, P., Staschulat, J., Stenström, P.: The worst-case execution-time problem - overview of methods and survey of tools. ACM Trans. Embed. Comput. Syst. 7(3), 36:1–36:53 (2008). http://doi.acm.org/10.1145/1347375.1347389
Tracey, N., Clark, J., McDermid, J., Mander, K.: A search-based automated test-data generation framework for safety-critical systems. In: Henderson, P. (ed.) Systems Engineering for Business Process Change: New Directions, pp. 174–213. Springer, New York (2002). https://doi.org/10.1007/978-1-4471-0135-2_12. http://dl.acm.org/citation.cfm?id=763012.763025
McMinn, P.: Search-based software testing: past, present and future. In: Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2011, pp. 153–163. IEEE Computer Society, Washington (2011). http://dx.doi.org/10.1109/ICSTW.2011.100
Poulding, S., Clark, J.A.: The problems with multi-core: Challenges for the critical systems community. Technical report, Department of Computer Science, University of York, York, United Kingdom (2012). version 0.1
Saidi, S., Ernst, R., Uhrig, S., Theiling, H., de Dinechin, B.D.: The shift to multicores in real-time and safety-critical systems. In: 2015 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), pp. 220–229, October 2015
Boussad, I., Lepagnot, J., Siarry, P.: A survey on optimization metaheuristics. Inf. Sci. 237, 82–117 (2013). http://www.sciencedirect.com/science/article/pii/S0020025513001588
Afzal, W., Torkar, R., Feldt, R.: A systematic review of search-based testing for non-functional system properties. Inf. Softw. Technol. 51(6), 957–976 (2009). http://www.sciencedirect.com/science/article/pii/S0950584908001833
Wegener, J., Grimm, K., Grochtmann, M., Sthamer, H., Jones, B.: Systematic testing of real-time systems. In: 4th International Conference on Software Testing Analysis and Review (EuroSTAR 1996) (1996)
Wegener, J., Sthamer, H., Jones, B.F., Eyres, D.E.: Testing real-time systems using genetic algorithms. Softw. Qual. J. 6(2), 127–135 (1997). https://doi.org/10.1023/A:1018551716639
Tracey, N., Clark, J.A., Mander, K.: The way forward for unifying dynamic test-case generation: The optimisation-based approach. In: Proceedings of the IFIP International Workshop on Dependable Computing and Its Applications (DCIA), York (1998)
Alander, J.T., Mantere, T., Moghadampour, G., Matila, J.: Searching protection relay response time extremes using genetic algorithm-software quality by optimization. In: 1997 Fourth International Conference on Advances in Power System Control, Operation and Management, APSCOM-1997, (Conf. Publ. No. 450), vol. 1, pp. 95–99, November 1997
Gross, H.G., Jones, B.F., Eyres, D.E.: Structural performance measure of evolutionary testing applied to worst-case timing of real-time systems. IEE Proc. Softw. 147(2), 25–30 (2000)
Wegener, J., Mueller, F.: A comparison of static analysis and evolutionary testing for the verification of timing constraints. Real-Time Syst. 21(3), 241–268 (2001). https://doi.org/10.1023/A:1011132221066
Tlili, M., Wappler, S., Sthamer, H.: Improving evolutionary real-time testing. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, GECCO 2006, pp. 1917–1924. ACM, New York (2006). http://doi.acm.org/10.1145/1143997.1144316
O’Sullivan, M., Vössner, S., Wegener, J.: Testing temporal correctness of real-time systems-a new approach using genetic algorithms and cluster analysis. In: Proceedings of the 6th European Conference on Software Testing, Analysis and Review (EuroSTAR 1998), Munich, Germany (1998)
Groß, H.G.: A prediction system for dynamic optimisation-based execution time analysis. In: Proceedings of the First International Workshop on Software Engineering using Metaheuristic Innovative Algorithms (SEMINAL), ISCE 2001, Toronto, Canada (2001)
Harman, M., Jia, Y., Zhang, Y.: Achievements, open problems and challenges for search based software testing. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–12, April 2015
Bate, I., Khan, U.: Wcet analysis of modern processors using multi-criteria optimisation. Empirical Softw. Eng. 16(1), 5–28 (2011). https://doi.org/10.1007/s10664-010-9133-9
Burger, D., Austin, T.M.: The simplescalar tool set, version 2.0. SIGARCH Comput. Archit. News. 25(3), 13–25 (1997). http://doi.acm.org/10.1145/268806.268810
Khan, U., Bate, I.: WCET analysis of modern processors using multi-criteria optimisation. In: 2009 1st International Symposium on Search Based Software Engineering, pp. 103–112, May 2009
Burke, E.K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., Qu, R.: Hyper-heuristics: a survey of the state of the art. J. Oper. Res. Soc. 64(12), 1695–1724 (2013). https://doi.org/10.1057/jors.2013.71
Talbi, E.G.: Metaheuristics: From Design to Implementation. Wiley Publishing, Hoboken (2009)
Luke, S.: Essentials of Metaheuristics, 2nd edn. Lulu (2013). http://cs.gmu.edu/~sean/book/metaheuristics/
Galassi, M., Davies, J., Theiler, J., Gough, B., Jungman, G., Alken, P., Booth, M., Rossi, F., Ulerich, R.: GNU Scientific Library Reference Manual, 3rd edn. Network Theory Limited, UK (2015). The GSL Team
The Apache Software Foundation: Commons math: The apache commons mathematics library (2016). http://commons.apache.org/proper/commons-math/. Accessed 29 Mar 2017
Free Software Foundation Inc: Gsl - gnu scientific library (2017). https://www.gnu.org/software/gsl/. Accessed 21 Oct 2017
Breshears, C.: The Art of Concurrency: A Thread Monkey’s Guide to Writing Parallel Applications. O’Reilly Media Inc., Sebastopol (2009)
IEEE: Standard for information technology-portable operating system interface (posix(r)) base specifications, issue 7. IEEE Std 1003.1, 2016 Edition (incorporates IEEE Std 1003.1-2008, IEEE Std 1003.1-2008/Cor 1–2013, and IEEE Std 1003.1-2008/Cor 2–2016), pp. 1–3957, September 2016
Sparger, J.: COSC 462 - Parallel Programming, Department of Electrical Engineering & Computer Science at the University of Tennessee, Knoxville. http://web.eecs.utk.edu/~jsparger/cosc462/pp/parallelProject3/. Accessed 09 Sept 2017
Weems, B.: CSE 4351: Parallel Processing, Department of Computer Science and Engineering at the University of Texas, Arlington. http://ranger.uta.edu/~weems/NOTES4351/cse4351.html. Accessed 09 Sept 2017
Stough, J.: Strategies for Introducing Parallelism with Python, Department of Computer Science at Washington and Lee University, Lexington. http://home.wlu.edu/~stoughj/SC13/. Accessed 09 Sept 2017
Freescale semiconductor: P4080 Development System User’s Guide. Technical report (2011)
Levy, M., Conte, T.M.: Embedded multicore processors and systems. IEEE Micro 29(3), 7–9 (2009)
White, D.R.: Software review: the ECJ toolkit. Genetic Program. Evol. Mach. 13(1), 65–67 (2012). http://dx.doi.org/10.1007/s10710-011-9148-z
Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983). http://science.sciencemag.org/content/220/4598/671
Abramson, D., Krishnamoorthy, M., Dang, H.: Simulated annealing cooling schedules for the school timetabling problem. Asia Pac. J. Oper. Res. 16 (1998)
Acknowledgements
Komsan Srivisut’s work is sponsored by the Royal Thai Government. John A. Clark is sponsored by the Engineering and Physical Sciences Research Council (EPSRC) under the Dynamic Adaptive Automated Software Engineering (DAASE) project EP/J017515/1. The authors would also like to thank Nathan Burles (also sponsored by DAASE) for his contribution to the development of execution infrastructure on the P4080.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Srivisut, K., Clark, J.A., Paige, R.F. (2018). Search-Based Temporal Testing in an Embedded Multicore Platform. In: Sim, K., Kaufmann, P. (eds) Applications of Evolutionary Computation. EvoApplications 2018. Lecture Notes in Computer Science(), vol 10784. Springer, Cham. https://doi.org/10.1007/978-3-319-77538-8_53
Download citation
DOI: https://doi.org/10.1007/978-3-319-77538-8_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77537-1
Online ISBN: 978-3-319-77538-8
eBook Packages: Computer ScienceComputer Science (R0)