Michael Szvetits
ABSTRACT
Models describe a software system on an abstraction level higher than its actual implementation. Recent research results show that bringing models and a running system closer together by establishing traceability links between recorded runtime events and corresponding model elements improves the analysis performance of human observers when assessing the behaviour of the running system. Despite these results, common techniques for analyzing runtime events are rarely integrated into the models that are used for assessing the system behaviour from a high-level perspective. This paper presents a controlled experiment where model-integrated analysis facilities are compared with a more traditional analysis approach based on SQL queries to a system's database in terms of correctness and completion time of analysis tasks. The results show that model-integrated analyses allow analysts to give more correct answers to questions about the system behaviour, but provide no improvement of the time spent for completing the analysis tasks.
- Gordon Blair, Nelly Bencomo, and Robert B. France. 2009. [email protected]. Computer 42, 10 (Oct. 2009), 22--27. Google Scholar
Digital Library
- James Cheney, Sam Lindley, and Philip Wadler. 2013. A Practical Theory of Language-integrated Query. In Proceedings of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP '13). ACM, New York, NY, USA, 403--416. Google ScholarDigital Library
- Norman Cliff. 1996. Ordinal Methods for Behavioral Data Analysis. Erlbaum. https://books.google.at/books?id=bIJFvgAACAAJGoogle Scholar
- Thomas D. Cook and D.T. Campbell. 1979. Quasi-experimentation: design & analysis issues for field settings. Rand McNally College. https://books.google.at/books?id=68HynQEACAAJGoogle Scholar
- Alan J. Demers, Johannes Gehrke, Biswanath Panda, Mirek Riedewald, Varun Sharma, and Walker M. White. 2007. Cayuga: A General Purpose Event Monitoring System. In CIDR'07. 412--422.Google Scholar
- Françoise Fabret, H. Arno Jacobsen, François Llirbat, Joăo Pereira, Kenneth A. Ross, and Dennis Shasha. 2001. Filtering Algorithms and Implementation for Very Fast Publish/Subscribe Systems. SIGMOD Rec. 30, 2 (May 2001), 115--126. Google ScholarDigital Library
- Ronald Aylmer Fisher. 1925. Statistical Methods For Research Workers. Edinburgh Oliver & Boyd.Google Scholar
- William Sealy Gosset. 1908. The Probable Error of a Mean. Biometrika 6, 1 (March 1908), 1--25. Originally published under the pseudonym "Student".Google ScholarCross Ref
- Carmine Gravino, Giuseppe Scanniello, and Genoveffa Tortora. 2015. Source-code Comprehension Tasks Supported by UML Design Models. J. Vis. Lang. Comput. 28, C (June 2015), 23--38. Google ScholarDigital Library
- Thomas Haitzer and Uwe Zdun. 2013. Controlled Experiment on the Supportive Effect of Architectural Component Diagrams for Design Understanding of Novice Architects. In Proceedings of the 7th European Conference on Software Architecture (ECSA'13). Springer-Verlag, Berlin, Heidelberg, 54--71. Google ScholarDigital Library
- John Hutchinson, Jon Whittle, Mark Rouncefield, and Steinar Kristoffersen. 2011. Empirical assessment of MDE in industry. In Software Engineering (ICSE), 2011 33rd International Conference on. 471--480. Google ScholarDigital Library
- Jaakko Järvi and John Freeman. 2010. C++ Lambda Expressions and Closures. Sci. Comput. Program. 75, 9 (Sept. 2010), 762--772. Google ScholarDigital Library
- Muhammad Atif Javed and Uwe Zdun. 2014. The Supportive Effect of Traceability Links in Architecture-Level Software Understanding: Two Controlled Experiments. In Software Architecture (WICSA), 2014IEEE/IFIP Conference on. 215--224. Google ScholarDigital Library
- Dileepa Jayathilake. 2012. Towards structured log analysis. In Computer Science and Software Engineering (JCSSE), 2012 International Joint Conference on. 259--264.Google ScholarCross Ref
- Barbara A. Kitchenham, Lech Madeyski, David Budgen, Jacky Keung, Pearl Brereton, Stuart Charters, Shirley Gibbs, and Amnart Pohthong. 2017. Robust Statistical Methods for Empirical Software Engineering. Empirical Software Engineering 22, 2 (01 Apr 2017), 579--630. Google ScholarDigital Library
- Barbara A. Kitchenham, Shari Lawrence Pfleeger, Lesley M. Pickard, Peter W. Jones, David C. Hoaglin, Khaled El Emam, and Jarrett Rosenberg. 2002. Preliminary Guidelines for Empirical Research in Software Engineering. IEEE Trans. Softw. Eng. 28, 8 (Aug. 2002), 721--734. Google ScholarDigital Library
- Henry B. Mann and Donald R. Whitney. 1947. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. Ann. Math. Statist. 18, 1 (03 1947), 50--60.Google Scholar
- HaiTao Mei, Ian Gray, and Andy Wellings. 2016. Real-Time Stream Processing in Java. In Reliable Software Technologies -- Ada-Europe 2016, Marko Bertogna, Luis Miguel Pinho, and Eduardo Quiñones (Eds.). Springer International Publishing, Cham, 44--57. Google ScholarDigital Library
- Rajeev Motwani, Jennifer Widom, Arvind Arasu, Brian Babcock, Shivnath Babu, Mayur Datar, Gurmeet Manku, Chris Olston, Justin Rosenstein, and Rohit Varma. 2002. Query Processing, Resource Management, and Approximation in a Data Stream Management System. Technical Report 2002--41. Stanford InfoLab. http://ilpubs.stanford.edu:8090/549/Google Scholar
- Cornelis Joost van Rijsbergen. 1979. Information Retrieval (2nd ed.). Butterworth-Heinemann, Newton, MA, USA.Google Scholar
- Mojtaba Shahin, Peng Liang, and Mohammad Reza Khayyambashi. 2010. Rationale Visualization of Software Architectural Design Decision Using Compendium. In Proceedings of the 2010 ACM Symposium on Applied Computing (SAC '10). ACM, New York, NY, USA, 2367--2368. Google ScholarDigital Library
- Mojtaba Shahin, Peng Liang, and Zengyang Li. 2011. Architectural Design Decision Visualization for Architecture Design: Preliminary Results of a Controlled Experiment. In Proceedings of the 5th European Conference on Software Architecture: Companion Volume (ECSA '11). ACM, New York, NY, USA, Article 2, 8 pages. Google ScholarDigital Library
- Michael Szvetits and Uwe Zdun. 2015. Reusable event types for models at runtime to support the examination of runtime phenomena. In 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS). 4--13.Google ScholarCross Ref
- Michael Szvetits and Uwe Zdun. 2016. Controlled Experiment on the Comprehension of Runtime Phenomena Using Models Created at Design Time. In Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems (MODELS 16). ACM, New York, NY, USA, 151--161. Google ScholarDigital Library
- Huy Tran, Uwe Zdun, and Schahram Dustdar. 2011. VbTrace: using view-based and model-driven development to support traceability in process-driven SOAs. Software & Systems Modeling 10, 1 (2011), 5--29. Google ScholarDigital Library
- Jon Whittle, John Hutchinson, and Mark Rouncefield. 2014. The State of Practice in Model-Driven Engineering. IEEE Software 31, 3 (May 2014), 79--85.Google ScholarCross Ref
- Claes Wohlin, Martin Höst, and Kennet Henningsson. 2003. Empirical Research Methods in Software Engineering. In Empirical Methods and Studies in Software Engineering, Reidar Conradi and AlfInge Wang (Eds.). Lecture Notes in Computer Science, Vol. 2765. Springer Berlin Heidelberg, 7--23.Google Scholar
- Eugene Wu, Yanlei Diao, and Shariq Rizvi. 2006. High-performance Complex Event Processing over Streams. In Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data (SIGMOD '06). ACM, New York, NY, USA, 407--418. Google ScholarDigital Library
- Detlef Zimmer and Rainer Unland. 1999. On the semantics of complex events in active database management systems. In Data Engineering, 1999. Proceedings., 15th International Conference on. 392--399. Google ScholarDigital Library
Index Terms
- Model-Integrated Queries for the Analysis of Runtime Events: A Controlled Experiment
Recommendations
Controlled experiment on the comprehension of runtime phenomena using models created at design time
MODELS '16: Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and SystemsUtilizing models for software construction is a well-studied research topic. Recent research investigates the integration of models into running systems to provide additional information about the system configuration at runtime. While this additional ...
A Pattern Language for Manual Analysis of Runtime Events Using Design Models
EuroPLoP '18: Proceedings of the 23rd European Conference on Pattern Languages of ProgramsModeling is an important activity in the software development process whose output are design artefacts that describe the resulting software from a high-level perspective. Recent research investigates the role of models at runtime and the results ...
Reusable event types for models at runtime to support the examination of runtime phenomena
MODELS '15: Proceedings of the 18th International Conference on Model Driven Engineering Languages and SystemsToday's software is getting more and more complex and harder to understand. Models help to organize knowledge and emphasize the structure of a software at a higher abstraction level. While the usage of model-driven techniques is widely adopted during ...
Comments