skip to main content
10.1145/2635868.2635895acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

An empirical study on program comprehension with reactive programming

Published: 11 November 2014 Publication History

Abstract

Starting from the first investigations with strictly functional languages, reactive programming has been proposed as THE programming paradigm for reactive applications. The advantages of designs based on this style over designs based on the Observer design pattern have been studied for a long time. Over the years, researchers have enriched reactive languages with more powerful abstractions, embedded these abstractions into mainstream languages – including object-oriented languages – and applied reactive programming to several domains, like GUIs, animations, Web applications, robotics, and sensor networks. However, an important assumption behind this line of research – that, beside other advantages, reactive programming makes a wide class of otherwise cumbersome applications more comprehensible – has never been evaluated. In this paper, we present the design and the results of the first empirical study that evaluates the effect of reactive programming on comprehensibility compared to the traditional object-oriented style with the Observer design pattern. Results confirm the conjecture that comprehensibility is enhanced by reactive programming. In the experiment, the reactive programming group significantly outperforms the other group.

References

[1]
H. Abelson and G. J. Sussman. Structure and Interpretation of Computer Programs. 2nd. Cambridge, MA, USA: MIT Press, 1996.
[2]
E. Bainomugisha, A. L. Carreton, T. v. Cutsem, S. Mostinckx, and W. d. Meuter. “A survey on reactive programming.” In: ACM Comput. Surv. 45.4 (Aug. 2013), 52:1–52:34.
[3]
G. H. Cooper and S. Krishnamurthi. “Embedding Dynamic Dataflow in a Call-by-value Language.” In: Proceedings of the 15th European Conference on Programming Languages and Systems. ESOP’06. Vienna, Austria: Springer-Verlag, 2006, pp. 294–308.
[4]
B. Cornelissen, A. Zaidman, and A. van Deursen. “A Controlled Experiment for Program Comprehension Through Trace Visualization.” In: IEEE Trans. Softw. Eng. 37.3 (May 2011), pp. 341–355.
[5]
C. L. Corritore and S. Wiedenbeck. “Mental Representations of Expert Procedural and Object-oriented Programmers in a Software Maintenance Task.” In: Int. J. Hum.-Comput. Stud. 50.1 (Jan. 1999), pp. 61–83.
[6]
F. Détienne. Software Design—cognitive Aspects. Ed. by F. Bott. New York, NY, USA: Springer-Verlag New York, Inc., 2002.
[7]
M. Di Penta, R. E. K. Stirewalt, and E. Kraemer. “Designing your Next Empirical Study on Program Comprehension.” In: Program Comprehension, 2007. ICPC ’07. 15th IEEE International Conference on. 2007, pp. 281–285.
[8]
C. Elliott. “Functional Implementations of Continuous Modeled Animation.” In: Proceedings of the 10th International Symposium on Principles of Declarative Programming. PLILP ’98/ALP ’98. London, UK, UK: Springer-Verlag, 1998, pp. 284–299.
[9]
C. Elliott and P. Hudak. “Functional reactive animation.” In: Proceedings of the second ACM SIGPLAN international conference on Functional programming. ICFP ’97. Amsterdam, The Netherlands: ACM, 1997, pp. 263–273.
[10]
B. Ellis, J. Stylos, and B. Myers. “The Factory Pattern in API Design: A Usability Evaluation.” In: Proceedings of the 29th International Conference on Software Engineering. ICSE ’07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 302–312.
[11]
S. Endrikat and S. Hanenberg. “Is Aspect-Oriented Programming a Rewarding Investment into Future Code Changes? A Socio-technical Study on Development and Maintenance Time.” In: Program Comprehension (ICPC), 2011 IEEE 19th International Conference on. 2011, pp. 51–60.
[12]
S. D. Fleming, E. Kraemer, R. E. K. Stirewalt, S. Xie, and L. K. Dillon. “A Study of Student Strategies for the Corrective Maintenance of Concurrent Software.” In: Proceedings of the 30th International Conference on Software Engineering. ICSE ’08. Leipzig, Germany: ACM, 2008, pp. 759–768.
[13]
B. N. Freeman-Benson. “Kaleidoscope: mixing objects, constraints, and imperative programming.” In: OOPSLA/ECOOP ’90. Ottawa, Canada: ACM, 1990, pp. 77–88.
[14]
Gamma, Helm, Johnson, and Vlissides. Design Patterns Elements of Reusable Object-Oriented Software. Addison-Wesley, 2000.
[15]
S. Hanenberg. “An Experiment About Static and Dynamic Type Systems: Doubts About the Positive Impact of Static Type Systems on Development Time.” In: Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. OOPSLA ’10. Reno/Tahoe, Nevada, USA: ACM, 2010, pp. 22–35.
[16]
S. Hanenberg. “An Experiment About Static and Dynamic Type Systems: Doubts About the Positive Impact of Static Type Systems on Development Time.” In: SIGPLAN Not. 45.10 (Oct. 2010), pp. 22–35.
[17]
S. Hanenberg, S. Kleinschmager, and M. Josupeit-Walter. “Does Aspect-oriented Programming Increase the Development Speed for Crosscutting Code? An Empirical Study.” In: Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement. ESEM ’09. Washington, DC, USA: IEEE Computer Society, 2009, pp. 156–167.
[18]
M. Hoppe and S. Hanenberg. “Do Developers Benefit from Generic Types?: An Empirical Comparison of Generic and Raw Types in Java.” In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages and Applications. OOPSLA ’13. Indianapolis, Indiana, USA: ACM, 2013, pp. 457–474.
[19]
P. Hudak, A. Courtney, H. Nilsson, and J. Peterson. “Arrows, Robots, and Functional Reactive Programming.” In: Summer School on Advanced Functional Programming 2002, Oxford University. Vol. 2638. Lecture Notes in Computer Science. Springer-Verlag, 2003, pp. 159–187.
[20]
N. Juristo and A. M. Moreno. Basics of Software Engineering Experimentation. 1st. Springer Publishing Company, Incorporated, 2010.
[21]
S. Kleinschmager and S. Hanenberg. “How to Rate Programming Skills in Programming Experiments?: A Preliminary, Exploratory, Study Based on University Marks, Pretests, and Self-estimation.” In: Proceedings of the 3rd ACM SIGPLAN Workshop on Evaluation and Usability of Programming Languages and Tools. PLATEAU ’11. Portland, Oregon, USA: ACM, 2011, pp. 15–24.
[22]
T. D. LaToza, D. Garlan, J. D. Herbsleb, and B. A. Myers. “Program Comprehension As Fact Finding.” In: Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering. ESEC-FSE ’07. Dubrovnik, Croatia: ACM, 2007, pp. 361–370.
[23]
T. D. LaToza and B. A. Myers. “Developers Ask Reachability Questions.” In: Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering - Volume 1. ICSE ’10. Cape Town, South Africa: ACM, 2010, pp. 185–194.
[24]
T. D. LaToza, G. Venolia, and R. DeLine. “Maintaining Mental Models: A Study of Developer Work Habits.” In: Proceedings of the 28th International Conference on Software Engineering. ICSE ’06. Shanghai, China: ACM, 2006, pp. 492– 501.
[25]
J. Liberty and P. Betts. Programming Reactive Extensions and LINQ. 1st. Berkely, CA, USA: Apress, 2011.
[26]
I. Maier and M. Odersky. Deprecating the Observer Pattern with Scala.React. Tech. rep. 2012.
[27]
I. Maier and M. Odersky. “Higher-Order Reactive Programming with Incremental Lists.” In: ECOOP 2013 - Object-Oriented Programming. Ed. by G. Castagna. Vol. 7920. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2013, pp. 707–731.
[28]
C. Mayer, S. Hanenberg, R. Robbes, É. Tanter, and A. Stefik. “An Empirical Study of the Influence of Static Type Systems on the Usability of Undocumented Software.” In: Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. OOPSLA ’12. Tucson, Arizona, USA: ACM, 2012, pp. 683– 702.
[29]
L. A. Meyerovich, A. Guha, J. Baskin, G. H. Cooper, M. Greenberg, A. Bromfield, and S. Krishnamurthi. “Flapjax: a programming language for Ajax applications.” In: OOPSLA ’09. Orlando, Florida, USA: ACM, 2009, pp. 1–20.
[30]
B. A. Myers, R. G. McDaniel, R. C. Miller, A. S. Ferrency, A. Faulring, B. D. Kyle, A. Mickish, A. Klimovitski, and P. Doane. “The Amulet Environment: New Models for Effective User Interface Software Development.” In: IEEE Trans. Softw. Eng. 23.6 (June 1997), pp. 347–365.
[31]
R. Newton, G. Morrisett, and M. Welsh. “The Regiment Macroprogramming System.” In: Information Processing in Sensor Networks, 2007. IPSN 2007. 6th International Symposium on. 2007, pp. 489–498.
[32]
F. Olivero, M. Lanza, M. D’ambros, and R. Robbes. “Tracking Human-centric Controlled Experiments with Biscuit.” In: Proceedings of the ACM 4th Annual Workshop on Evaluation and Usability of Programming Languages and Tools. PLATEAU ’12. Tucson, Arizona, USA: ACM, 2012, pp. 1–6.
[33]
V. Pankratius, F. Schmidt, and G. Garreton. “Combining functional and imperative programming for multicore software: An empirical study evaluating Scala and Java.” In: Software Engineering (ICSE), 2012 34th International Conference on. 2012, pp. 123–133.
[34]
N. Pennington. “Stimulus structures and mental representations in expert comprehension of computer programs.” In: Cognitive Psychology 19.3 (1987), pp. 295–341.
[35]
L. Prechelt. “An Empirical Comparison of Seven Programming Languages.” In: Computer 33.10 (Oct. 2000), pp. 23– 29.
[36]
J. Quante. “Do Dynamic Object Process Graphs Support Program Understanding? - A Controlled Experiment.” In: Program Comprehension, 2008. ICPC 2008. The 16th IEEE International Conference on. June 2008, pp. 73–82.
[37]
V. Ramalingam and S. Wiedenbeck. “An empirical study of novice program comprehension in the imperative and objectoriented styles.” In: Papers presented at the seventh workshop on Empirical studies of programmers. ESP ’97. Alexandria, Virginia, USA: ACM, 1997, pp. 124–139.
[38]
T. Roehm, R. Tiarks, R. Koschke, and W. Maalej. “How do professional developers comprehend software?” In: Proceedings of the 2012 International Conference on Software Engineering. ICSE 2012. Zurich, Switzerland: IEEE Press, 2012, pp. 255–265.
[39]
G. Salvaneschi, J. Drechsler, and M. Mezini. “Towards Distributed Reactive Programming.” In: Coordination Models and Languages. Springer. 2013, pp. 226–235.
[40]
G. Salvaneschi, G. Hintz, and M. Mezini. “REScala: Bridging Between Object-oriented and Functional Style in Reactive Applications.” In: Proceedings of the 13th International Conference on Modularity. MODULARITY ’14. Lugano, Switzerland: ACM, 2014, pp. 25–36.
[41]
G. Salvaneschi and M. Mezini. “Towards Reactive Programming for Object-Oriented Applications.” English. In: Transactions on Aspect-Oriented Software Development XI. Ed. by S. Chiba, É. Tanter, E. Bodden, S. Maoz, and J. Kienzle. Vol. 8400. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2014, pp. 227–261.
[42]
M.-A. Storey. “Theories, Tools and Research Methods in Program Comprehension: Past, Present and Future.” In: Software Quality Control 14.3 (Sept. 2006), pp. 187–208.
[43]
J. Stylos and S. Clarke. “Usability Implications of Requiring Parameters in Objects’ Constructors.” In: Proceedings of the 29th International Conference on Software Engineering. ICSE ’07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 529–539.
[44]
J. Stylos and B. A. Myers. “The Implications of Method Placement on API Learnability.” In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. SIGSOFT ’08/FSE-16. Atlanta, Georgia: ACM, 2008, pp. 105–112.
[45]
R. Wettel, M. Lanza, and R. Robbes. “Software Systems As Cities: A Controlled Experiment.” In: Proceedings of the 33rd International Conference on Software Engineering. ICSE ’11. Honolulu, HI, USA: ACM, 2011, pp. 551–560.

Cited By

View all
  • (2024)Predicting Code Comprehension: A Novel Approach to Align Human Gaze with Code using Deep Neural NetworksProceedings of the ACM on Software Engineering10.1145/36607951:FSE(1982-2004)Online publication date: 12-Jul-2024
  • (2023)40 Years of Designing Code Comprehension Experiments: A Systematic Mapping StudyACM Computing Surveys10.1145/362652256:4(1-42)Online publication date: 9-Nov-2023
  • (2023)An empirical evaluation of the “Cognitive Complexity” measure as a predictor of code understandabilityJournal of Systems and Software10.1016/j.jss.2022.111561197:COnline publication date: 1-Mar-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
FSE 2014: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering
November 2014
856 pages
ISBN:9781450330565
DOI:10.1145/2635868
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Controlled Experiment
  2. Empirical Study
  3. Reactive Programming

Qualifiers

  • Research-article

Conference

SIGSOFT/FSE'14
Sponsor:

Acceptance Rates

Overall Acceptance Rate 17 of 128 submissions, 13%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)41
  • Downloads (Last 6 weeks)2
Reflects downloads up to 05 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Predicting Code Comprehension: A Novel Approach to Align Human Gaze with Code using Deep Neural NetworksProceedings of the ACM on Software Engineering10.1145/36607951:FSE(1982-2004)Online publication date: 12-Jul-2024
  • (2023)40 Years of Designing Code Comprehension Experiments: A Systematic Mapping StudyACM Computing Surveys10.1145/362652256:4(1-42)Online publication date: 9-Nov-2023
  • (2023)An empirical evaluation of the “Cognitive Complexity” measure as a predictor of code understandabilityJournal of Systems and Software10.1016/j.jss.2022.111561197:COnline publication date: 1-Mar-2023
  • (2023)A mapping study of language features improving object-oriented design patternsInformation and Software Technology10.1016/j.infsof.2023.107222160:COnline publication date: 1-Aug-2023
  • (2023)An empirical study on software understandability and its dependence on code characteristicsEmpirical Software Engineering10.1007/s10664-023-10396-728:6Online publication date: 15-Nov-2023
  • (2022)Interacto: A Modern User Interaction Processing ModelIEEE Transactions on Software Engineering10.1109/TSE.2021.308332148:9(3206-3226)Online publication date: 1-Sep-2022
  • (2021)Towards a Methodology for Participant Selection in Software Engineering ExperimentsProceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1145/3475716.3484273(1-6)Online publication date: 11-Oct-2021
  • (2021)Vers la complétude interactive: exigences pour une machine abstraite orientée interactionAdjunct Proceedings of the 32nd Conference on l'Interaction Homme-Machine10.1145/3451148.3458644(1-6)Online publication date: 13-Apr-2021
  • (2020)An empirical study of multi-threading paradigms Reactive programming vs continuation-passing styleProceedings of the 2020 3rd International Conference on Computing and Big Data10.1145/3418688.3418695(37-41)Online publication date: 5-Aug-2020
  • (2020)An Empirical Validation of Cognitive Complexity as a Measure of Source Code UnderstandabilityProceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1145/3382494.3410636(1-12)Online publication date: 5-Oct-2020
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media