skip to main content
10.1145/2950290.2950315acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

Efficiency of projectional editing: a controlled experiment

Published: 01 November 2016 Publication History

Abstract

Projectional editors are editors where a user's editing actions directly change the abstract syntax tree without using a parser. They promise essentially unrestricted language com position as well as flexible notations, which supports aligning languages with their respective domain and constitutes an essential ingredient of model-driven development. Such editors have existed since the 1980s and gained widespread attention with the Intentional Programming paradigm, which used projectional editing at its core. However, despite the benefits, programming still mainly relies on editing textual code, where projectional editors imply a very different -- typically perceived as worse -- editing experience, often seen as the main challenge prohibiting their widespread adoption. We present an experiment of code-editing activities in a projectional editor, conducted with 19 graduate computer-science students and industrial developers. We investigate the effects of projectional editing on editing efficiency, editing strategies, and error rates -- each of which we also compare to conventional, parser-based editing. We observe that editing is efficient for basic-editing tasks, but that editing strategies and typical errors differ. More complex tasks require substantial experience and a better understanding of the abstract-syntax-tree structure -- then, projectional editing is also efficient. We also witness a tradeoff between fewer typing mistakes and an increased complexity of code editing.

References

[1]
CamStudio Desktop Screen Recorder. http://sourceforge.net/projects/camstudio.
[2]
Code Orchestra IDE. http://codeorchestra.com/ide.
[3]
Concrete. http://concrete-editor.org.
[4]
Jetbrains MPS. http://www.jetbrains.com/mps.
[5]
mbeddr. http://mbeddr.com.
[6]
mbeddr Graphical State Machines. http://mbeddr.com/2015/03/05/graphicalSM.html.
[7]
MPS Tutorial. http://www.jetbrains.com/mps/docs/tutorial.html.
[8]
Online Appendix. http://gsd.uwaterloo.ca/projectional-workbenches.
[9]
Whole Platform. http://whole.sourceforge.net.
[10]
T. W. Anderson and J. D. Finn. The New Statistical Analysis of Data. Springer, 1996.
[11]
Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B (Methodological), 57(1):289–300, 1995.
[12]
A. F. Blackwell. First steps in programming: A rationale for attention investment models. In Proc. HCC, 2002.
[13]
M. Broy, S. Kirstan, H. Krcmar, and B. Schätz. What is the Benefit of a Model-Based Design of Embedded Software Systems in the Car Industry? In J. Rech and C. Bunse, editors, Emerging Technologies for the Evolution and Maintenance of Software Models. IGI Global, 2011.
[14]
M. Christerson and H. Kolk. Domain expert DSLs, 2009. talk at QCon London 2009, available at http://www.infoq.com/presentations/ DSL-Magnus-Christerson-Henk-Kolk.
[15]
T. Clark. A Declarative Approach to Heterogeneous Multi-Mode Modelling Languages. In Proc. GEMOC, 2014.
[16]
N. Cliff. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin, 114(3):494, 1993.
[17]
J. O. Coplien. Multi-paradigm Design for C++. Addison-Wesley, 1999.
[18]
K. Czarnecki and U. W. Eisenecker. Generative Programming: Methods, Tools, and Applications. Addison-Wesley, 2000.
[19]
S. Erdweg, P. G. Giarrusso, and T. Rendel. Language Composition Untangled. In Proc. LDTA, 2012.
[20]
S. Erdweg, T. van der Storm, M. Völter, M. Boersma, R. Bosman, W. R. Cook, A. Gerritsen, A. Hulshout, S. Kelly, A. Loh, et al. The State of the Art in Language Workbenches. In Proc. SLE, 2013.
[21]
M. Fowler. Language Workbenches: The Killer-App for Domain Specific Languages? http://www.martinfowler. com/articles/languageWorkbench.html, 2005.
[22]
J. Gray and G. Karsai. An Examination of DSLs for Concisely Representing Model Traversals and Transformations. In Proc. HICSS, 2003.
[23]
M. Höst, B. Regnell, and C. Wohlin. Using Students As Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment. Empirical Softw. Engg., 5(3):201–214, Nov. 2000.
[24]
J. Hutchinson, J. Whittle, M. Rouncefield, and S. Kristoffersen. Empirical Assessment of MDE in Industry. In Proc. ICSE, 2011.
[25]
R. B. Kieburtz, L. McKinney, J. M. Bell, J. Hook, A. Kotov, J. Lewis, D. P. Oliva, T. Sheard, I. Smith, and L. Walton. A Software Engineering Experiment in Software Component Generation. In Proc. ICSE, 1996.
[26]
A. J. Ko, H. H. Aung, and B. A. Myers. Design Requirements for More Flexible Structured Editors from a Study of Programmers’ Text Editing. In Proc. CHI EA, 2005.
[27]
A. Kuhn, G. C. Murphy, and C. A. Thompson. An Exploratory Study of Forces and Frictions Affecting Large-Scale Model-Driven Development. In Proc. MODELS. 2012.
[28]
C. M. Lewis. How programming environment shapes perception, learning and goals: Logo vs. scratch. In Proc. SIGCSE, 2010.
[29]
G. Liebel, N. Marko, M. Tichy, A. Leitner, and J. Hansson. Assessing the state-of-practice of model-based engineering in the embedded systems domain. In Proc. MODELS, 2014.
[30]
P. Liggesmeyer and M. Trapp. Trends in Embedded Software Engineering. IEEE Softw., 26(3):19–25, May 2009.
[31]
M. Lillack, T. Berger, and R. Hebig. Experiences from reengineering and modularizing a legacy software generator with a projectional language workbench. In Proc. SPLC. 2016.
[32]
R. Medina-Mora and P. H. Feiler. An Incremental Programming Environment. IEEE Trans. Softw. Eng., 7(5):472–482, Sept. 1981.
[33]
R. C. Miller and B. A. Myers. Multiple Selections in Smart Text Editing. In Proc. IUI, 2002.
[34]
Z. Molotnikov, M. Völter, and D. Ratiu. Automated domain-specific C verification with mbeddr. In Proc. ASE, 2014.
[35]
D. Notkin. The GANDALF Project. J. Syst. Softw., 5(2):91–105, May 1985.
[36]
M. F. Poller and S. K. Garter. A Comparative Study of Moded and Modeless Text Editing by Experienced Editor Users. In Proc. CHI, 1983.
[37]
S. W. Porter. Design of a Syntax Directed Editor for PSDL (Prototype Systems Design Language). Master’s thesis, Naval Postgraduate School, Monterey, CA, USA, 1988.
[38]
D. Ratiu, B. Schaetz, M. Voelter, and B. Kolb. Language engineering as an enabler for incrementally defined formal analyses. In Proc. FormSERA, 2012.
[39]
T. W. Reps and T. Teitelbaum. The Synthesizer Generator. In Proc. SDE, 1984.
[40]
P. Runeson. Using Students as Experiment Subjects—An Analysis on Graduate and Freshmen Student Data. In Proc. EASE, 2003.
[41]
B. Selic. The Pragmatics of Model-Driven Development. IEEE Softw., 20(5):19–25, Sept. 2003.
[42]
W. Shadish, T. Cook, and D. Campbell. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, 2002.
[43]
M. Simi and F. Campagne. Composable Languages for Bioinformatics: The NYoSh Experiment. PeerJ, 2:e241, 2014.
[44]
C. Simonyi. The death of computer languages, the birth of intentional programming. In Proc. NATO Science Committee Conference, 1995.
[45]
C. Simonyi, M. Christerson, and S. Clifford. Intentional Software. In Proc. OOPSLA, 2006.
[46]
A. Strauss and J. Corbin. Open Coding. Basics of Qualitative Research: Grounded Theory Procedures and Techniques, 2:101–121, 1990.
[47]
M. W. Van Someren, Y. F. Barnard, J. A. Sandberg, et al. The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. Academic Press London, 1994.
[48]
E. Visser. WebDSL: A Case Study in Domain-Specific Language Engineering. In R. Lämmel, J. Visser, and J. a. Saraiva, editors, Generative and Transformational Techniques in Software Engineering II, pages 291–373. Springer, 2008.
[49]
M. Voelter. mbeddr C User Guide. Itemis AG. http://mbeddr.com/userguide/UserGuideExport.html.
[50]
M. Voelter. Language and IDE Modularization and Composition with MPS. In GTTSE, LNCS. Springer, 2011.
[51]
M. Voelter and S. Lisson. Supporting Diverse Notations in MPS’ Projectional Editor. 2014.
[52]
M. Voelter, D. Ratiu, B. Kolb, and B. Schätz. mbeddr: Instantiating a Language Workbench in the Embedded Software Domain. Autom. Softw. Eng., 20(3):339–390, 2013.
[53]
M. Voelter, D. Ratiu, B. Schaetz, and B. Kolb. mbeddr: an extensible c-based programming language and ide for embedded systems. In Proceedings of the 3rd annual conference on Systems, programming, and applications: software for humanity, pages 121–140. ACM, 2012.
[54]
M. Voelter, D. Ratiu, and F. Tomassetti. Requirements as First-Class Citizens. In Proc. ACES-MB, 2013.
[55]
M. Voelter, J. Siegmund, T. Berger, and B. Kolb. Towards User-Friendly Projectional Editors. In Proc. SLE, 2014.
[56]
M. Voelter, A. van Deursen, B. Kolb, and S. Eberle. Using C Language Extensions for Developing Embedded Software: A Case Study. In Proc. OOPSLA, 2015.
[57]
M. Voelter, A. van Deursen, B. Kolb, and S. Eberle. Using c language extensions for developing embedded software: A case study. In Proc. OOPSLA, 2015.

Cited By

View all
  • (2025)Pantograph: A Fluid and Typed Structure EditorProceedings of the ACM on Programming Languages10.1145/37048649:POPL(802-831)Online publication date: 9-Jan-2025
  • (2024)Machine learning experiment management tools: a mixed-methods empirical studyEmpirical Software Engineering10.1007/s10664-024-10444-w29:4Online publication date: 29-May-2024
  • (2024)Modeling with Gentleman: a web-based projectional editorSoftware and Systems Modeling10.1007/s10270-024-01219-4Online publication date: 9-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering
November 2016
1156 pages
ISBN:9781450342186
DOI:10.1145/2950290
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. experiment
  2. language workbench
  3. projectional editing

Qualifiers

  • Research-article

Conference

FSE'16
Sponsor:

Acceptance Rates

Overall Acceptance Rate 17 of 128 submissions, 13%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)31
  • Downloads (Last 6 weeks)1
Reflects downloads up to 24 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Pantograph: A Fluid and Typed Structure EditorProceedings of the ACM on Programming Languages10.1145/37048649:POPL(802-831)Online publication date: 9-Jan-2025
  • (2024)Machine learning experiment management tools: a mixed-methods empirical studyEmpirical Software Engineering10.1007/s10664-024-10444-w29:4Online publication date: 29-May-2024
  • (2024)Modeling with Gentleman: a web-based projectional editorSoftware and Systems Modeling10.1007/s10270-024-01219-4Online publication date: 9-Oct-2024
  • (2023)Structured Editing for All: Deriving Usable Structured Editors from GrammarsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580785(1-16)Online publication date: 19-Apr-2023
  • (2023)Gradual Structure Editing with Obligations2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00016(71-81)Online publication date: 3-Oct-2023
  • (2023)Projectional Editors for JSON-Based DSLs2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00015(60-70)Online publication date: 3-Oct-2023
  • (2023)A model-based infrastructure for the specification and runtime execution of self-adaptive IoT architecturesComputing10.1007/s00607-022-01145-7105:9(1883-1906)Online publication date: 9-Feb-2023
  • (2022)Experimental Comparison of Editor Types for Domain-Specific LanguagesApplied Sciences10.3390/app1219989312:19(9893)Online publication date: 1-Oct-2022
  • (2022)Partial Parsing for Structured EditorsProceedings of the 15th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3567512.3567522(110-120)Online publication date: 29-Nov-2022
  • (2022)Forest: Structural Code Editing with Multiple CursorsProceedings of the 2022 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3563835.3567663(137-152)Online publication date: 29-Nov-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media