Preston Tunnell Wilson
ABSTRACT
Most programming languages have been designed by committees or individuals. What happens if, instead, we throw open the design process and let lots of programmers weigh in on semantic choices? Will they avoid well-known mistakes like dynamic scope? What do they expect of aliasing? What kind of overloading behavior will they choose?
We investigate this issue by posing questions to programmers on Amazon Mechanical Turk. We examine several language features, in each case using multiple-choice questions to explore programmer preferences. We check the responses for consensus (agreement between people) and consistency (agreement across responses from one person). In general we find low consistency and consensus, potential confusion over mainstream features, and arguably poor design choices. In short, this preliminary evidence does not argue in favor of designing languages based on programmer preference.
- 2017. Betteridge’s law of headlines. https://en.wikipedia.org/wiki/ Betteridge%27s_law_of_headlines . (2017). Accessed: 2017-04-20.Google Scholar
- Piraye Bayman and Richard E. Mayer. 1983. A diagnosis of beginning programmers’ misconceptions of BASIC programming statements. Commun. ACM 26, 9 (1983). DOI: Google ScholarDigital Library
- Gary Bernhardt. 2012. Wat. A lightning talk from CodeMash. (2012). https://www.destroyallsoftware.com/talks/wat .Google Scholar
- Benedict Du Boulay. 1986. Some Difficulties of Learning to Program. Journal of Educational Computing Research 2, 1 (1986). DOI: Google ScholarCross Ref
- M.T.H. Chi. 2005. Common sense conceptions of emergent processes: Why some misconceptions are robust. Journal of the Learning Sciences 14 (2005), 161–199. Google ScholarCross Ref
- Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales. In Educational and Psychological Measurement. SAGE. DOI: Google ScholarCross Ref
- Matthias Felleisen. 1992. The revised report on the syntactic theories of sequential control and state. Theoretical Computer Science 103, 2 (1992), 235–271. DOI: Google ScholarDigital Library
- Kathi Fisler, Shriram Krishnamurthi, and Preston Tunnell Wilson. 2017. Assessing and Teaching Scope, Mutation, and Aliasing in UpperLevel Undergraduates. In Special Interest Group on Computer Science Education. ACM, New York, NY, USA. DOI: Google ScholarDigital Library
- Matthew Flatt. 2002. Composable and compilable macros: you want it when? ACM, New York, NY, USA, 12. DOI: Google ScholarDigital Library
- Joseph L. Fleiss. 1971. Measuring Nominal Scale Agreement Among Many Raters. In Psychological Bulletin. APA. DOI: Google ScholarCross Ref
- Ann E. Fleury. 1991. Parameter Passing: The Rules the Students Construct. In Special Interest Group on Computer Science Education. ACM, New York, NY, USA. DOI: Google ScholarDigital Library
- J. C. Gower. 1971. A General Coefficient of Similarity and Some of Its Properties. Biometrics 27, 4 (1971), 857–871. DOI: Google ScholarCross Ref
- R. W. Hamming. 1950. Error Detecting and Error Correcting Codes. The Bell System Technical Journal 29, 2 (1950), 147–160. DOI: Google ScholarCross Ref
- Björn Hartmann. 2010. What Would Other Programmers Do? Suggesting Solutions to Error Messages. In Special Interest Group on Computer–Human Interaction. ACM, New York, NY, USA, 1019–1028. DOI: Google ScholarDigital Library
- Antti-Juhani Kaijanaho. Evidence-Based Programming Language Design. (????).Google Scholar
- Leonard Kaufman and Peter J. Rousseeuw. 2008. Finding Groups in Data: An Introduction to Cluster Analysis. Wiley.Google Scholar
- Aniket Kittur, Ed H. Chi, and Bongwon Suh. 2008. Crowdsourcing User Studies with Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08). ACM, New York, NY, USA, 453–456. DOI: Google ScholarDigital Library
- Sebastian Kleinschmager, Stefan Hanenberg, Romain Robbes, Éric Tanter, and Andreas Stefik. 2012. Do Static Type Systems Improve the Maintainability of Software Systems? An Empirical Study. In International Conference on Program Comprehension. IEEE, 153–162. DOI: Google ScholarCross Ref
- Andrew J. Ko, Brad A. Myers, and Htet Htet Aung. 2004. Six Learning Barriers in End-User Programming Systems. In Visual Languages and Human Centric Computing. IEEE, 199–206. DOI: Google ScholarDigital Library
- L. Layman and G. Sigurđsson. 2013. Using Amazon’s Mechanical Turk for User Studies: Eight Things You Need to Know. In International Symposium on Empirical Software Engineering and Measurement. ACM, 275–278. DOI: Google ScholarCross Ref
- Martin Maechler, Peter Rousseeuw, Anja Struyf, Mia Hubert, Kurt Hornik, Matthias Studer, Pierre Roudier, and Juan Gonzalez. 2017. Package ’cluster’. https://cran.r-project.org/web/packages/cluster/ cluster.pdf . (2017). v2.0.6. Accessed: 2017-04-20.Google Scholar
- Guillaume Marceau, Kathi Fisler, and Shriram Krishnamurthi. 2011. Mind Your Language: On Novices’ Interactions with Error Messages. In Symposium on New ideas, new paradigms, and reflections on programming and software (Onward!). ACM. DOI: Google ScholarDigital Library
- Winter Mason and Siddharth Suri. 2012. Conducting behavioral research on Amazon’s Mechanical Turk. Behavior research methods 44, 1 (2012), 1–23. Google ScholarCross Ref
- Clemens Mayer, Stefan Hanenberg, Romain Robbes, Éric Tanter, and Andreas Stefik. 2012. An Empirical Study of the Influence of Static Type Systems on the Usability of Undocumented Software. In ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages and Applications. ACM. DOI: Google ScholarDigital Library
- Mark S Miller, Daniel Von Dincklage, Vuk Ercegovac, and Brian Chin. 2017. Uncanny Valleys in Declarative Language Design. In LIPIcs-Leibniz International Proceedings in Informatics, Vol. 71. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google Scholar
- Mathew Mooty, Andrew Faulring, Jeffrey Stylos, and Brad A. Myers. 2010. Calcite: Completing Code Completion for Constructors Using Crowds. In Visual Languages and Human Centric Computing. IEEE, 15–22. DOI: Google ScholarDigital Library
- Brad A. Myers, John F. Pane, and Andy Ko. 1976. Natural Programming Languages and Environment. Mathematisch Centrum, Afdeling Informatica (1976).Google Scholar
- Brad A. Myers, John F. Pane, and Andy Ko. 2004. Natural Programming Languages and Environment. Commun. ACM 47, 9 (2004). Google ScholarDigital Library
- Seymour Papert. 1993. Mind-Storms: Children, Computers, and Powerful Ideas. Basic Books, New York, NY, USA.Google Scholar
- Justus J. Randolph. 2005. Free-Marginal Multirater Kappa: An Alternative to Fleiss’ Fixed-Marginal Multirater Kappa. In Joensuu Learning and Instruction Symposium. ERIC, 20.Google Scholar
- Amber Settle. 2014. What’s Motivation Got to Do with It? A Survey of Recursion in the Computing Education Literature. Technical Report at DePaul University 23 (2014).Google Scholar
- Juha Sorva. 2012. Visual Program Simulation in Introductory Programming Education. Ph.D. Dissertation. Aalto University.Google Scholar
- Guy L. Steele, Jr. and Jon L. White. 1990. How to Print Floating-point Numbers Accurately. In Proceedings of the ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation (PLDI ’90). ACM, New York, NY, USA, 112–126. DOI: Google ScholarDigital Library
- Andreas Stefik and Susanna Siebert. 2013. An Empirical Investigation into Programming Language Syntax. In ACM Transactions on Computing Education. ACM, New York, NY, USA. DOI: Google ScholarDigital Library
- Robert E. Strom, David F. Bacon, Arthur P. Goldberg, Andy Lowry, Daniel M. Yellin, and Shaula Alexander Yemini. 1991. Hermes: A Language for Distributed Computing. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.Google Scholar
- Phillip Merlin Uesbeck, Andreas Stefik, Stefan Hanenberg, Jan Pedersen, and Patrick Daleiden. 2016. An empirical study on the impact of C++ lambdas and programmer experience. In International Conference on Software Engineering. ACM, New York, NY, USA. DOI: Google ScholarDigital Library
Index Terms
- Can we crowdsource language design?
Recommendations
Assessing user preferences in programming language design
Onward! 2017: Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and SoftwareThe design of new programming languages has primarily been guided by the preferences of a few (the authors of the language), rather than systematic study of the various options available. This is in part due to the fact that user studies to effectively ...
Language Design for Program Manipulation
The design of procedural and object-oriented programming languages is considered with respect to how easily programs written in those languages can be formally manipulated. Current procedural languages such as Pascal, Modula-2 and Ada; generally support ...
Comments on "Language Design for Program Manipulation"
The paper by E.A.T. Merks et al. "Language design for program manipulation" identifies design principles for a procedural or object-oriented language whose programs will be easier to manipulate. However, it neglects to relate these design principles to ...
Comments