skip to main content
10.1145/3133850.3133851acmconferencesArticle/Chapter ViewAbstractPublication PagessplashConference Proceedingsconference-collections
research-article
Public Access

Assessing user preferences in programming language design

Published: 25 October 2017 Publication History

Abstract

The design of new programming languages has primarily been guided by the preferences of a few (the authors of the language), rather than systematic study of the various options available. This is in part due to the fact that user studies to effectively test usability or understandability hypotheses are cumbersome and expensive. An interesting question is whether crowdsourcing techniques can be leveraged to improve this situation.
We explore this idea using a specific example. While the streaming data paradigm is a popular one for expressing parallelism within applications, there has been little consensus on the methods used to express streaming topologies. Here, we explore the use of Mechanical Turk to recruit self-described programmers as a community to assess user preferences and code readability for two techniques currently in use for the expression of streaming application topology.
The positive results of this study point to the idea that crowdsourcing techniques can be an effective technique that can inexpensively assist language developers in making good design choices.

Supplementary Material

Auxiliary Archive (onward17-papers4-aux.zip)

References

[1]
Jonathan C. Beard, Peng Li, and Roger D. Chamberlain. 2015. RaftLib: A C++ Template Library for High Performance Stream Parallel Processing. In Proc. of 6th Int'l Workshop on Programming Models and Applications for Multicores and Manycores. 96s105.
[2]
Aggelos Biboudis, Nick Palladinos, George Fourtounis, and Yannis Smaragdakis. 2015. Streams a la carte: Extensible Pipelines with Object Algebras. In Proc. of 29th European Conf. on Object-Oriented Programming. 591s613.
[3]
Ian Buck, Tim Foley, Daniel Horn, Jeremy Sugerman, Kayvon Fatahalian, Mike Houston, and Pat Hanrahan. 2004. Brook for GPUs: Stream Computing on Graphics Hardware. ACM Trans. on Graphics 23, 3 (Aug. 2004), 777-786.
[4]
Michael Buhrmester, Tracy Kwang, and Samuel D. Gosling. 2011. Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality Data? Perspectives on Psychological Science 6, 1 (Jan. 2011), 3-5.
[5]
Raymond P.L. Buse, Caitlin Sadowski, and Westley Weimer. 2011. Benefits and Barriers of User Evaluation in Software Engineering Research. In Proc. of ACM Int'l Conf. on Object Oriented Programming Systems Languages and Applications. 643-656.
[6]
Charles Consel, Hedi Hamdi, Laurent Reveillere, Lenin Singaravelu, Haiyan Yu, and Calton Pu. 2003. Spidle: A DSL approach to specifying streaming applications. In Proc. of Int'l Conf. on Generative Programming and Component Engineering. 1s17.
[7]
Leonardo Dagum and Ramesh Menon. 1998. OpenMP: an industry standard API for shared-memory programming. IEEE Computational Science and Engineering 5, 1 (Jan. 1998), 46s55.
[8]
Charles N. Fischer, Ron K. Cytron, and Richard J. LeBlanc. 2009. Crafting A Compiler. Addison-Wesley, Boston, MA, USA.
[9]
Mark A. Franklin, Eric J. Tyson, James Buckley, Patrick Crowley, and John Maschmeyer. 2006. Auto-Pipe and the X Language: A Pipeline Design Tool and Description Language. In Proc. of Int'l Parallel and Distributed Processing Symp. 10 pp.
[10]
Mohamed Medhat Gaber, Arkady Zaslavsky, and Shonali Krishnaswamy. 2005. Mining data streams: A review. SIGMOD Rec. 34, 2 (2005), 18-26.
[11]
Bugra Gedik, Henrique Andrade, Kun-Lung Wu, Philip S. Yu, and Myungcheol Doo. 2008. SPADE: The System S declarative stream processing engine. In Proc. of ACM SIGMOD Int'l Conf. on Management of Data. 1123-1134.
[12]
James Gosling. 1997. The feel of Java. Computer 30, 6 (June 1997), 53s57.
[13]
Clemens Grelck, Sven-Bodo Scholz, and Alex Shafarenko. 2008. A Gentle Introduction to S-Net: Typed Stream Processing and Declarative Coordination of Asynchronous Components. Parallel Processing Letters 18, 2 (2008), 221s237.
[14]
Jayanth Gummaraju, Joel Coburn, Yoshio Turner, and Mendel Rosenblum. 2008. Streamware: programming general-purpose multicore processors using streams. In Proc. of 13th Int'l Conf. on Architectural Support for Programming Languages and Operating Systems. 297-307.
[15]
Stefan Hanenberg. 2010. Faith, Hope, and Love: An Essay on Software Science's Neglect of Human Factors. In Proc. of ACM Int'l Conf. on Object Oriented Programming Systems Languages and Applications. 933- 946.
[16]
Martin Hirzel, Robert Soule, Scott Schneider, Bugra Gedik, and Robert Grimm. 2014. A Catalog of Stream Processing Optimizations. Comput. Surveys 46, 4 (April 2014), 46:1-46:34.
[17]
Ken Kennedy, Charles Koelbel, and Hans Zima. 2007. The Rise and Fall of High Performance Fortran: An Historical Object Lesson. In Proc. of 3rd ACM SIGPLAN Conf. on History of Programming Languages. 7-1-7-22.
[18]
Aniket Kittur, Ed H. Chi, and Bongwon Suh. 2008. Crowdsourcing User Studies with Mechanical Turk. In Proc. of SIGCHI Conf. on Human Factors in Computing Systems. 453s456.
[19]
Andrew J Ko, Thomas D Latoza, and Margaret M Burnett. 2015. A practical guide to controlled experiments of software engineering tools with human participants. Empirical Software Engineering 20, 1 (2015), 110-141.
[20]
Edward A. Lee. 2006. The Problem with Threads. Computer 39, 5 (May 2006), 33-42.
[21]
Edward A. Lee and David G. Messerschmitt. 1987. Synchronous data flow. Proc. IEEE 75, 9 (Sept. 1987), 1235-1245.
[22]
I-Ting Angelina Lee, Charles E. Leiserson, Tao B. Schardl, Zhunping Zhang, and Jim Sukha. 2015. On-the-Fly Pipeline Parallelism. ACM Trans. on Parallel Computing 2, 3, Article 17 (Sept. 2015), 42 pages.
[23]
Michael J. Lee and Andrew J. Ko. 2012. Investigating the role of purposeful goals on novices' engagement in a programming game. In Proc. of IEEE Symp. on Visual Languages and Human-Centric Computing. 163-166.
[24]
Peng Li, Kunal Agrawal, Jeremy Buhler, and Roger D. Chamberlain. 2010. Deadlock Avoidance for Streaming Computation with Filtering. In Proc. of 22nd ACM Symp. on Parallelism in Algorithms and Architectures. 243-252.
[25]
Chuan-Kai Lin and Andrew P. Black. 2007. DirectFlow: A domain-specific language for information-flow systems. In Proc. of 21st European Conf. on Object-Oriented Programming. 299-322.
[26]
Ying Liu, Nithya Vijayakumar, and Beth Plale. 2006. Stream processing in data-driven computational science. In Proc. of IEEE/ACM Int'l Conf. on Grid Computing. 160-167.
[27]
William R. Mark, R. Steven Glanville, Kurt Akeley, and Mark J. Kilgard. 2003. Cg: a system for programming graphics hardware in a C-like language. ACM Trans. on Graphics 22, 3 (July 2003), 896-907.
[28]
Shane Markstrum. 2010. Staking Claims: A History of Programming Language Design Claims and Evidence: A Positional Work in Progress. In Proc. of Workshop on Evaluation and Usability of Programming Languages and Tools. 7:1-7:5.
[29]
Peter Mattson, Andrew Chang, Ujval J. Kapasi, Scott Rixner, John D. Owens, Jinyung Namkoong, Brucek Khailany, William J. Dally, and Brian Towles. 2001. Imagine: Media processing with streams. IEEE Micro 21, 2 (March/April 2001), 35-46.
[30]
L. Neumeyer, B. Robbins, A. Nair, and A. Kesari. 2010. S4: Distributed Stream Computing Platform. In Proc. of IEEE Int'l Conf. on Data Mining Workshops. 170-177.
[31]
Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu. 2014. A Large Scale Study of Programming Languages and Code Quality in Github. In Proc. of ACM Int'l Symp. on Foundations of Software Engineering. 155-165.
[32]
Dennis M. Ritchie. 1993. The Development of the C Language. SIGPLAN Not. 28, 3 (March 1993), 201s208.
[33]
JohnW. Romein, P. Chris Broekema, Ellen van Meijeren, Kjeld van der Schaaf, and Walther H. Zwart. 2006. Astronomical real-time streaming signal processing on a Blue Gene/L supercomputer. In Proc. of ACM Symp. on Parallelism in Algorithms and Architectures. 59-66.
[34]
Ben Shneiderman, Catherine Plaisant, Maxine Cohen, Niklas Elmqvist, Steven Jacobs, and Nicholas Diakopoulos. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction (6 ed.). Pearson Education, Limited, London, England.
[35]
Andreas Stefik, Stefan Hanenberg, Mark McKenney, Anneliese Andrews, Srinivas Kalyan Yellanki, and Susanna Siebert. 2014. What is the Foundation of Evidence of Human Factors Decisions in Language Design? An Empirical Study on Programming Language Workshops. In Proc. of 22nd Int'l Conf. on Program Comprehension. 223s231.
[36]
Andreas Stefik and Susanna Siebert. 2013. An Empirical Investigation into Programming Language Syntax. Trans. Comput. Educ. 13, 4 (Nov. 2013), 19:1-19:40.
[37]
Robert Stephens. 1997. A Survey of Stream Processing. Acta Informatica 34, 7 (1997), 491-541.
[38]
Bjarne Stroustrup. 1986. The C++ Programming Language. Addison-Wesley, Boston, MA, USA.
[39]
Jeffrey Stylos and Steven Clarke. 2007. Usability Implications of Requiring Parameters in Objects' Constructors. In Proc. of 29th Int'l Conference on Software Engineering. 529-539.
[40]
William Thies and Saman Amarasinghe. 2010. An empirical characterization of stream programs and its implications for language and compiler design. In Proc. of Int'l Conf. on Parallel Architectures and Compilation Techniques. 365-376.
[41]
William Thies, Michal Karczmarek, and Saman Amarasinghe. 2002. StreamIt: A Language for Streaming Applications. In Proc. of 11th Int'l Conf. on Compiler Construction. 179-196.
[42]
Joseph G. Wingbermuehle, Roger D. Chamberlain, and Ron K. Cytron. 2012. ScalaPipe: A Streaming Application Generator. In Proc. of Symp. on Application Accelerators in High-Performance Computing. 44-53.

Cited By

View all
  • (2022)Game Design, Gender and Personalities in Programming EducationFrontiers in Computer Science10.3389/fcomp.2022.8249954Online publication date: 8-Feb-2022
  • (2021)PLIERSACM Transactions on Computer-Human Interaction10.1145/345237928:4(1-53)Online publication date: 23-Jul-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
Onward! 2017: Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software
October 2017
261 pages
ISBN:9781450355308
DOI:10.1145/3133850
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2017

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. Crowdsourcing
  2. streaming language design
  3. user studies

Qualifiers

  • Research-article

Funding Sources

Conference

SPLASH '17
Sponsor:

Acceptance Rates

Overall Acceptance Rate 40 of 105 submissions, 38%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)104
  • Downloads (Last 6 weeks)19
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Game Design, Gender and Personalities in Programming EducationFrontiers in Computer Science10.3389/fcomp.2022.8249954Online publication date: 8-Feb-2022
  • (2021)PLIERSACM Transactions on Computer-Human Interaction10.1145/345237928:4(1-53)Online publication date: 23-Jul-2021

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media