skip to main content
10.1145/2089155.2089160acmconferencesArticle/Chapter ViewAbstractPublication PagessplashConference Proceedingsconference-collections
research-article

Heuristic evaluation of programming language features: two parallel programming case studies

Published: 24 October 2011 Publication History

Abstract

Usability is an important feature for programming languages, and user evaluations can provide invaluable feedback on language design. However, user studies which compare programming languages or systems are often very expensive and typically inconclusive. In this paper, we posit that discount usability methods can be successfully applied to programming languages concepts such as language features. We give examples of useful feedback received from applying heuristic evaluation to two language features targeted at parallel programming.

References

[1]
K. Asanovic, R. Bodik, J. Demmel, T. Keaveny, K. Keutzer, J. Kubiatowicz, N. Morgan, D. Patterson, K. Sen, J. Wawrzynek, et al. A view of the parallel computing landscape. Communications of the ACM, 52(10):56--67, 2009.
[2]
A. Blackwell and T. Green. A Cognitive Dimensions questionnaire optimised for users. In Annual Meeting of the Psychology of Programming Interest Group, volume 12, pages 137--152, 2000.
[3]
T. Buxton, A. Tarrell, and A. Fruhling. Heuristic Evaluation of Mission-Critical Software Using a Large Team. International Conference on Human-Computer Interaction, pages 673--682, 2009.
[4]
S. Clarke. Evaluating a new programming language. In Workshop of the Psychology of Programming Interest Group, pages 275--289, 2001.
[5]
J. Dagit, J. Lawrance, C. Neumann, M. Burnett, R. Metoyer, and S. Adams. Using cognitive dimensions: advice from the trenches. Journal of Visual Languages & Computing, 17(4):302--327, 2006.
[6]
H. Desurvire, M. Caplan, and J. Toth. Using heuristics to evaluate the playability of games. In Conference on Human factors In computing systems (CHI), 2004. Extended Abstract.
[7]
C. Flanagan and S. Qadeer. A type and effect system for atomicity. In Conference on Programming Language Design and Implementation (PLDI), 2003.
[8]
W. Gray. Who Ya Gonna Call? You're on Your Own. IEEE Software, 14(4):26, 1997.
[9]
T. Green and M. Petre. Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework. Journal of Visual Languages and Computing, 7(2):131--174, 1996.
[10]
R. Jeffries, J. Miller, C. Wharton, and K. Uyeda. User interface evaluation in the real world: a comparison of four techniques. In Conference on Human factors In computing systems (CHI), pages 119--124. ACM, 1991.
[11]
S. Jones, A. Blackwell, and M. Burnett. A user-centred approach to functions in Excel. In International Conference on Functional Programming (ICFP). ACM, 2003.
[12]
D. Kane. Finding a place for discount usability engineering in agile development: throwing down the gauntlet. In Agile Development Conference (ADC), 2003.
[13]
S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. SIGPLAN Notices, 43(3):329--339, 2008.
[14]
J. Mankoff, A. Dey, G. Hsieh, J. Kientz, S. Lederer, and M. Ames. Heuristic evaluation of ambient displays. In Conference on Human factors In computing systems (CHI), 2003.
[15]
J. Nielsen. Usability Inspection Methods. Wiley, 1994.
[16]
J. Nielsen. Applying discount usability engineering. IEEE Software, 12(1):98--100, 1995.
[17]
J. Nielsen and R. Molich. Heuristic evaluation of user interfaces. In Conference on Human factors In computing systems (CHI). ACM, 1990.
[18]
J. Pane and B. Myers. Usability issues in the design of novice programming systems. Human-Computer Interaction Institute Technical Report Carnegie Mellon University-HCII-96-101, 1996.
[19]
J. Pane, B. Myers, and L. Miller. Using HCI techniques to design a more usable programming system. In Symposium on Human Centric Computing Languages and Environments, 2002.
[20]
D. Pinelle, N. Wong, and T. Stach. Heuristic evaluation for games: usability principles for video game design. In Conference on Human factors In computing systems (CHI), pages 1453--1462. ACM, 2008.
[21]
G. Sim, J. Read, and G. Cockton. Evidence based design of heuristics for computer assisted assessment. Human-Computer Interaction (INTERACT), 2009.
[22]
J. Yi and C. Flanagan. Effects for cooperable and serializable threads. In Workshop on Types in Language Design and Implementation (TLDI), 2010.
[23]
J. Yi, C. Sadowski, and C. Flanagan. Cooperative reasoning for preemptive execution. In Symposium on Principles and Practice of Parallel Programming (PPoPP), 2011.

Cited By

View all
  • (2024)On the Usefulness of Python Structural Pattern Matching: An Empirical Study2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00058(501-511)Online publication date: 12-Mar-2024
  • (2022)Programming Language Evaluation Criteria for Safety-Critical Software in the Air Domain2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)10.1109/ISSREW55968.2022.00072(230-237)Online publication date: Oct-2022
  • (2020)Usable and Accessible Robot Programming System for People Who Are Visually ImpairedUniversal Access in Human-Computer Interaction. Design Approaches and Supporting Technologies10.1007/978-3-030-49282-3_32(445-464)Online publication date: 10-Jul-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PLATEAU '11: Proceedings of the 3rd ACM SIGPLAN workshop on Evaluation and usability of programming languages and tools
October 2011
90 pages
ISBN:9781450310246
DOI:10.1145/2089155
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. heuristic evaluation
  2. programming languages
  3. usability

Qualifiers

  • Research-article

Conference

SPLASH '11
Sponsor:

Acceptance Rates

Overall Acceptance Rate 5 of 8 submissions, 63%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)On the Usefulness of Python Structural Pattern Matching: An Empirical Study2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00058(501-511)Online publication date: 12-Mar-2024
  • (2022)Programming Language Evaluation Criteria for Safety-Critical Software in the Air Domain2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)10.1109/ISSREW55968.2022.00072(230-237)Online publication date: Oct-2022
  • (2020)Usable and Accessible Robot Programming System for People Who Are Visually ImpairedUniversal Access in Human-Computer Interaction. Design Approaches and Supporting Technologies10.1007/978-3-030-49282-3_32(445-464)Online publication date: 10-Jul-2020
  • (2019)Programmer eXperience: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2019.29201247(71079-71094)Online publication date: 2019
  • (2015)Object-oriented programming course revisitedProceedings of the 15th Koli Calling Conference on Computing Education Research10.1145/2828959.2828974(23-32)Online publication date: 19-Nov-2015
  • (2012)The evolution of data racesProceedings of the 9th IEEE Working Conference on Mining Software Repositories10.5555/2664446.2664473(171-174)Online publication date: 2-Jun-2012
  • (2012)How can a DSL for expert end-users be designed for better usability?CHI '12 Extended Abstracts on Human Factors in Computing Systems10.1145/2212776.2223855(2673-2678)Online publication date: 5-May-2012
  • (2012)The evolution of data races2012 9th IEEE Working Conference on Mining Software Repositories (MSR)10.1109/MSR.2012.6224277(171-174)Online publication date: Jun-2012

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media