skip to main content
10.1145/1852786.1852808acmconferencesArticle/Chapter ViewAbstractPublication PagesesemConference Proceedingsconference-collections
research-article

An empirical study of specification by example in a software engineering tool

Published: 16 September 2010 Publication History

Abstract

Meta-CASE tools offer CASE tool specialisation by enabling a designer to specify a tool which is then generated automatically. Constraints are often used in such meta-CASE tools for governing the syntax and semantics of model elements and the values of their attributes. However, the constraint definition process is complex, time-consuming and error-prone. This paper presents an empirical study of the use of Specification by Example (SBE), based on the well-known notion of Programming by Example (PBE), as a user-computer interactive technique for such constraint specification. Two constraint specification techniques have been implemented in a meta-CASE tool a wizard that represents a conventional form-filling technique and an SBE technique that depends on the user providing one or more examples and the system inferring a list of possible intended constraints. The empirical study compared the wizard and SBE with respect to constraint definition correctness, task completion time, and user satisfaction. Two common modelling diagrams have been used, a State Transition Diagram and a Use Case Diagram. Results suggest that SBE is superior to the wizard in terms of measured criteria described above. Observations on the interaction of users with the system and opinions of participants are also presented.

References

[1]
Amant, R., Lieberman, H., Potter, R., & Zettlemoyer, L. (2000). Visual Generalisation in Programming By Example. Communications of the ACM, 43 (3), 107--114.
[2]
Argall, B., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57 (5), 469--483.
[3]
Ferguson, R., & Hunter, A. (2000). MetaBuilder: The Diagrammer's Diagrammer. Lecture Notes In Computer Science; Vol. 1889 (pp. 407--421). London, UK: SpringerVerlag.
[4]
Findeisen, P. (1994). The Metaview System. University of Alberta. Alberta: University of Alberta.
[5]
Gong, M., Scott, L., & Offen, R. (1997). MetaBuilder: a generic CASE tool builder. Software Engineering Conference, 1997. Asia Pacific and International Computer Science Conference 1997. APSEC '97 and ICSC '97. Proceedings, (pp. 435--444).
[6]
Gray, P., & Welland, R. (1999). Increasing the flexibility of modelling tools via constraint-based specification. Proceedings of the 1999 conference of the Centre for Advanced Studies on Collaborative research. Mississauga, Ontario, Canada: IBM Press.
[7]
Hudson, S., & Hsi, C.-N. (1993). A Synergistic Approach to Specifying Simple Number Independent Layouts by Example. Conference on Human Factors in Computing Systems (pp. 285--292). New York, USA: ACM.
[8]
Isazadeh, H., & Lamb, D. A. (1997). CASE Environments and MetaCASE Tools. Kingston, Ontario, Canada: Department of Computing and Information Science, Queen's University.
[9]
Jankowski, D. (1997). Computer-Aided Systems Engineering Methodology Support and Its Effect on the Output of Structured Analysis. Empirical Softw. Eng., 2 (1), 11--38.
[10]
Kelly, S. (2009). Domain-Specific Modelling: A Toolmaker Perspective. Retrieved March 4, 2009, from MetaEdit+ Workbench questions: http://www.metacase.com/faq/showfaq.asp?cate=MWB
[11]
Kurlander, D., & Feiner, S. (1993). Inferring Constraints From Multiple Snapshots. ACM Transactions on Graphics, 12 (4), 277--304.
[12]
Liu, N., Hosking, J., & Grundy, J. (2007). A Visual Language and Environment for Specifying User Interface Event Handling in Design Tools. Conferences in Research and Practice in Information Technology Series; Vol. 241 (pp. 87--94). Ballarat, Victoria, Australia: Australian Computer Society, Inc., Australia.
[13]
Maulsby, D., & Witten, I. (1993). Metamouse: An Instructivle Agent for Programming by Demonstration. In A. Cypher, Watch What I Do: Programming by Demonstration. Cambridge: MIT Press.
[14]
MetaCase. (2009). MetaEdit+. Retrieved 7 5, 2009, from http://www.metacase.com/
[15]
Myers, B. (1993). Peridot: Creating User Interfaces by Demonstration. In A. Cypher, Watch What I Do: Programming by Demonstration. Cambridge, Mass.: MIT Press.
[16]
Myers, B., McDaniel, R., & Wolber, D. (2000). Intelligence in Demonstrational. Communications of the ACM, 43 (3), 82--89.
[17]
Offen, R. (2000). CASE Tools and Constraints. North Ryde: Macquarie University Joint Research Centre for Advanced Systems Engineering.
[18]
Prabaker, M., Bergman, L., & Castelli, V. (2006). An Evaluation of Using Programming by Demonstration and Guided Walkthrough Techniques for Authoring and Utilizing Documentation. Conference on Human Factors in Computing Systems, Proceedings of the SIGCHI conference on Human Factors in computing systems (pp. 241--250). Montréal, Québec, Canada: ACM, New York, USA.
[19]
Qattous, H. (2009). Constraint Specification by Example in a Meta-CASE Tool. Foundations of Software Engineering, Proceedings of the doctoral symposium for ESEC/FSE on Doctoral Symposium (pp. 13--16). Amsterdam: ACM.
[20]
Scott, L., Horvath, L., & Day, D. (2000). Characterising CASE Constraints. Communications of the ACM, 43 (11), 232--238.
[21]
Smith, D., Cypher, A., & Spohrer, J. (1994). KidSim: programming agents without a programming language. Communications of the ACM, 37, 54--67.
[22]
Zhu, N., Grundy, J., & Hosking, J. (2004). Pounamu: A Meta-Tool For Multi-View Visual Language Environment Construction. Proceedings of the 2004 IEEE Symposium on Visual Languages - Human Centric Computing (pp. 254--256). IEEE Computer Society.

Cited By

View all
  • (2019)FlexiSketchSoftware and Systems Modeling (SoSyM)10.1007/s10270-017-0623-818:2(1513-1541)Online publication date: 1-Apr-2019
  • (2015)Sketching and notation creation with FlexiSketch Team: Evaluating a new means for collaborative requirements elicitation2015 IEEE 23rd International Requirements Engineering Conference (RE)10.1109/RE.2015.7320421(186-195)Online publication date: Aug-2015
  • (2013)Semi-automatic generation of metamodels from model sketchesProceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2013.6693130(664-669)Online publication date: 11-Nov-2013
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEM '10: Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
September 2010
423 pages
ISBN:9781450300391
DOI:10.1145/1852786
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 September 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. domain specific language
  2. example-based interface
  3. meta-CASE tools
  4. programming by example
  5. specification by example
  6. user study

Qualifiers

  • Research-article

Conference

ESEM '10
Sponsor:

Acceptance Rates

ESEM '10 Paper Acceptance Rate 30 of 102 submissions, 29%;
Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)FlexiSketchSoftware and Systems Modeling (SoSyM)10.1007/s10270-017-0623-818:2(1513-1541)Online publication date: 1-Apr-2019
  • (2015)Sketching and notation creation with FlexiSketch Team: Evaluating a new means for collaborative requirements elicitation2015 IEEE 23rd International Requirements Engineering Conference (RE)10.1109/RE.2015.7320421(186-195)Online publication date: Aug-2015
  • (2013)Semi-automatic generation of metamodels from model sketchesProceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2013.6693130(664-669)Online publication date: 11-Nov-2013
  • (2012)Creating visual domain-specific modeling languages from end-user demonstrationProceedings of the 4th International Workshop on Modeling in Software Engineering10.5555/2664431.2664435(22-28)Online publication date: 2-Jun-2012
  • (2012)Creating visual Domain-Specific Modeling Languages from end-user demonstration2012 4th International Workshop on Modeling in Software Engineering (MISE)10.1109/MISE.2012.6226010(22-28)Online publication date: Jun-2012

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media