Skip to main content
SpringerLink
Log in

Semiotic Engineering: A Cohering Theory to Connect EUD with HCI, CMC and More

  • Chapter
  • First Online:
New Perspectives in End-User Development

Abstract

Theories have an important role to play in research areas whose application faces rapid technological changes. They can provide longer-term intellectual references that shape deeper investigations and contribute to consolidate the identity of such research areas. A recent survey of EUD-related work published between 2004 and 2013 suggests that our field is remarkably techno-centered and could increase its scientific impact by diversifying some of its research approaches and practices. In this paper we show concrete examples of how Semiotic Engineering, originally a semiotic theory of human-computer interaction, can provide a unified theoretical framing for various EUD-related topics of investigation. Our contribution to the collection of chapters in this book is to demonstrate this particular theory’s potential as a catalyst of new kinds of transdisciplinary debate, as well as a source of inspiration for new breeds of technological developments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Tableau – http://www.tableau.com/

  2. 2.

    Klipfolio – https://www.klipfolio.com/

  3. 3.

    iMacros – http://imacros.net/

  4. 4.

    UiPath – http://www.uipath.com/

  5. 5.

    IFTTT – https://ifttt.com/recipes

  6. 6.

    Microsoft Flow – https://flow.microsoft.com/en-us/

  7. 7.

    Cayenne - http://www.cayenne-mydevices.com/

  8. 8.

    App Inventor - http://appinventor.mit.edu/explore/

  9. 9.

    AppsGeyser - https://www.appsgeyser.com/

  10. 10.

    Scratch – https://scratch.mit.edu/

  11. 11.

    Alice - http://www.alice.org/index.php

  12. 12.

    AgentSheets – http://www.agentsheets.com/

  13. 13.

    Kidsim has been known as Stagecast Creator since 1997 – more about it at http://web.archive.org/web/20150517004640/http://www.stagecast.com/index.html

  14. 14.

    PoliFacets sample projects can be accessed at http://www.serg.inf.puc-rio.br/polifacets/project/list.lua?lg=en&page=example. The user can use the search box to access “Fireworks,” “AnotherFireworks” and “YetAnotherFireworks” more quickly. To run the programs, a Java-enabled browser is required.

References

  • Andersen, P. B. (1997). A theory of computer semiotics: semiotic approaches to construction and assessment of computer systems. Cambridge: Cambridge University Press.

    Google Scholar 

  • Andersen, P. B. (1985). Semiotics and informatics: computers as media Proceedings of the Conference on Information Technology for Information Use. Copenhagen: The Royal School of Librarianship.

    Google Scholar 

  • Andersen, P. B., Holmqvist, B., Jensen, J. F. (1993). The computer as medium. Cambridge: Cambridge University Press.

    Google Scholar 

  • Arias, E. G., Eden, H., Fischer, G. (2015). The envisionment and discovery collaboratory (EDC): explorations in human-centered informatics with tabletop computing environments. San Rafael, California: Morgan & Claypool Publishers.

    Google Scholar 

  • Barbosa, S. D. J., & de Souza, C. S. (2001). Extending software through metaphors and metonymies. Knowledge-Based Systems, 14(1), 15–27.

    Article  Google Scholar 

  • Blackwell, A. (2006). Psychological issues in end-user programming. In H. Liberman, F. Parternò, V. Wulf (Eds.). End user development (pp. 9–30). Dordrecht, Netherlands: Springer.

    Chapter  Google Scholar 

  • Blackwell, A., & Green, T. (2003). Notational systems–the cognitive dimensions of notations framework. In J. Carroll (Ed.), HCI models, theories, and frameworks: toward an interdisciplinary science (pp. 103–133). San Francisco, Calif: Morgan Kaufmann.

    Google Scholar 

  • Blandford, A., Furniss, D., Makri, S. (2016). “Qualitative HCI research: going behind the scenes.” Synthesis Lectures on Human-Centered Informatics, 9(1), 1–115.

    Article  Google Scholar 

  • Boden, M. A., & Edmonds, E. A. (2009). What is generative art? Digital Creativity, 20, 21–46.

    Article  Google Scholar 

  • Burke, Q., & Kafai, Y. B. (2010). Programming & storytelling: opportunities for learning about coding & composition. In Proceedings of the 9th International Conference on Interaction Design and Children (IDC ’10) (pp. 348–351). New York, NY, USA: ACM. doi:10.1145/1810543.1810611.

    Google Scholar 

  • Burke, Q., & Kafai, Y. B. (2012). The writers’ workshop for youth programmers: digital storytelling with scratch in middle school classrooms. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (SIGCSE ’12) (pp. 433–438). New York, NY, USA: ACM. doi:10.1145/2157136.2157264.

    Google Scholar 

  • Bush, V. (1990). Science: the endless frontier: a report to the president on a program for postwar scientific research. Washington, DC: National Science Foundation. 192 p.

    Google Scholar 

  • Calvo, R. A., & Peters, D. (2013). The irony and re-interpretation of our quantified self. In H. Shen, R. Smith, J. Paay, P. Calder & T. Wyeld (Eds.), Proceedings of the 25th Australian computer-human interaction conference: augmentation, application, innovation, collaboration (OzCHI ’13) (pp. 367–370). New York, NY, USA: ACM. doi:10.1145/2541016.2541070.

    Google Scholar 

  • de Souza, C., Barbosa, S., da Silva, S. (2001). Semiotic engineering principles for evaluating end-user programming environments. Interacting with Computers, 13, 467–495.

    Article  Google Scholar 

  • de Souza, C. S. (1993). The semiotic engineering of user interface languages. International Journal of Man-Machine Studies, Academic Press, 39, 753–773.

    Article  Google Scholar 

  • de Souza, C. S. (2005a). The semiotic engineering of human-computer interaction. Cambridge, MA: The MIT Press.

    Google Scholar 

  • de Souza, C. S. (2005b). Semiotic engineering: bringing designers and users together at interaction time. Interacting with Computers, 17(3), 317–341.

    Article  Google Scholar 

  • de Souza, C. S. (2013). Semiotic perspectives on interactive languages for life on the screen. Journal of Visual Languages & Computing, 24(3), 218–221.

    Article  Google Scholar 

  • de Souza, C. S., & Barbosa, S. D. J. (2006). A semiotic framing for end-user development. In H. Lieberman, F. Paternò, V. Wulf (Eds.). End user development (pp. 401–426). Netherlands: Springer.

    Chapter  Google Scholar 

  • de Souza, C. S., Cerqueira, R. F. G., Afonso, L. M., Brandão, R. M. R., Ferreira, J. S. J. (2016). Software developers as users: semiotic investigations in human-centered software development. Cham: Springer International Publishing.

    Book  Google Scholar 

  • de Souza, C. S., Garcia, A. C. B., Slaviero, C., Pinto, H., Repenning, A. (2011). End-user development: third international symposium, IS-EUD 2011, Torre Canne (BR), Italy, June 7–10, 2011. Proceedings (pp. 155–170). Berlin, Heidelberg: Springer Berlin Heidelberg.

    Book  Google Scholar 

  • de Souza, C. S., & Leitão, C. F. (2009). Semiotic engineering methods for scientific research in HCI. Synthesis Lectures on Human-Centered Informatics, 2(1), 1–122.

    Article  Google Scholar 

  • de Souza, C. S., Salgado, L. C., Leitão, C. F., Serra, M. M. (2014). Cultural appropriation of computational thinking acquisition research: seeding fields of diversity. In Proceedings of the 2014 conference on Innovation & technology in computer science education (ITiCSE ’14) (pp. 117–122). New York, NY, USA: ACM. doi:10.1145/2591708.2591729.

    Google Scholar 

  • diSessa, A. A., & Cobb, P. (2004). Ontological innovation and the role of theory in design experiments. The Journal of the Learning Sciences, 13(1), 77–103.

    Article  Google Scholar 

  • Edmonds, E. A. (2007) Reflections on the Nature of Interaction CoDesign, 3, 139–143.

    Google Scholar 

  • Ferreira, J. J., de Souza, C. S., de Castro Salgado, L. C., Slaviero, C., Leitão, C. F., de F. Moreira F. (2012), Combining cognitive, semiotic and discourse analysis to explore the power of notations in visual programming. In 2012 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)., September, 2012 (pp. 101–108). Los Alamitos, CA: IEEE Computer Society.

    Google Scholar 

  • Fischer, G., & Giaccardi, E. (2006). Meta-design: a framework for the future of end-user development. End user development (pp. 427–457). Netherlands: Springer.

    Chapter  Google Scholar 

  • Franchi, S., & Güzeldere, G. (2005). Machinations of the mind: cybernetics and artificial intelligence from automata to cyborgs. In S. Franchi, & G. Güzeldere (Eds.). Mechanical bodies, computational minds (pp. 15–149). Cambridge, MA: The MIT Press.

    Google Scholar 

  • Gerd Waloszek. (2005). Book review: the semiotic engineering of human-computer interaction. https://experience.sap.com/archived/review_semiotic_eng/ Accessed 24 May 2016.

  • Green, T. (2006). Aims, achievements, agenda—where CDs stand now. Journal of Visual Languages & Computing, 17(4), 288–291.

    Article  Google Scholar 

  • Green, T. R. G., & Petre, M. (1996). Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. Journal of Visual Languages & Computing, 7(2), 131–174.

    Article  Google Scholar 

  • Greenwood, D. J., & Levin, M. (2007). Introduction to action research. Social research for social change (2nd Edition) Thousand Oaks, CA: Sage Publications, Inc.

    Book  Google Scholar 

  • Hundhausen, C. D. (2005). Using end user visualization environments to mediate conversations: a ‘communicative dimensions’ framework. Journal of Visual Languages and Computing, 16(3), 153–185.

    Article  Google Scholar 

  • Intrator, C. (2009). Using web scripts to improve accessibility. MSc dissertation. Departamento de Informática. PUC-Rio. 103 p. http://www2.dbd.puc-rio.br/pergamum/tesesabertas/0711270_09_pretextual.pdf.

  • Intrator, C., & de Souza, C. S. (2008). Using Web scripts to improve accessibility. In IHC 2008 - VIII Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais, 2008, Porto Alegre. Proceedings of the VIII Brazilian Symposium on Human Factors in Computing Systems (v. 378. pp. 292–295). New York: ACM.

    Google Scholar 

  • Jones, S. P., Blackwell, A., Burnett, M. (2003). A user-centred approach to functions in Excel. In Proceedings of the eighth ACM SIGPLAN international conference on Functional programming (ICFP ’03) (pp. 165–176). New York: ACM.

    Google Scholar 

  • Kammersggard, J. (1988). Four different perspectives on human–computer interaction. International Journal of Man-Machine Studies, 28, 343–362.

    Article  Google Scholar 

  • Kelleher, C., & Pausch, R. (2007). Using storytelling to motivate programming. Communications of the ACM, 50, 58–64.

    Article  Google Scholar 

  • Kuutti, K. (1996). Activity theory as a potential framework for human-computer interaction research. In Context and consciousness: activity theory and human-computer interaction (pp. 17–44). Cambridge, Mass.: MIT Press.

    Google Scholar 

  • Leshed, G., Haber, E. M., Matthews, T., Lau, T. (2008). CoScripter: automating & sharing how-to knowledge in the enterprise. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08) (pp. 1719–1728). New York, NY, USA: ACM. doi:10.1145/1357054.1357323.

    Google Scholar 

  • Lieberman, H., Paternò, F., Wulf, V. (Eds.) (2006). End user development. Netherlands, Dordrecht: Springer.

    Google Scholar 

  • Liu, K. (2000). Semiotics in information systems engineering. Cambridge: Cambridge University Press.

    Google Scholar 

  • Maceli, M., & Atwood, M. E. (2011). From human crafters to human factors to human actors and back again: bridging the design time–use time divide. In International symposium on end user development (pp. 76–91). Berlin Heidelberg: Springer.

    Google Scholar 

  • Maceli, M., & Atwood, M. E. (2013). “Human crafters” once again: supporting users as designers in continuous co-design. In International symposium on end user development (pp. 9–24). Berlin Heidelberg: Springer.

    Google Scholar 

  • Mehra, A., Grundy J., Hosking, J. (2005). A generic approach to supporting diagram differencing and merging for collaborative design. In Proceedings of the 20th IEEE/ACM international conference on automated software engineering (ASE ’05) (pp. 204–213). New York: ACM.

    Google Scholar 

  • Monteiro, I. T., da Silva Alves, A., de Souza, C. S. (2013). Using mediated communication to teach vocational concepts to deaf users. In C. Stephanidis, & M. Antona (Eds.). Universal access in human-computer interaction. applications and services for quality of life: 7th international conference, UAHCI 2013, held as part of HCI international 2013, Las Vegas, NV, USA, July 21–26, 2013, Proceedings, Part III (pp. 213–222). Berlin Heidelberg: Springer.

    Chapter  Google Scholar 

  • Monteiro, I. T., & de Souza, C. S. (2012). The representation of self in mediated interaction with computers. In Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems (IHC ’12) (pp. 219–228). Porto Alegre, Brazil: Brazilian Computer Society.

    Google Scholar 

  • Monteiro, I. T., de Souza, C. S., Tolmasquim, E. T. (2015). My program, my world: insights from 1st-person reflective programming in EUD education. In P. Díaz, V. Pipek, C. Ardito, C. Jensen, I. Aedo, A. Boden (Eds.), End-user development: 5th international symposium, IS-EUD 2015, Madrid, Spain, May 26–29, 2015. Proceedings (pp. 76–91). Cham: Springer.

    Google Scholar 

  • Monteiro, I. T., de Castro Salgado, L. C., Mota, M. P., Sampaio, A. L., de Souza, C. S. (June 2017). Signifying software engineering to computational thinking learners with AgentSheets and PoliFacets. Journal of Visual Languages & Computing, 40, 91–112. doi:10.1016/j.jvlc.2017.01.005.

    Article  Google Scholar 

  • Monteiro, I. T., Tolmasquim, E. T., & de Souza, C. S. (2013). Going back and forth in metacommunication threads. In Proceedings of the 12th Brazilian symposium on human factors in computing systems (IHC ’13). Brazilian computer society, Porto Alegre, Brazil (pp. 102–111).

    Google Scholar 

  • Mota, M. P., Faria, L. S., de Souza, C. S. (2012). Documentation comes to life in computational thinking acquisition with agentsheets. In Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems (IHC ’12) (pp. 151–160). Porto Alegre, Brazil: Brazilian Computer Society.

    Google Scholar 

  • Mota, M. P., Monteiro, I. T., Ferreira, J. J., Slaviero, C., de Souza, C. S. (2013). On signifying the complexity of inter-agent relations in AgentSheets games and simulations. In Proceedings of the 31st ACM international conference on design of communication (pp. 133–142). New York: ACM.

    Google Scholar 

  • Nadin, M. (1988). Interface design and evaluation – semiotic implications. In H. R. Hartson, & D. Hix (Ed.), Advances in human-computer interaction ablex (v. 2, pp. 45–100).

    Google Scholar 

  • Nadin, M. (2011). Computation, information, meaning. Anticipation and Games International Journal of Applied Research on Information Technology and Computing, 2, 50–76.

    Google Scholar 

  • Nake, F. (2005). Computer art: a personal recollection. In Proceedings of the 5th conference on Creativity & cognition (C&C ’05) (pp. 54–62). New York, NY, USA: ACM. doi:10.1145/1056224.1056234.

    Chapter  Google Scholar 

  • Nake, F., & Grabowski, S. (2001). Human–computer interaction viewed as pseudo-communication. Knowledge-Based Systems, 14, 441–447.

    Article  Google Scholar 

  • Norman, D. A. (1986). Cognitive engineering. In D. Norman, & S. Draper (Eds.), User centered systems design (pp. 31–629). Hillsdale, N.J.: L. Erlbaum Associates.

    Google Scholar 

  • Norman, D. A. (2009). THE WAY I SEE IT: Systems thinking: a product is more than the product. Interactions, 16, 52–54.

    Article  Google Scholar 

  • Oberquelle, H., Kupka, I., Maass, S. (1983). A view of human—machine communication and co-operation. International Journal of Man-Machine Studies, 19, 309–333.

    Article  Google Scholar 

  • Peirce, C. S. (1992). The Essential Peirce. Selected Philosophical Writings Volume 1 (1987–1893) edited by Nathan Houser and Christian Kloesel. Bloomington IN: Indiana University Press.

    Google Scholar 

  • Peirce, C. S. (1998). The Essential Peirce. Selected Philosophical Writings Volume 2 (1893–1913) edited by Nathan Houser and Christian Kloesel. Bloomington IN: Indiana University Press.

    Google Scholar 

  • Repenning, A. (2011). Making programming more conversational. 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (pp. 191–194). Los Alamitos, CA: IEEE Computer Society.

    Chapter  Google Scholar 

  • Repenning, A., & Ioannidou, A. (2004). Agent-based end-user development. Communications of the ACM, 47(9), 43–46.

    Article  Google Scholar 

  • Repenning, A., & Ioannidou, A. (2006). What makes end-user development tick? 13 design guidelines. In H. Lieberman, F. Paternò, V. Wulf, (Eds.). End user development (pp. 51–85). Netherlands: Springer.

    Chapter  Google Scholar 

  • Repenning, A., Webb, D., Ioannidou, A. (2010). Scalable game design and the development of a checklist for getting computational thinking into public schools. In Proceedings of the 41st ACM technical symposium on Computer science education (SIGCSE ’10) (pp. 265–269). New York, NY, USA: ACM. doi:10.1145/1734263.1734357.

    Chapter  Google Scholar 

  • Rogers, Y. (2012). HCI theory: classical, modern, and contemporary. Synthesis Lectures on Human-Centered Informatics 5.2, 1–129.

    Google Scholar 

  • Salgado, L. C. C., Leitão, C. F., de Souza, C. (2012). A journey through cultures: metaphors for guiding the design of cross-cultural interactive systems. London; New York: Springer.

    Google Scholar 

  • Searle, J. R. (1985). Expression and meaning: studies in the theory of speech acts. Cambridge: Cambridge University Press.

    Google Scholar 

  • Simon, H. A. (1996). The sciences of the artificial. 3rd Edition Cambridge, MA: The MIT Press.

    Google Scholar 

  • Stokes, D. E. (2011). Pasteur’s quadrant: basic science and technological innovation. Washington, DC: Brookings Institution Press.

    Google Scholar 

  • Tanaka-Ishii, K. (2010). Semiotics of programming. Cambridge: Cambridge University Press.

    Google Scholar 

  • Tetteroo, D., & Markopoulos. (2015). A review of research methods in end user development. In P. Díaz, V. Pipek, C. Ardito, C. Jensen, I. Aedo, A. Boden (Eds.), End-user development: 5th international symposium, IS-EUD 2015, Madrid, Spain, May 26–29, 2015 (pp. 58–75). Cham: Springer.

    Google Scholar 

  • Turkle, S. (2005). The second self: computers and the human spirit. Twentieth Anniversary Edition. Cambridge, MA: The MIT Press.

    Google Scholar 

  • Venable, J. (2006). The role of theory and theorising in design science research. In S. Chatterjee & A. Hevner (Eds), Proceedings of the First International Conference on Design Science in Information Systems and Technology (DESRIST 2006) (pp. 1–18). Claremont, CA: Claremont Graduate University.

    Google Scholar 

  • Winograd, T. (1997). The design of interaction. In P. Denning (Ed.). Beyond calculation: the next fifty years of computing (pp. 149–161). New York: Springer.

    Chapter  Google Scholar 

  • Winograd, T. (2006). Designing a new foundation for design. Communications of the ACM, 49, 71–74.

    Article  Google Scholar 

  • Winograd, T., & Flores, F. (1986). Understanding computers and cognition: a new foundation for design. Boston, Mass.: Addison-Wesley.

    Google Scholar 

  • Wolz, U., Stone, M., Pearson, K., Pulimood, S. M., Switzer, M. (2011). Computational thinking and expository writing in the middle school. Transactions on Computing Education, 11, 9:1–9:22.

    Article  MATH  Google Scholar 

Download references

Acknowledgements

I thank CNPq, FAPERJ and The AMD Foundation for financial support to the research reported in this paper. I am also grateful to Alex Repenning, AgentSheets Inc. and all the participants, researchers and partner schools involved in SGD-Br. I am additionally thankful to Ernest Edmonds and Alan Blackwell for comments, suggestions and interesting discussions about some of the ideas exposed in this paper. Finally, I am greatly obliged to the anonymous reviewers whose insightful comments have helped me improve this chapter significantly.

Author information

Authors and Affiliations

  1. Semiotic Engineering Research Group (SERG), Departamento de Informática, PUC-Rio, Rio de Janeiro, Brazil

    Clarisse Sieckenius de Souza

Authors
  1. Clarisse Sieckenius de Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Clarisse Sieckenius de Souza .

Editor information

Editors and Affiliations

  1. ISTI - C. N. R. , Pisa, Italy

    Fabio Paternò

  2. FB 5, Wirtschaftsinf.k u. N. Medien, Universität Siegen FB 5, Wirtschaftsinf.k u. N. Medien, Siegen, Nordrhein-Westfalen, Germany

    Volker Wulf

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

de Souza, C.S. (2017). Semiotic Engineering: A Cohering Theory to Connect EUD with HCI, CMC and More. In: Paternò, F., Wulf, V. (eds) New Perspectives in End-User Development. Springer, Cham. https://doi.org/10.1007/978-3-319-60291-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-60291-2_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-60290-5

  • Online ISBN: 978-3-319-60291-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation