Skip to main content
SpringerLink
Log in

Software-Embedded Evaluation Support in Design Science Research

  • Conference paper
  • First Online:
Designing for a Digital and Globalized World (DESRIST 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10844))

Abstract

Even though the practice of integrating evaluative features into software has long been applied in commercially available software, it is still underrepresented in the Information Systems (IS) community. This paper presents a framework for embedded evaluation support. We are aware of the challenges of evaluation of socio-technical systems and take this issue into consideration in our framework. Our framework is the result of conceptualizations drawing from the evaluation discourse discussion on the topics of artifact evaluation in DSR. We also demonstrate our ideas through two examples of embedded evaluation support mechanisms designed and used in a DSR project in the Swedish healthcare sector.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Venable, J., Pries-Heje, J., Baskerville, R.: FEDS: a framework for evaluation in design science research. Eur. J. Inf. Syst. 25, 77–89 (2016)

    Article  Google Scholar 

  2. Hevner, A.R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28, 75–105 (2004)

    Article  Google Scholar 

  3. Le Goues, C., Nguyen, T., Forrest, S., Weimer, W.: GenProg: a generic method for automatic software repair. IEEE Trans. Softw. Eng. 38, 54–72 (2012)

    Article  Google Scholar 

  4. Le Goues, C., Forrest, S., Weimer, W.: Current challenges in automatic software repair. Softw. Qual. J. 21, 421–443 (2013)

    Article  Google Scholar 

  5. Sidiroglou, S., Laadan, O., Perez, C., Viennot, N., Nieh, J., Keromytis, A.D.: ASSURE: automatic software self-healing using rescue points. ACM SIGARCH Comput. Archit. News. 37, 37–48 (2009)

    Article  Google Scholar 

  6. Dave, R.H.: A built-in system of evaluation for reform projects and programmes in education. Int. Rev. Educ. 26, 475–482 (1980)

    Article  Google Scholar 

  7. Jervan, G., Peng, Z., Ubar, R., Kruus, H.: A hybrid BIST architecture and its optimization for SoC testing. In: Proceedings of the International Symposium on Quality Electronic Design 2002, pp. 273–279 (2002)

    Google Scholar 

  8. Nagle, H.T., Roy, S.C., Hawkins, C.F., McNamer, M.G., Fritzemeier, R.R.: Design for testability and built-in self test: a review. IEEE Trans. Ind. Electron. 36, 129–140 (1989)

    Article  Google Scholar 

  9. Agrawal, V.D., Kime, C.R., Saluja, K.K.: A tutorial on built-in self-test. I. Principles. IEEE Des. Test Comput. 10, 73–82 (1993)

    Article  Google Scholar 

  10. Agrawal, V.D., Kime, C.R., Saluja, K.K.: A tutorial on built-in self-test. 2. Applications. IEEE Des. Test Comput. 10, 69–77 (1993)

    Article  Google Scholar 

  11. Simm, W., Ferrario, M.A., Gradinar, A., Whittle, J.: Prototyping ‘clasp’: implications for designing digital technology for and with adults with autism. In: Proceedings of the 2014 Conference on Designing Interactive Systems, pp. 345–354 (2014)

    Google Scholar 

  12. Atterer, R., Wnuk, M., Schmidt, A.: Knowing the user’s every move: user activity tracking for website usability evaluation and implicit interaction. In: Proceedings of the 15th International Conference on World Wide Web, pp. 203–212 (2006)

    Google Scholar 

  13. Van Gemert-Pijnen, J.E.W.C., Kelders, S.M., Bohlmeijer, E.T.: Understanding the usage of content in a mental health intervention for depression: an analysis of log data. J. Med. Internet Res. 16 (2014)

    Google Scholar 

  14. Remenyi, D., Sherwood-Smith, M.: Maximise information systems value by continuous participative evaluation. Logist. Inf. Manag. 12, 14–31 (1999)

    Article  Google Scholar 

  15. Stufflebeam, D.L.: The CIPP model for evaluation. In: Kellaghan, T., Stufflebeam, D.L. (eds.) International Handbook of Educational Evaluation. Kluwer International Handbooks of Education, vol. 9, pp. 31–62. Springer, Heidelberg (2003). https://doi.org/10.1007/978-94-010-0309-4_4

    Chapter  Google Scholar 

  16. Smithson, S., Hirschheim, R.: Analysing information systems evaluation: another look at an old problem. Eur. J. Inf. Syst. 7, 158–174 (1998)

    Article  Google Scholar 

  17. Venable, J., Pries-Heje, J., Baskerville, R.: A comprehensive framework for evaluation in design science research. In: Peffers, K., Rothenberger, M., Kuechler, B. (eds.) DESRIST 2012. LNCS, vol. 7286, pp. 423–438. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29863-9_31

    Chapter  Google Scholar 

  18. Sein, M., Henfridsson, O., Purao, S., Rossi, M., Lindgren, R.: Action design research. MIS Q. 35, 37–56 (2011)

    Article  Google Scholar 

  19. Cronholm, S., Goldkuhl, G.: Strategies for information systems evaluation-six generic types. Electron. J. Inf. Syst. Eval. 6, 65–74 (2003)

    Google Scholar 

  20. Bunge, M.: Philosophy of Science: From Explanation to Justification, vol. 2. Transaction Publishers, New Brunswick (1998)

    Google Scholar 

  21. Ågerfalk, P.J., Sjöström, J.: Sowing the seeds of self: a socio-pragmatic penetration of the web artefact. In: ACM International Conference Proceeding Series (2007)

    Google Scholar 

  22. Grönqvist, H., Olsson, E.M.G., Johansson, B., Held, C., Sjöström, J., Norberg, A.L., Hovén, E., Sanderman, R., van Achterberg, T., von Essen, L.: Fifteen challenges in establishing a multidisciplinary research program on eHealth research in a university setting: a case study. J. Med. Internet Res. 19, e173 (2017)

    Article  Google Scholar 

  23. Sjöström, J., von Essen, L., Grönqvist, H.: The origin and impact of ideals in eHealth research: experiences from the U-CARE research environment. JMIR Res. Protoc. 3 (2014)

    Article  Google Scholar 

  24. Sjöström, J., Ågerfalk, P.J., Hevner, A.R.: Scrutinizing privacy and accountability in online psychosocial care. IT Prof. 19, 45–51 (2017)

    Article  Google Scholar 

  25. Sjöström, J., Hermelin, M.: In-place translation in information systems development. In: Helfert, M., Donnellan, B. (eds.) EDSS 2012. CCIS, vol. 388, pp. 88–98. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-04090-5_9

    Chapter  Google Scholar 

  26. Sjöström, J., Alfonsson, S.: Supporting the therapist in online therapy. In: ECIS 2012 Proceedings, Barcelona, Spain (2012)

    Google Scholar 

  27. Mustafa, M.I., Sjöström, J.: Design principles for research data export: lessons learned in e-health design research. In: vom Brocke, J., Hekkala, R., Ram, S., Rossi, M. (eds.) DESRIST 2013. LNCS, vol. 7939, pp. 34–49. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38827-9_3

    Chapter  Google Scholar 

  28. Sjöström, J., Rahman, M.H., Rafiq, A., Lochan, R., Ågerfalk, P.J.: Respondent behavior logging: an opportunity for online survey design. In: vom Brocke, J., Hekkala, R., Ram, S., Rossi, M. (eds.) DESRIST 2013. LNCS, vol. 7939, pp. 511–518. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38827-9_44

    Chapter  Google Scholar 

  29. Iivari, J.: Distinguishing and contrasting two strategies for design science research. Eur. J. Inf. Syst. 24(1), 107–115 (2015). Nature Publishing Group

    Article  Google Scholar 

  30. Purao, S., Baldwin, C.I., Hevner, A.R., Storey, V.C., Pries-Heje, J., Smith, B., Zhu, Y.: The Sciences of Design: Observations on an Emerging Field. Working Paper 09-056. Harvard Business School (2008)

    Google Scholar 

  31. Baskerville, R.L., Kaul, M., Storey, V.C.: Genres of inquiry in design science research: justification and evaluation of knowledge production. MIS Quart. 39(3), 541–564 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics and Media, Uppsala University, Uppsala, Sweden

    Jonas Sjöström

  2. Institute of Information Systems, University of Liechtenstein, ‎Vaduz, Liechtenstein

    Leona Chandra Kruse

  3. Division of Informatics, University West, Trollhättan, Sweden

    Amir Haj-Bolouri & Per Flensburg

Authors
  1. Jonas Sjöström
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leona Chandra Kruse
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amir Haj-Bolouri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Per Flensburg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonas Sjöström .

Editor information

Editors and Affiliations

  1. School of Information Systems and Technology, Claremont Graduate University, Claremont, California, USA

    Samir Chatterjee

  2. Muma College of Business, University of South Florida, Tampa, Florida, USA

    Kaushik Dutta

  3. Department of Management Studies, IIT Madras, Chennai, Tamil Nadu, India

    Rangaraja P. Sundarraj

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sjöström, J., Kruse, L.C., Haj-Bolouri, A., Flensburg, P. (2018). Software-Embedded Evaluation Support in Design Science Research. In: Chatterjee, S., Dutta, K., Sundarraj, R. (eds) Designing for a Digital and Globalized World. DESRIST 2018. Lecture Notes in Computer Science(), vol 10844. Springer, Cham. https://doi.org/10.1007/978-3-319-91800-6_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91800-6_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91799-3

  • Online ISBN: 978-3-319-91800-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation