skip to main content
10.1145/3627508.3638327acmotherconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Design Principles for a Study Planning Assistant in Higher Education

Published: 10 March 2024 Publication History

Abstract

Digital study assistants (DSA) aim to support the challenging tasks of searching for relevant information and organizing this data for individual study planning. Although such systems are characterized by complex process flows, their user experience (UX) has rarely been examined adequately. This research comprehensively analyzes the UX of such a system at the University of Bamberg, which includes short-term planning for one semester as well as the distinctive feature of long-term planning beyond one semester. Via remote usability testing including an online questionnaire and involving 26 participants, this study explores students’ interactions with the system and evaluates the impact of the system design on the UX, identifying strengths and weaknesses. The study has revealed that participants faced major challenges related to complex processes resulting from the lack of functional and terminological differentiation between short- and long-term study planning for users. In addition, certain features, including extended search options, were hidden and could not be found immediately. Derived from these findings, we present nine design principles to guide the development of effective DSA and similar support systems.

References

[1]
Ahmed S Alghamdi, Alih Al-Badi, Roobaea Alroobaea, and Pamj Mayhew. 2013. A comparative study of synchronous and asynchronous remote usability testing methods. International Review of Basic and Applied Sciences 1, 3 (2013), 61–97.
[2]
Morten Sieker Andreasen, Henrik Villemann Nielsen, Simon Ormholt Schrøder, and Jan Stage. 2007. What happened to remote usability testing? An empirical study of three methods. In Proceedings of the SIGCHI conference on Human factors in computing systems. 1405–1414.
[3]
Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies 4, 3 (2009), 114–123.
[4]
John Brooke. 1996. Sus: a “quick and dirty’usability. Usability evaluation in industry 189, 3 (1996), 189–194.
[5]
John Brooke. 2013. SUS: a retrospective. Journal of usability studies 8, 2 (2013), 29–40.
[6]
Thorsten Dresing, Udo Kuckartz, and Stefan Rädiker. 2007. Qualitative Evaluation: Der Einstieg in die Praxis. VS Verlag für Sozialwissenschaften! GWV Fachverlage GmbH, Wiesbaden.
[7]
Mingming Fan, Jinglan Lin, Christina Chung, and Khai N. Truong. 2019. Concurrent Think-Aloud Verbalizations and Usability Problems. ACM Trans. Comput.-Hum. Interact. 26, 5, Article 28 (jul 2019), 35 pages. https://doi.org/10.1145/3325281
[8]
Daniel Hechler and Peer Pasternack. 2017. Das Elektronische Hochschulökosystem. die hochschule 2017, 1 (Jan. 2017), 7–18.
[9]
Andreas Henrich, Stefanie Sieber, and Sven-Uwe Wolf. 2007. Integration eines hochschulweiten LMS in die Systemlandschaft einer Hochschule–ein pragmatischer Ansatz. Flexibel integrierbares e-Learning–Nahe Zukunft oder Utopie (2007), 57–76.
[10]
Tobias Hirmer, Jana Etschmann, and Andreas Henrich. 2022. Requirements and Prototypical Implementation of a Study Planning Assistant in CS Programs. In 2022 International Symposium on Educational Technology (ISET). 281–285. https://doi.org/10.1109/ISET55194.2022.00066
[11]
Sven Judel, Rene Roepke, Maximilian Azendorf, and Ulrik Schroeder. 2023. Supporting Individualized Study Paths Using an Interactive Study Planning Tool. In 21. Fachtagung Bildungstechnologien (DELFI). Gesellschaft für Informatik e.V., Bonn, 225–230. https://doi.org/10.18420/delfi2023-36
[12]
Christin Karrenbauer, Claudia M. König, and Michael H. Breitner. 2021. Individual Digital Study Assistant for Higher Education Institutions: Status Quo Analysis and Further Research Agenda. In Innovation Through Information Systems: Volume III: A Collection of Latest Research on Management Issues. Springer International Publishing, Cham, 108–124. https://doi.org/10.1007/978-3-030-86800-0_8
[13]
Timothy L Keiningham, Lerzan Aksoy, Bruce Cooil, Tor Wallin Andreassen, and Luke Williams. 2008. A holistic examination of Net Promoter. Journal of Database Marketing & Customer Strategy Management 15, 2 (2008), 79–90.
[14]
Steve Krug. 2009. Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems. New Riders.
[15]
Udo Kuckartz. 2007. Einführung in die computergestützte Analyse qualitativer Daten. VS, Verl. für Sozialwiss., Wiesbaden.
[16]
Udo Kuckartz and Stefan Rädiker. 2022. Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung: Grundlagentexte Methoden, 5. Auflage. ed, Grundlagentexte Methoden. Beltz Juventa.
[17]
Kultusministerkonferenz. 2010. Ländergemeinsame Strukturvorgaben für die Akkreditierung von Bachelor und Masterstudiengängen: (Beschluss der Kultusministerkonferenz vom 10.10.2003 i.d.F. vom 04.02.2010). https://www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/2003/2003_10_10-Laendergemeinsame-Strukturvorgaben.pdf [last accessed: 03.01.2024].
[18]
Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In HCI and Usability for Education and Work: 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2008, Graz, Austria, November 20-21, 2008. Proceedings 4. Springer, 63–76.
[19]
James R Lewis. 1994. Sample sizes for usability studies: Additional considerations. Human factors 36, 2 (1994), 368–378.
[20]
Martin Maguire and Nigel Bevan. 2002. User requirements analysis: a review of supporting methods. In IFIP World Computer Congress, TC 13. Springer, 133–148.
[21]
Frank Multrus, Sandra Majer, Tino Bargel, and Monika Schmidt. 2017. Studiensituation Und Studentische Orientierungen: 13. Studierendensurvey an Universitäten und Fachhochschulen.
[22]
Jakob Nielsen. 1994. Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 152–158.
[23]
Jakob Nielsen. 1994. Severity Ratings for Usability Problems. online:< https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/>, ISSN (1994).
[24]
Jakob Nielsen. 1994. Usability engineering. Morgan Kaufmann.
[25]
Jakob Nielsen. 2000. Why you only need to test with 5 users. Nielsen Norman Group. online:< https://www. nngroup. com/articles/why-you-only-need-to-test-with-5-users/>, ISSN (2000), 0737–8939.
[26]
Michaela Ochs, Tobias Hirmer, and Andreas Henrich. 2023. Concept and Possible Impacts of a Study Planning Assistant in Higher Education. (2023). unpublished.
[27]
Michaela Ochs, Tobias Hirmer, and Andreas Henrich. 2023. Studierende und die Studienplanung: Untersuchung von Herausforderungen und Entwicklungsperspektiven eines digitalen Studienplanungsassistenten. https://doi.org/10.5281/zenodo.8232557
[28]
Frederick F Reichheld. 2003. The one number you need to grow. Harvard business review 81, 12 (2003), 46–55.
[29]
Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2014. Applying the user experience questionnaire (UEQ) in different evaluation scenarios. In Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience: Third International Conference, DUXU 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proceedings, Part I 3. Springer, 383–392.
[30]
Herbert Alexander Simon and K Anders Ericsson. 1984. Protocol analysis: Verbal reports as data. MIT.
[31]
Robert A Virzi. 1992. Refining the test phase of usability evaluation: How many subjects is enough?Human factors 34, 4 (1992), 457–468.

Cited By

View all
  • (2024)Baula – die digitale Studienplanungsassistentin an der Universität BambergBaula – Digital study planning assistant at the University of BambergZeitschrift für Hochschulentwicklung10.21240/zfhe/19-4/0219:4(15-36)Online publication date: 22-Dec-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
CHIIR '24: Proceedings of the 2024 Conference on Human Information Interaction and Retrieval
March 2024
481 pages
ISBN:9798400704345
DOI:10.1145/3627508
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 March 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. design principles
  2. information-intensive decision tasks
  3. study planning assistant
  4. usability study

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Stiftung Innovation in der Hochschullehre
  • Künstliche Intelligenz in der Hochschulbildung

Conference

CHIIR '24

Acceptance Rates

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)86
  • Downloads (Last 6 weeks)8
Reflects downloads up to 30 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Baula – die digitale Studienplanungsassistentin an der Universität BambergBaula – Digital study planning assistant at the University of BambergZeitschrift für Hochschulentwicklung10.21240/zfhe/19-4/0219:4(15-36)Online publication date: 22-Dec-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media