research-article

Towards Value-Sensitive Learning Analytics Design

Authors:

Haiyi ZhuAuthors Info & Claims

LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge

Pages 343 - 352

https://doi.org/10.1145/3303772.3303798

Published: 04 March 2019 Publication History

Abstract

To support ethical considerations and system integrity in learning analytics, this paper introduces two cases of applying the Value Sensitive Design methodology to learning analytics design. The first study applied two methods of Value Sensitive Design, namely stakeholder analysis and value analysis, to a conceptual investigation of an existing learning analytics tool. This investigation uncovered a number of values and value tensions, leading to design trade-offs to be considered in future tool refinements. The second study holistically applied Value Sensitive Design to the design of a recommendation system for the Wikipedia WikiProjects. To proactively consider values among stakeholders, we derived a multi-stage design process that included literature analysis, empirical investigations, prototype development, community engagement, iterative testing and refinement, and continuous evaluation. By reporting on these two cases, this paper responds to a need of practical means to support ethical considerations and human values in learning analytics systems. These two cases demonstrate that Value Sensitive Design could be a viable approach for balancing a wide range of human values, which tend to encompass and surpass ethical issues, in learning analytics design.

References

[1]

ACM US Public Policy Council. 2017. Statement on Algorithmic Transparency and Accountability. Online.

[2]

Simon Buckingham Shum. 2017. Black box learning analytics? Beyond algorithmic transparency. Keynote presented at the Learning Analytics Summer Institute, Ann Arbor, MI.

[3]

Bodong Chen, Yu-Hui Chang, Fan Ouyang, and Wanying Zhou. 2018. Fostering student engagement in online discussion through social learning analytics. The Internet and Higher Education 37 (2018), 21--30.

[4]

Dan Cosley, Dan Frankowski, Loren Terveen, and John Riedl. 2007. SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia. In Proceedings of the 12th International Conference on Intelligent User Interfaces (IUI '07). ACM, New York, NY, USA, 32--41.

Digital Library

[5]

Ulrike Cress, Johannes Moskaliuk, and Heisawn Jeong. 2016. Mass Collaboration and Education. Springer.

[6]

Betul C Czerkawski. 2016. Networked learning: design considerations for online instructors. Interactive Learning Environments 24, 8 (Nov. 2016), 1850--1863.

[7]

Alexei Czeskis, Ivayla Dermendjieva, Hussein Yapit, Alan Borning, Batya Friedman, Brian Gill, and Tadayoshi Kohno. 2010. Parenting from the pocket: value tensions and technical directions for secure and private parent-teen mobile safety. In Proceedings of the Sixth Symposium on Usable Privacy and Security. ACM, New York, NY, USA, 15.

Digital Library

[8]

Janet Davis and Lisa P Nathan. 2013. Value Sensitive Design: Applications, Adaptations, and Critiques. In Handbook of Ethics, Values, and Technological Design: Sources, Theory, Values and Application Domains, Jeroen van den Hoven, Pieter E Vermaas, and Ibo van de Poel (Eds.). Springer Netherlands, Dordrecht, 1--26.

[9]

Shane Dawson, Aneesha Bakharia, and Elizabeth Heathcote. 2010. SNAPP: Realising the affordances of real-time SNA within networked learning environments. In Proceedings of the 7th International Conference on Networked Learning, Lone Dirckinck-Holmfeld, Vivien Hodgson, Chris Jones, Maarten de Laat, David McConnell, and Thomas Ryberg (Eds.). Lancaster University, Lancaster, UK, 125--133.

[10]

Nia M M Dowell, Tristan M Nixon, and Arthur C Graesser. 2018. Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior research methods (Aug. 2018).

[11]

Hendrik Drachsler and Wolfgang Greller. 2016. Privacy and analytics: It's a DELICATE issue a checklist for trusted learning analytics. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. ACM, New York, NY, USA, 89--98.

Digital Library

[12]

Hendrik Drachsler, Tore Hoel, Adam Cooper, Gábor Kismihók, Alan Berg, Maren Scheffel, Weiqin Chen, and Rebecca Ferguson. 2016. Ethical and Privacy Issues in the Design of Learning Analytics Applications. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16). ACM, New York, NY, USA, 492--493.

Digital Library

[13]

Rebecca Ferguson, Tore Hoel, Maren Scheffel, and Hendrik Drachsler. 2016. Guest Editorial: Ethics and Privacy in Learning Analytics. Journal of Learning Analytics 3, 1 (2016), 5--15.

[14]

Andrea Forte, Niki Kittur, Vanessa Larco, Haiyi Zhu, Amy Bruckman, and Robert E Kraut. 2012. Coordination and beyond: social functions of groups in open content production. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. ACM, 417--426.

Digital Library

[15]

Batya Friedman, David G Hendry, and Alan Borning. 2017. A Survey of Value Sensitive Design Methods. Foundations and Trends® in Human--Computer Interaction 11, 2 (2017), 63--125.

Digital Library

[16]

Batya Friedman, Peter H Kahn, Alan Borning, and Alina Huldtgren. 2013. Value Sensitive Design and Information Systems. In Early engagement and new technologies: Opening up the laboratory, Neelke Doorn, Daan Schuurbiers, Ibo van de Poel, and Michael E Gorman (Eds.). Springer Netherlands, Dordrecht, 55--95.

[17]

R Stuart Geiger. 2017. Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture. Big Data & Society 4, 2 (Sept. 2017), 2053951717730735.

[18]

Christine Greenhow and Emilia Askari. 2017. Learning and teaching with social network sites: A decade of research in K-12 related education. Education and Information Technologies 22, 2 (March 2017), 623--645.

Digital Library

[19]

Iassen Halatchliyski, Johannes Moskaliuk, Joachim Kimmerle, and Ulrike Cress. 2014. Explaining authors' contribution to pivotal artifacts during mass collaboration in the Wikipedia's knowledge base. International Journal of Computer-Supported Collaborative Learning 9, 1 (2014), 97--115.

[20]

Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan, and John Riedl. 2013. The Rise and Decline of an Open Collaboration System: How Wikipedia's Reaction to Popularity Is Causing Its Decline. The American behavioral scientist 57, 5 (May 2013), 664--688.

[21]

Madeline Huberth, Patricia Chen, Jared Tritz, and Timothy A McKay. 2015. Computer-tailored student support in introductory physics. PloS one 10, 9 (Sept. 2015), e0137001.

[22]

Dirk Ifenthaler and Monica W Tracey. 2016. Exploring the relationship of ethics and privacy in learning analytics and design: implications for the field of educational technology. Educational technology research and development: ETR & D 64, 5 (Oct. 2016), 877--880.

[23]

Charles Lang, Leah P Macfadyen, Sharon Slade, Paul Prinsloo, and Niall Sclater. 2018. The Complexities of Developing a Personal Code of Ethics for Learning Analytics Practitioners: Implications for Institutions and the Field. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18). ACM, New York, NY, USA, 436--440.

Digital Library

[24]

P Long and G Siemens. 2011. Penetrating the Fog: Analytics in Learning and Education. Educause Review 46, 5 (2011), 30--32.

[25]

Paul Prinsloo and Sharon Slade. 2016. Student Vulnerability, Agency and Learning Analytics: An Exploration. Journal of Learning Analytics 3, 1 (2016), 159--182.

[26]

Paul Prinsloo and Sharon Slade. 2017. An Elephant in the Learning Analytics Room: The Obligation to Act. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK '17). ACM, New York, NY, USA, 46--55.

Digital Library

[27]

Paul Prinsloo and Sharon Slade. 2017. Ethics and Learning Analytics: Charting the (Un)Charted. In Handbook of Learning Analytics (first ed.), Charles Lang, George Siemens, Alyssa Wise, and Dragan Gasevic (Eds.). Society for Learning Analytics Research (SoLAR), 49--57.

[28]

Priscilla M Regan. 1995. Legislating Privacy: Technology, Social Values, and Public Policy. University of North Carolina Press, Chapel Hill, NC.

Digital Library

[29]

María Jesús Rodríguez-Triana, Alejandra Martínez-Monés, and Sara Villagrá-Sobrino. 2016. Learning Analytics in Small-Scale Teacher-Led Innovations: Ethical and Data Privacy Issues. Journal of Learning Analytics 3, 1 (2016), 43--65.

[30]

Carolyn Penstein Rosé, Oliver Ferschke, Gaurav Tomar, Diyi Yang, Iris Howley, Vincent Aleven, George Siemens, Matthew Crosslin, Dragan Gasevic, and Ryan Baker. 2015. Challenges and opportunities of dual-layer MOOCs: Reflections from an edX deployment study. In Proceedings of the 11th International Conference on Computer Supported Collaborative Learning. (CSCL 2015). Vol. 2, Vol. 15. International Society of the Learning Sciences, Gothenburg, Sweden, 848--851.

[31]

Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry, a pre-conference at the 64th Annual Meeting of the International Communication Association.

[32]

Niall Sclater. 2016. Developing a Code of Practice for Learning Analytics. Journal of Learning Analytics 3, 1 (2016), 16--42.

[33]

George Siemens. 2013. Learning analytics: The emergence of a discipline. The American behavioral scientist 57, 10 (2013), 1380--1400.

[34]

Sharon Slade and Paul Prinsloo. 2013. Learning analytics: Ethical issues and dilemmas. The American behavioral scientist 57, 10 (Oct. 2013), 1510--1529.

[35]

Dan Suthers and Katrien Verbert. 2013. Learning analytics as a "middle space". In Proceedings of the Third International Conference on Learning Analytics and Knowledge - LAK '13. ACM Press, New York, New York, USA, 1--4.

Digital Library

[36]

Luis M Vaquero and Manuel Cebrian. 2013. The rich club phenomenon in the classroom. Scientific reports 3 (2013), 1--8.

[37]

Josine Verhagen, Lucie Dalibert, Federica Lucivero, Tjerk Timan, and Larissa Co. 2016. Designing values in an adaptive learning platform. In 2nd workshop on Ethics & Privacy in Learning Analytic at the 6th International Learning Analytics & Knowledge Conference. CEUR Proceedings, Edinburgh, UK, 1--5.

[38]

Kari Edison Watkins, Brian Ferris, Yegor Malinovskiy, and Alan Borning. 2013. Beyond Context-Sensitive Solutions: Using Value-Sensitive Design to Identify Needed Transit Information Tools. In Urban Public Transportation Systems 2013. American Society of Civil Engineers, Reston, VA, 296--308.

[39]

Miaomiao Wen, Diyi Yang, and Carolyn Penstein Rosé. 2015. Virtual Teams in Massive Open Online Courses. In Artificial Intelligence in Education (Lecture Notes in Computer Science). Springer, Cham, 820--824.

[40]

James E Willis, Sharon Slade, and Paul Prinsloo. 2016. Ethical oversight of student data in learning analytics: A typology derived from a cross-continental, cross-institutional perspective. Educational Technology Research and Development 64, 5 (Oct. 2016), 881--901.

[41]

Alyssa Friend Wise and J Vytasek. 2017. Learning analytics implementation design. In Handbook of learning analytics, Charles Lang, George Siemens, Alyssa Friend Wise, and Dragan Gašević (Eds.). Society for Learning Analytics Research, 151--160.

[42]

Haiyi Zhu, Robert Kraut, and Aniket Kittur. 2012. Organizing without formal organization: group identification, goal setting and social modeling in directing online production. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. ACM, 935--944.

Digital Library

[43]

Haiyi Zhu, Bowen Yu, Aaron Halfaker, and Loren Terveen. 2018. Value-Sensitive Algorithm Design: Method, Case Study, and Lessons. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 194.

Digital Library

Cited By

Prieto LAlfredo RDíaz-Chavarría HMartinez-Maldonado REcheverría V(2025)VALA/AID: A Method for Rapid, Participatory Value-sensitive Learning Analytics and Artificial Intelligence DesignProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706553(950-956)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706553
Li FWise A(2025)From Filling Gaps to Amplifying Strengths: Exploring an Asset-Based Approach to Learning AnalyticsProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706542(924-930)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706542
Abramenka‐Lachheb VLachheb AOzogul G(2025)Value‐sensitive design in the praxis of instructional design: A view of designers in situBritish Journal of Educational Technology10.1111/bjet.13574Online publication date: 10-Feb-2025
https://doi.org/10.1111/bjet.13574
Show More Cited By

Index Terms

Towards Value-Sensitive Learning Analytics Design

Recommendations

The problem of defining values for design: a lack of common ground between industry and academia?
CHI EA '10: CHI '10 Extended Abstracts on Human Factors in Computing Systems

The HCI community recognizes the importance of value-centric design methodologies as reflected in the number of publications on the topic in recent years. However, the adoption of these methodologies by industry has been slower than desirable. This ...
VALA/AID: A Method for Rapid, Participatory Value-sensitive Learning Analytics and Artificial Intelligence Design
LAK '25: Proceedings of the 15th International Learning Analytics and Knowledge Conference
The adoption of learning analytics (LA) and artificial intelligence (AI) in education has long been a challenge, in part due to the ethical issues it engenders (e.g., the alignment of technology-embedded and human stakeholder values). Value-sensitive ...
Why value sensitive design needs ethical commitments
Abstract
Currently, value sensitive design (VSD) does not commit to a particular ethical theory. Critiques contend that without such an explicit commitment, VSD lacks a methodology for distinguishing genuine moral values from mere stakeholders-preferences ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge

March 2019

565 pages

ISBN:9781450362566

DOI:10.1145/3303772

General Chairs:
Sharon Hsiao
Arizona State University, USA
,
Jim Cunningham
Arizona State University, USA
,
Katie McCarthy
Georgia State University, USA
,
Grace Lynch
Society for Learning Analytics Research, Australia
,
Program Chairs:
Christopher Brooks
University of Michigan, USA
,
Rebecca Ferguson
The Open University, UK
,
Ulrich Hoppe
University of Duisburg-Essen, Germany

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
SoLAR: The Society for Learning Analytics Research
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

LAK19

LAK19: The 9th International Learning Analytics & Knowledge Conference

March 4 - 8, 2019

AZ, Tempe, USA

Acceptance Rates

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

39
Total Citations
View Citations
906
Total Downloads

Downloads (Last 12 months)85
Downloads (Last 6 weeks)10

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Prieto LAlfredo RDíaz-Chavarría HMartinez-Maldonado REcheverría V(2025)VALA/AID: A Method for Rapid, Participatory Value-sensitive Learning Analytics and Artificial Intelligence DesignProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706553(950-956)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706553
Li FWise A(2025)From Filling Gaps to Amplifying Strengths: Exploring an Asset-Based Approach to Learning AnalyticsProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706542(924-930)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706542
Abramenka‐Lachheb VLachheb AOzogul G(2025)Value‐sensitive design in the praxis of instructional design: A view of designers in situBritish Journal of Educational Technology10.1111/bjet.13574Online publication date: 10-Feb-2025
https://doi.org/10.1111/bjet.13574
Qudrat-Ullah H(2024)Exploring AI-Driven Pedagogical ToolsEmpowering Educational Leaders Using Analytics, AI, and Systems Thinking10.4018/979-8-3693-7190-9.ch010(275-322)Online publication date: 27-Dec-2024
https://doi.org/10.4018/979-8-3693-7190-9.ch010
Alfredo REcheverria VJin YSwiecki ZGašević DMartinez-Maldonado R(2024)SLADE: A Method for Designing Human-Centred Learning Analytics SystemsProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636847(24-34)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636847
Shankar SPothancheri GSasi DMishra S(2024)Bringing Teachers in the Loop: Exploring Perspectives on Integrating Generative AI in Technology-Enhanced LearningInternational Journal of Artificial Intelligence in Education10.1007/s40593-024-00428-835:1(155-180)Online publication date: 3-Sep-2024
https://doi.org/10.1007/s40593-024-00428-8
Yang KBorchers CFalhs AEcheverria VKarumbaiah SRummel NAleven V(2024)Leveraging Multimodal Classroom Data for Teacher Reflection: Teachers’ Preferences, Practices, and Privacy ConsiderationsTechnology Enhanced Learning for Inclusive and Equitable Quality Education10.1007/978-3-031-72315-5_34(498-511)Online publication date: 13-Sep-2024
https://doi.org/10.1007/978-3-031-72315-5_34
Shum S(2024)What Could Learning Analytics Learn from Human-Computer Interaction Theory?Theory Informing and Arising from Learning Analytics10.1007/978-3-031-60571-0_10(153-172)Online publication date: 29-Dec-2024
https://doi.org/10.1007/978-3-031-60571-0_10
Lundström NÖberg LViberg O(2024)Teachers’ Expectations of Learning Analytics from a Value-Based PerspectiveOpen Science in Engineering10.1007/978-3-031-42467-0_77(811-822)Online publication date: 1-Jan-2024
https://doi.org/10.1007/978-3-031-42467-0_77
Campos FNguyen HAhn JJackson K(2023)Leveraging cultural forms in human‐centred learning analytics designBritish Journal of Educational Technology10.1111/bjet.1338455:3(769-784)Online publication date: 10-Sep-2023
https://doi.org/10.1111/bjet.13384
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten