Elsevier

Computers & Education

Volume 69, November 2013, Pages 237-249
Computers & Education

Constructing and evaluating online goal-setting mechanisms in web-based portfolio assessment system for facilitating self-regulated learning

https://doi.org/10.1016/j.compedu.2013.07.016Get rights and content

Highlights

  • Design goal-setting mechanisms in WBPAS and examine effects of these mechanisms on SRL.

  • The participants were 11th graders taking the website design class in a vocational high school.

  • The online goal-setting mechanisms significantly enhanced SRL.

  • The online goal setting with the WBPAS had a significantly positive effect on SRL.

Abstract

The purpose of the present study was to construct goal-setting mechanisms in a web-based portfolio assessment system (WBPAS), based on the self-regulated learning (SRL) process proposed by Zimmerman, and to examine effects of these mechanisms on SRL. The participants were two classes of 11th graders taking the website design class in a vocational high school. The participants were assigned randomly to either an experimental group (n = 40) learning with a WBPAS or a control group (n = 41) learning with a paper-based portfolio. The study results revealed the following: a) the quality of goal-setting mechanisms may facilitate SRL. b) Students setting learning goals with the WBPAS demonstrated significantly better SRL than students setting learning goals with the paper-based portfolio.

Introduction

A web-based portfolio assessment (WBPA) aims at collecting and presenting students' learning experiences, thoughts, work, progresses and self-reflections for assessment on their learning (Lougheed, Bogyo, Brokenshire, & Kumar, 2005). A web-based portfolio assessment system (WBPAS) usually includes goal setting, work in different stages, reflection, teacher assessment and feedback, peer assessment and feedback, students' self-assessment and feedback, and progress data (Chang, Tseng, Yueh, & Lin, 2011). The learning goals are one of the most important items in a portfolio. Based on above, the mechanisms for goal setting may at least contain setting goals, viewing the progress and revising the goals. Additionally, students are able to browse others' portfolios, to give feedback, to self-regulate, and to review their own progress by the system. Teachers are able to do the assessment for students according to the effort a student put during his learning, the progress, and the learning goals a student achieves.

Goal setting is one of the essential activities in the procedure of web-based portfolio assessment (WBPA) (Chang et al., 2011). In order to achieve self-set goals, students regulate their own learning as time goes by. Accordingly, goal-setting facilitates self-regulated learning (SRL) and is an important factor that affects SRL. Some studies confirmed that SRL is facilitated by goal setting (Latham & Locke, 1991; Zimmerman, 2008). Actually, the portfolio itself shares features with SRL. For example, during the development of portfolios, students improve themselves through reflections which are often performed based on self-set goals. Portfolios guide students during a learning process, and continuous self-regulation is performed based on self-set goals (Heo, 2000). Abrami et al. (2008), Carneiro, Lefrere, Steffens, and Underwood (2011) argued that an e-portfolio is helpful to SRL. Consequently, goal-setting mechanisms are obviously crucial. In addition, Riedinger (2004) pointed out advantages of a web-based portfolio, including that: a) it is not restricted by time and space; b) it is convenient to be browsed by peers; c) it is easy for peers to share with one another and to give feedback; and d) it is convenient for peers to view others' learning goals. Therefore, how online goal-setting mechanisms can be implemented with the advantages of the Internet to enhance students' SRL is an important issue.

Since goal setting facilitates SRL, goal-setting mechanisms in a web-based portfolio assessment system (WBPAS) are apparently crucial. There is a lack of relevant studies about goal-setting mechanisms that facilitate students' SRL. However, recent studies about formative assessment systems or SRL systems can be the reference for researchers. Wang (2011) enhanced students' SRL and learning performance by review and feedback mechanisms in a formative assessment system. Arsal (2010) adopted a diary as a tool of SRL for pre-service science teachers. The attributes of the diary are similar to the portfolio. Both the diary and the portfolio allow students to review their learning progress. Actually, the diary is appropriate for keeping track of students' development of SRL (Neber & Schommer-Aikins, 2002; Schmitz & Wiese, 2006; Winne, 2005). A WBPAS can facilitate SRL more because it features the attributes of diary and formative assessment. Hwang et al. (2007) enhanced students' learning performance through SRL mechanisms in an online SRL system and online learning activities. Their study results revealed that SRL process was helpful to their learning. Hence, the system did facilitate students' SRL and learning performance.

There is some software for goal setting, but most software is for task management not for students' learning. For example, Lifetick is an online system for goal setting and achievement. The system helps users set goals online and reminds users to attain their goals. Users are able to track their own task progress and determine if they achieved the preset goals. The features of the system include defining values, setting task, setting goals, creating tasks, staying motivated, tracking progress, measuring performance, and journal thoughts (Lifetick, 2011). Although most of these systems are for task management, they can still be the references for the development of online goal-setting mechanisms in the present study.

However, are the online goal-setting mechanisms mentioned above appropriate to be implemented in a WBPAS? Are they enough for a WBPAS? What goal-setting mechanisms should be included in a WBPAS? Are these goal-setting mechanisms helpful to students' SRL? Do students learning with goal-setting mechanisms in a WBPAS perform better than students learning with paper-based portfolio? What aspects of SRL can be enhanced? As Azevedo (2005) and Kollar and Fischer (2006) argued that digital environment is beneficial to SRL. According to the background above, the purpose of the present study was to construct online goal-setting mechanisms in a WBPAS and to examine its effects on SRL. The research questions are the following:

  • 1)

    What is the level of student satisfaction about the online goal-setting mechanisms in the WBPAS?

  • 2)

    Are there any significant differences on SRL between students who are highly satisfied with the goal-setting mechanisms in the WBPAS and students who are not highly satisfied?

  • 3)

    Are there any significant differences on SRL between students learning with the WBPAS and students learning with paper-based portfolio?

Section snippets

Participants

Participants in the present study were two classes of 11th graders taking a “Digital Data Processing” class at a vocational high school in USA. The participants were randomly assigned to either an experimental group learning with a WBPAS (40 participants) or a control group learning with paper-based portfolio (41 participants). There were a total of 81 participants, with 36 males and 45 females. The content of the course was primarily webpage design. It was a hands-on computer course which

Students' satisfaction toward online goal-setting mechanisms

According to Table 5, for the aspect of ease of use toward goal-setting mechanisms, the greatest percentage of students marked was on rank 4.0 for writing (item 1), editing (item 2), feedback (item 4) and assessment (item 5). For the aspect of usefulness, the greatest percentage of students marked was on rank 4.0 for writing (item 6), editing (item 7) and assessment (item 10), and the percentage of students was al higher than 40%. Besides, the scores that the greatest percentage of students

Discussions

According to the study results, students who were highly satisfied with goal-setting mechanisms had a significantly better performance in overall SRL than students who were less satisfied. This revealed that the better the online goal-setting mechanisms in a WBPAS, the better the SRL. This also implied that the online goal-setting mechanisms had a significantly positive effect on SRL. Besides, students using WBPAS to set goals outperformed students using paper-based portfolio to set goals in

Conclusion and implication

Regarding the revision of the WBPAS, the feedback mechanism for goal setting ranked the lowest in terms of student satisfaction, so further improvement was required. Besides, auto assessment, statistics of feedback and peer assessment mechanisms could be enhanced for a better goal-setting mechanism. Among the goal-setting mechanisms in the system, the peer model mechanism ranked the highest in terms of student satisfaction, revealing that students believed that peer model was beneficial to

References (41)

  • N. Dabbagh et al.

    Personal learning environments, social media, and self-regulated learning: a natural formula for connecting formal and informal learning

    Internet and Higher Education

    (2011)
  • J.K. Ford et al.

    The application of exploratory factor analysis in applied psychology: a critical review and analysis

    Personnel Psychology

    (1986)
  • F.J. Gravetter et al.

    Statistics for behavioral science

    (2008)
  • J.A. Greene et al.

    Adolescents' use of self-regulatory processes and their relation to qualitative mental model shifts while using hypermedia

    Journal of Educational Computing Research

    (2007)
  • J.F. Hair et al.

    Multivariate data analysis: A global perspective

    (2010)
  • H.H. Harman

    Modern factor analysis

    (1976)
  • H. Heo

    Theoretical underpinnings for structuring the classroom as self-regulated learning environment

    Educational Technology International

    (2000)
  • G.J. Hwang et al.

    The development of a computer-assisted self-regulation system on the internet

    Chinese Journal of Science Education

    (2007)
  • H.F. Kaiser

    Little Jiffy, Mark IV

    Educational and Psychological Measurement

    (1974)
  • T.L. Kelley

    The selection of upper and lower groups for the validation of test items

    Journal of Educational Psychology

    (1939)
  • Cited by (27)

    • How does the pedagogical design of a technology-enhanced collaborative academic course promote digital literacies, self-regulation, and perceived learning of students?

      2020, Internet and Higher Education
      Citation Excerpt :

      One way to develop and support self-regulation skills is through communication in a digital learning community (Lin, Lai, & Chang, 2016), as well as through computational tools themselves and social interactions in collaborative learning (for review see Järvelä et al., 2016). A study that examined the impact of peer support on the development of self-regulation skills in digital environments (Chang, Tseng, Liang, & Liao, 2013) demonstrated that community learning helps students develop learning proficiency by increasing their motivation to cope with the task and by providing peer feedback. In addition, exposure to learning strategies and to the learning outcomes of others raises awareness to ideas of others and enables learners to adopt new effective strategies, as well as improve quality standards for their own learning outcomes (Kitsantas, 2013).

    • Decision-making determinants of students participating in MOOCs: Merging the theory of planned behavior and self-regulated learning model

      2019, Computers and Education
      Citation Excerpt :

      SRL not only has been positively associated with academic outcomes in formal, offline learning contexts, but also a number of studies have investigated the role that SRL performs well in the online learning context, and online learning with social media to attain better study effects (Cho & Heron, 2015). In an experimental study, Chang, Tseng, Liang, and Liao (2013) found that students setting learning goals with the web-based portfolio assessment system demonstrated significantly better self-regulation in learning than students setting learning goals with the paper-based portfolio. A recent study investigating learners with high and low levels of SRL in a MOOC identified five sub-processes that were associated with higher achievement, including elaboration, organization, critical thinking, metacognitive self-regulation, and peer learning. (

    • Comparative effect of online summative and formative assessment on EFL student writing ability

      2018, Studies in Educational Evaluation
      Citation Excerpt :

      The findings are in line with many other studies when it comes to electronic portfolio assessment. Chang, Tseng, Liang, and Liao (2013)'s investigation of web-based portfolio assessment with the purpose of promoting self-regulation indicated that the technology enhanced learning and interactive digital learning environments are helpful in students' regulating their own learning and increasing learning quality. The findings of this study are also supported by other studies in literature as far as achievements through online collaboration are concerned.

    • The impact of an online project-based learning environment with group awareness support on students with different self-regulation levels: An extended-period experiment

      2016, Computers and Education
      Citation Excerpt :

      Possible explanations for the above phenomena are as follows: The GA provides an external scaffold (peer stimulus) that motivates learners to persist in training tasks and facilitating a learners' reflection of their learning status (Chang, Tseng, Liang, & Liao, 2013; Lin et al., 2015). Notably, the learning and adapting learning behavior of high-SR students is intrinsically more persistent compared with low-SR students (Newman, 2002; Pintrich, 2004).

    • Context counts: How learners' contexts influence learning in a MOOC

      2015, Computers and Education
      Citation Excerpt :

      Cheng and Chau (2013), investigating the role of self-regulated learning in e-portfolios, identified five sub-processes that were associated with higher achievement – elaboration, organisation, critical thinking, metacognitive self-regulation and peer learning. In an experimental study Chang, Tseng, Liang, and Liao (2013) found that students in an online learning group demonstrated significantly higher self-regulation and more positive attitude toward learning motivation, self-efficacy and subject values than students in the offline learning control group. There have been few studies to date examining the role that SRL plays in influencing MOOC participation.

    View all citing articles on Scopus
    View full text