Skip to main content
SpringerLink
Log in

VisOJ: real-time visual learning analytics dashboard for online programming judge

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Online Judge (OJ) is an important aid for programming learning that can help students evaluate learning effects in real-time, while teachers can adjust practice tasks in time according to the records of the tool. With these advantages, OJ shows great value for promoting teaching and learning in programming. The existing OJ system usually only provides information such as problem status list and recent rank list. However, it is unable to provide teachers with more fine-grained analysis information, such as the distribution of students’ incorrect responses and level of knowledge mastery. And it also cannot provide students with effective comparative information on their learning status. This research developed a visual learning analytics dashboard named VisOJ for the OJ system, which includes two types of user interfaces: teacher and student. The teacher interface presents students' learning status and ranking trends, which help teachers monitor and give feedback on their learning activities. The student interface provides views such as error type analysis and evaluation, which promote students' self-reflection and self-regulation. Preliminary case studies and expert interviews prove the usability of the dashboard. In the end, we summarize our main work and suggest future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Baker, M., Hu, X., Luca, G.D., Chen, Y.: Intelligent voice instructor-assistant system for collaborative and interactive classes. J. Artif. Intell. Technol. (2021). https://doi.org/10.37965/jait.2021.0003

    Article  Google Scholar 

  2. Wang, M., Han, W., Chen, W.: MetaOJ: a massive distributed online judge system. Tsinghua Sci Technol 26(4), 548–557 (2021). https://doi.org/10.26599/TST.2020.9010016

    Article  Google Scholar 

  3. USACO: USACO. http://usaco.org/. Accessed 2021-6-7 2021

  4. UVA: UVA. https://uva.onlinejudge.org/. Accessed 2021-6-10 2021

  5. Zhao, W., Zhang, W., He, Y., Xie, X., Wen, J.: Automatically learning topics and difficulty levels of problems in online judge systems. ACM T Inform Syst 36(3), 1–33 (2018). https://doi.org/10.1145/3158670

    Article  Google Scholar 

  6. Wu, H., Liu, Y., Lin, Q., Yi, L.: Online judge system and its applications in c language teaching. Paper presented at the 2016 International Symposium on Educational Technology (ISET)

  7. Tica, I., Olteanu, A., Racec, E.: Supporting multiple programming languages in an online judge. Smart Innov. Syst. Technol. 197, 245–252 (2020)

    Article  Google Scholar 

  8. Lee, L., Cheung, S.K.S., Kwok, L.: Learning analytics: current trends and innovative practices. J. Comput. Educ. 7(1), 1–6 (2020). https://doi.org/10.1007/s40692-020-00155-8

    Article  Google Scholar 

  9. Hillaire, G.E., Schlichtmann, G., Ducharme, K.: Prototyping visual learning analytics guided by an educational theory informed goal. J. Learn. Anal. 3(3), 115–142 (2016). https://doi.org/10.18608/jla.2016.33.7

    Article  Google Scholar 

  10. Ji, L., Yuan, Y., Gao, F.: Multi-level and multi-perspective visual correlation analysis between general courses and program courses. Vis. Comput. 37(3), 477–495 (2020). https://doi.org/10.1007/s00371-020-01818-4

    Article  Google Scholar 

  11. Wang, R., Dong, H., Han, T.X., Mei, L.: Robust tracking via monocular active vision for an intelligent teaching system. Vis. Comput. 32(11), 1379–1394 (2016). https://doi.org/10.1007/s00371-015-1206-8

    Article  Google Scholar 

  12. Zhu, X., Chen, Z.: Dual-modality spatiotemporal feature learning for spontaneous facial expression recognition in e-learning using hybrid deep neural network. Vis. Comput. 36(4), 743–755 (2019). https://doi.org/10.1007/s00371-019-01660-3

    Article  Google Scholar 

  13. Vieira, C., Parsons, P., Byrd, V.: Visual learning analytics of educational data: a systematic literature review and research agenda. Comput. Educ., 122(JUL.), 119–135 (2018)

  14. Jivet, I., Scheffel, M., Specht, M., Drachsler, H.: License to evaluate: preparing learning analytics dashboards for educational practice. International Conference, 31–40 (2018)

  15. Verbert, K., Ochoa, X., Croon, R.D., Dourado, R.A., Laet, T.D.: Learning analytics dashboards: the past, the present and the future. LAK '20: 10th International Conference on Learning Analytics and Knowledge, 35–40 (2020). https://doi.org/10.1145/3375462.3375504

  16. Matcha, W., Uzir, N.A., Gasevic, D., Pardo, A.: A systematic review of empirical studies on learning analytics dashboards: a self-regulated learning perspective. IEEE T Learn Technol 13(2), 226–245 (2020). https://doi.org/10.1109/TLT.2019.2916802

    Article  Google Scholar 

  17. Chen, Q., Yue, X., Plantaz, X., Chen, Y., Shi, C., Pong, T., Qu, H.: Viseq: visual analytics of learning sequence in massive open online courses. IEEE T Vis Comput GR 26(3), 1622–1636 (2020). https://doi.org/10.1109/tvcg.2018.2872961

    Article  Google Scholar 

  18. He, H., Zheng, O., Dong, B.: VUSphere: Visual analysis of video utilization in online distance education. 2018 IEEE Conference on Visual Analytics Science and Technology, 25–35 (2018)

  19. Guerra, J.S.E.H.: TrAC: Visualizing students academic trajectories. Transforming Learning with Meaningful Technologies, 765–768 (2019). https://doi.org/10.1007/978-3-030-29736-7_84

  20. Charleer, S., Moere, A.V., Klerkx, J., Verbert, K., De Laet, T.: Learning analytics dashboards to support adviser-student dialogue. IEEE T Learn Technol 11(3), 389–399 (2018). https://doi.org/10.1109/TLT.2017.2720670

    Article  Google Scholar 

  21. Krumm, A.E., R.J.W.S.: A learning management system-based early warning system for academic advising in undergraduate engineering. Learn. Anal., 103–119 (2014). doi: https://doi.org/10.1007/978-1-4614-3305-7_6

  22. Jean Luca Bez, N.A.T.P.: URI online judge academic: a tool for algorithms and programming classes. 2014 9th International Conference on Computer Science and Education, (2014)

  23. Michele Selivon, J.L.B.N.: URI online judge academic: integrac¸˜ao e consolidac¸ ˜ ao da ferramenta no processo de ensino/aprendizagem. Paper presented at the 23º Wei-workshop Sobre Educação Em Computação Csbc

  24. Statistical Charts: Statistical Charts. http://poj.org/chart. Accessed 2021-6-10 (2021)

  25. URAL: URAL. http://acm.timus.ru/ (2021)

  26. SPOJ: SPOJ. http://www.spoj.pl/ (2021)

  27. Xu, B., Yan, S., Jiang, X., Feng, S.: SCFH: a student analysis model to identify students’ programming levels in online judge systems. Symmetry 12(4), 601 (2020). https://doi.org/10.3390/sym12040601

    Article  Google Scholar 

  28. Lu, X., Zheng, D., Liu, L.: Data driven analysis on the effect of online judge system. 2017 IEEE International Conference on Internet of Things, 573–577 (2017)

  29. Xia, M., Xu, M., Lin, Chuan-en, Cheng, T. Y., Qu, H., Ma, X.: SeqDynamics: visual analytics for evaluating online problem-solving dynamics. Eurograph. Conf. Vis., 39(3) (2020)

  30. Emmons, S.R., Light, R.P., Brner, K.: Mooc visual analytics: empowering students, teachers, researchers, and platform developers of massively open online courses. J. Am. Soc. Inf. Sci. 68(10), 2350–2363 (2017). https://doi.org/10.1002/asi.23852

    Article  Google Scholar 

  31. Law, C., Grundy, J., Cain, A., Vasa, R., Cummaudo, A.: User perceptions of using an open learner model visualization tool for facilitating self-regulated learning. Proceedings of the Nineteenth Australasian Computing Education Conference, 55–64 (2017)

  32. Meng Xia, M.S.H.W.: PeerLens: peer-inspired interactive learning path planning in online question pool. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–12 (2019). https://doi.org/10.1145/3290605.3300864

  33. Charleer, S., Klerkx, J., Duval, E., De Laet, T., Verbert, K.: Creating effective learning analytics dashboards: lessons learnt. European Conference on Technology Enhanced Learning, 42–56 (2016)

  34. Wu, T., Yao, Y., Duan, Y., Fan, X., Qu, H.: NetworkSeer: visual analysis for social network in moocs. Paper presented at the 2016 IEEE Pacific Visualization Symposium (PacificVis), Taipei, Taiwan, (2016)

  35. Echart: Echart. https://echarts.apache.org/zh/index.html (2021)

  36. Koko, M., Altun, A.: Effects of learner interaction with learning dashboards on academic performance in an e-learning environment. Behav. Inf. Technol., 1–15 (2019)

  37. Brooke, J.: Sus: A quick and dirty usability scale, pp. 189–194. Usability Evaluation in Industry (1996)

  38. Ghosh, S., Shivakumara, P., Roy, P., Pal, U., Lu, T.: Graphology based handwritten character analysis for human behaviour identification. CAAI Trans. Intell. Technol. 5(1), 55–65 (2020). https://doi.org/10.1049/trit.2019.0051

    Article  Google Scholar 

Download references

Acknowledgements

This research was funded by the National Natural Science Foundation of China (NSFC: 61907011, 62077005).

Author information

Authors and Affiliations

  1. School of Educational Technology, Faculty of Education, Beijing Normal University, Beijing, China

    Qian Fu

  2. School of Mathematics and Information Science, Henan University of Economics and Law, Zhengzhou, China

    Xue Bai

  3. Institute of Advanced Studies in Humanities and Social Sciences, Beijing Normal University at Zhuhai, Zhuhai, China

    Yafeng Zheng

  4. School of Computer and Information Engineering, Henan University of Economics and Law, Zhengzhou, China

    Runsheng Du & Dongqing Wang

  5. School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA

    Tianyi Zhang

Authors
  1. Qian Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xue Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yafeng Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Runsheng Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dongqing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tianyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yafeng Zheng.

Ethics declarations

Conflict of interest

Qian Fu declares that he has no conflict of interest. Xue Bai declares that she has no conflict of interest. Yafeng Zheng declares that she has no conflict of interest. Runsheng Du declares that he has no conflict of interest. Dongqing Wang declares that he has no conflict of interest. Tianyi Zhang declares that she has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, Q., Bai, X., Zheng, Y. et al. VisOJ: real-time visual learning analytics dashboard for online programming judge. Vis Comput 39, 2393–2405 (2023). https://doi.org/10.1007/s00371-022-02586-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02586-z

Keywords

Search

Navigation