Skip to main content
SpringerLink
Log in

An improved method of identifying learner's behaviors based on deep learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Nowadays, the evaluation of students’ learning effect in colleges and universities mainly rely on manual management and supervision, which is inefficient and inaccurate. What’s worse, it completely depends on the subjective judgment of the supervisors, many potential teaching quality data could be ignored. Using an unsupervised classroom monitoring system based on deep learning can master the learner’s learning result. This paper mainly researched a learner’s behavior evaluation method based on deep learning. Firstly, MTCNN is being used to detected learner’s facial feature in order to confirm their identification. Then, collected the data set in learning environment and a Mosaic data enhancement had been done. Later, an improved method, is proposed. Identifying the students’ actions of playing the mobile phone, moving to classroom, eating, reading, writing, this method has more advantages on classroom behavior detecting. In addition, by building a quantitative evaluation method–CFIndex (class focus index), the time of students' behaviors in classroom has been calculated. Therefore, CFIndex evaluation method can well reflect the real performance of students in the classroom. In this paper, compared with Fast R-CNN and classical algorithm on the same data set, the proposed method has better performance in classroom behavior detection, and it can better reflect the students’ real actions in the classroom, it has certain meaning of theoretical guidance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cao D et al (2021) Recognition of students’s behavior states in classroom based on improved mobile net V2 algorithm. Int J Electric Eng Edu. https://doi.org/10.1177/0020720921996595

    Article  Google Scholar 

  2. Ranjan R, Patel VM, Hyperface CR (2017) A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans Pattern Anal Mach Intell 41:121–135

    Article  Google Scholar 

  3. Zhang K, Zhang Z, Li Z et al (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23:1499–1503

    Article  Google Scholar 

  4. Liu Y, BingHang L, Peng J, Zhang Z (2021) Research on the use of YOLOv5 object detection algorithm in mask wearing recognition. World Sci Res J 11:276–284

    Google Scholar 

  5. Wu W, Liu H, Li L, Long Y, Wang X, Wang Z et al (2021) Application of local fully convolutional neural network combined with YOLO v5 algorithm in small target detection of remote sensing image. PLoS ONE 16(10):e0259283. https://doi.org/10.1371/journal.pone.0259283

    Article  Google Scholar 

  6. Sandha KS, Thakur A (2019) Comparative analysis of mixed CNTs and MWCNTs as VLSI interconnects for deep sub-micron technology nodes. J Electron Mater 48(4):2543–2554

    Article  Google Scholar 

  7. Chunming Wu, Zhang Y (2021) MTCNN and FACENET based access control system for face detection and recognition. Autom Control Comput Sci 55:231–235

    Google Scholar 

  8. Hongchang Ku, Dong W (2020) Face recognition based on mtcnn and convolutional neural network. Front Sig Process 4:232

    Google Scholar 

  9. Wang Y, Yuan G, Zheng D, Hao Wu, Yuanyuan Pu, Dan Xu (2019) Research on face detection method based on improved MTCNN network. Int Conf Dig Image Process 12:262

    Google Scholar 

  10. Akash AA, Akhand MAH, Siddique N (2021) Robust face detection integrating novel skin color matching under variant illumination conditions. Int J Image, Graph Signal Process (IJIGSP) 2:13

    Google Scholar 

  11. Shifan T, Yufeng Li, Yufeng H, Xiaoyu L (2020) Face detection algorithm based on deep residual network. J Phys: Conf Ser 3:1802

    Google Scholar 

  12. Deshmukh SP, Patwardhan MS, Mahajan AR (2018) Feedback based real time facial and head gesture recognition for e-learning system. Data Sci Manag Data 5:33–38

    Google Scholar 

  13. Lu XX (2018) A review of solutions for perspective-n-point problem in camera pose estimation. J Phys: Conf Ser 5:1087

    Google Scholar 

  14. Kubíčková H, Jedlička K, Fiala R, Beran D (2020) Indoor positioning using pnp problem on mobile phone images. ISPRS Int J Geo Inf 9:45–49

    Article  Google Scholar 

  15. Junting D, Qi F et al (2021) Fast and robust multi-person 3d pose estimation and tracking from multiple views. IEEE Trans Pattern Anal Mach Intell 585:1248

    Google Scholar 

  16. Deng Q, Zhili Wu (2018) Students’ attention assessment in elearning based on machine learning. IOP Conf Ser: Earth Environ Sci 3:199

    Google Scholar 

  17. Alejandra VRC, Raúl PE (2017) Automatic assessment of engagement and attention of the student by means of facial expressions. Front Artif Intell Appl 23:295–297

    Google Scholar 

  18. Weisler S (2015) Some perspectives on assessment of student learning. J Assess Inst Eff 5:234–235

    Google Scholar 

  19. Peng G, Lei Y, Li H, Wu D, Wang J, Liu F (2021) CORY-net: contrastive Res-YOLOv5 network for intelligent safety monitoring on power grid construction sites. IEEE Access 9:160461–160470. https://doi.org/10.1109/ACCESS.2021.3132301

    Article  Google Scholar 

  20. Huang G, Mengwei Xu, Lin FX, Liu Y, Ma Y, Pushp S, Liu X (2017) ShuffleDog: characterizing and adapting user-perceived latency of android apps. IEEE Trans Mob Comput 16(10):2913–2926

    Article  Google Scholar 

  21. Chen X, Lin J, Ma Y, Lin B, Wang H, Huang G (2019) Self-adaptive resource allocation for cloud-based software services based on progressive QoS prediction model. SCIENCE CHINA Information Sciences 62(11):219101

    Article  Google Scholar 

  22. Chen X, Wang H, Ma Y, Zheng X, Guo L (2020) Self-adaptive resource allocation for cloud-based software services based on iterative QoS prediction model. Futur Gener Comput Syst 105:287–296

    Article  Google Scholar 

  23. Chen X, Li A, Zeng X, Guo W, Huang G (2015) Runtime model based approach to IoT application development. Front Comput Sci 9(4):540–553

    Article  Google Scholar 

  24. Huang G, Ma Y, Liu X, Luo Y, Xuan Lu, Brian Blake M (2015) Model-based automated navigation and composition of complex service mashups. IEEE Trans Serv Comput 8(3):494–506

    Article  Google Scholar 

Download references

Acknowledgements

This project is supported by [University Distinguishing Innovation Project of Guangdong Provincial Department of education in 2021 (2021KTSCX149), Key Project of Science and Technology of Dongguan Social Development in 2021 (20211800905512)].

Author information

Authors and Affiliations

  1. School of Computer, Guangdong University of Science and Technology, Dongguan, 523083, China

    Suping Liu & Wenlong Su

  2. Software Engineering Institute of Guangzhou, Guangzhou, 510990, China

    Jibo Zhang

Authors
  1. Suping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jibo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenlong Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jibo Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Zhang, J. & Su, W. An improved method of identifying learner's behaviors based on deep learning. J Supercomput 78, 12861–12872 (2022). https://doi.org/10.1007/s11227-022-04402-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04402-w

Keywords

Search

Navigation