Skip to main content
SpringerLink
Log in

Learning behaviour recognition based on multi-object image in single viewpoint

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Though greatly significant for education evaluation and improvement the analysis of learning behaviour of students is, recognition of behaviour of students in the classroom surveillance based on one single image remains a challenging work due to problems including heavy occlusion, low resolution, small target size, large variability of camera viewpoints and significant perspective effect. In this paper, an innovative multiple-model method for recognizing students’ learning behaviour and counting the number of students in classroom scenes via detecting individuals’ head, which divides students’ learning behaviour into three categories according to the head posture, is proposed. Through viewing heads at different postures as targets of different categories, behaviour recognition problem is transformed into multi-target detection problem. As the density of students in different images varies dramatically, a multiple model consisting of three detection networks with different sized receptive fields and one switch net is applied that each detection network selects a large number of sub-windows with small stride size from input images to optimize the performance of the model in classroom scenes while the switch net is trained to predict the density of students in the input image and relays the classroom image to the corresponding detection network according to the students’ density prediction. This method is obtained to outperform other methods in terms of accuracy and speed in comparison with state-of-the-art methods on a real classroom surveillance dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ko KE, Sim KB (2018) . Eng Appl Artif Intell 67:226. https://doi.org/10.1016/j.engappai.2017.10.001. http://www.sciencedirect.com/science/article/pii/S0952197617302579

    Article  Google Scholar 

  2. Zheng Y, Yao H, Sun X, Zhao S, Porikli F (2017) Signal processing

  3. Sefidgar YS, Vahdat A, Se S, Mori G (2015) . Comput Vis Image Underst 135(4):16

    Article  Google Scholar 

  4. Dalal N, Triggs B (2005) In: 2005. CVPR 2005. IEEE computer society conference on computer vision and pattern recognition, pp 886–893

  5. Alonso IP, Llorca DF, Sotelo M, Bergasa LM, De Toro PR, Nuevo J, Ocaña M (2007) . IEEE Trans Intell Transp Syst 8(2):292

    Article  Google Scholar 

  6. Viola P, Jones MJ, Snow D (2005) . Int J Comput Vis 63(2):153

    Article  Google Scholar 

  7. Hotta K (2002) In: IEEE workshop on applications of computer vision, p 105

  8. Ye Q, Dong J, Zhang Y (2015) . Optik - International Journal for Light and Electron Optics 126 (23):4712

    Article  Google Scholar 

  9. Patrona F, Chatzitofis A, Zarpalas D, Daras P (2018) ., vol 76. https://doi.org/10.1016/j.patcog.2017.12.007. http://www.sciencedirect.com/science/article/pii/S0031320317304910

  10. Meshry M, Hussein ME, Torki M (2016) In: Applications of computer vision, pp 1–9

  11. Amor BB, Su J, Srivastava A (2015) . IEEE Trans Pattern Anal Mach Intell 38(1):1

    Article  Google Scholar 

  12. Tang GJ, Peng T (2012) In: International congress on image and signal processing, pp 267–270

  13. Dollar P, Wojek C, Schiele B, Perona P (2009) Proc Cvpr, pp 304–311

  14. Huang J, Su K, El-Den J, Hu T, Li J (2014) . Math Probl Eng, 2014,(2014-8-31) 2014(1):1

    Google Scholar 

  15. Pham MT, Chain TJ (2007) In: IEEE international conference on computer vision, pp 1–7

  16. Geismann P, Schneider G (2008) In: Intelligent vehicles symposium, pp 554–559

  17. Schuldt C, Laptev I, Caputo B (2004) In: International conference on pattern recognition, Vol 3, pp 32–36

  18. Felzenszwalb PF, Girshick RB, Mcallester D (2013) . Commun ACM 56(9):97

    Article  Google Scholar 

  19. Girshick R, Donahue J, Darrell T, Malik J (2014) In: 2014 IEEE conference on computer vision and pattern recognition (CVPR), vol 00, pp 580–587. https://doi.org/10.1109/CVPR.2014.81

  20. Uijlings JR, Sande KE, Gevers T, Smeulders AW (2013) . Int J Comput Vis 104(2):154

    Article  Google Scholar 

  21. He K, Zhang X, Ren S, Sun J (2014) . IEEE Trans Pattern Anal Mach Intell 37(9):1904

    Article  Google Scholar 

  22. Girshick R (2015) Computer science

  23. Ren S, He K, Girshick R, Sun J (2015) In: International conference on neural information processing systems, pp. 91–99

  24. Dai J, Li Y, He K, Sun J (2016)

  25. Redmon J, Divvala S, Girshick R, Farhadi A (2015) pp 779–788

  26. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2015) pp 21–37

  27. Sam DB, Surya S, Babu RV (2017) In: IEEE conference on computer vision and pattern recognition, pp 4031–4039

  28. He K, Zhang X, Ren S, Sun J (2016) In: Computer vision and pattern recognition, pp 770–778

  29. Su K, Chen W, Lei N, Zhang J, Qian K, Gu X (2017) . Comput Aided Des 82:42. https://doi.org/10.1016/j.cad.2016.05.020. http://www.sciencedirect.com/science/article/pii/S0010448516300495. Isogeometric Design and Analysis

    Article  MathSciNet  Google Scholar 

  30. Su K, Cui L, Qian K, Lei N, Zhang J, Zhang M, Gu XD (2016) . Comput Aided Geom Des 46:76. https://doi.org/10.1016/j.cagd.2016.05.005. http://www.sciencedirect.com/science/article/pii/S0167839616300528

    Article  Google Scholar 

  31. Su K, Chen W, Lei N, Cui L, Jiang J, Gu XD (2016) . Comput Aided Des 78:188. https://doi.org/10.1016/j.cad.2016.04.007. http://www.sciencedirect.com/science/article/pii/S0010448516300197. SPM 2016

    Article  Google Scholar 

Download references

Acknowledgements

We thank the editor and all the anonymous reviewers for their valuable advice and suggestions to improve the current work.

Funding

This work is financially supported by the National Key Research and Development Program of China (No. 2016YFB052204), the National Natural Science Foundation of China (No.61772379) and the Independent Scientific Research Project of Wuhan University (No. 2042018kf0246).

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, China

    Kehua Su, Xiang Li, Chengcheng Zhou & Xu Chen

Authors
  1. Kehua Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chengcheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xu Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, K., Li, X., Zhou, C. et al. Learning behaviour recognition based on multi-object image in single viewpoint. Pers Ubiquit Comput 25, 1081–1090 (2021). https://doi.org/10.1007/s00779-019-01286-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-019-01286-1

Keywords

Search

Navigation