Skip to main content

Advertisement

Springer Nature Link
Log in

1D Multi-Point Local Ternary Pattern: A Novel Feature Extraction Method for Analyzing Cognitive Engagement of students in Flipped Learning Pedagogy

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Flipped learning is a blended learning method based on academic engagement of students online (outside class) and offline (inside class). In this learning pedagogy, students receive lesson any time from lecture videos pre-loaded on digital platform at their convenience places and it is followed by in-classroom activities such as doubt clearing, problem solving, etc. However, students are constantly exposed to high levels of distraction in this age of the Internet. Therefore, it is hard for an instructor to know whether a student has paid attention while watching pre-loaded lecture video. In order to analyze attention level of individual students, captured brain signal or electroencephalogram (EEG) of students can be utilized. In this study, we utilize a popular feature extraction technique called Local Binary Pattern (LBP) and improvise it to develop an enhanced feature selection method. The adapted feature selection method termed as 1D Multi-Point Local Ternary Pattern (1D MP-LTP) is used to extract unique features from collected electroencephalogram (EEG) signals. Standard classification techniques are exploited to classify the attention level of students. Experiments are conducted with the data captured at Intelligent Data Analysis Lab, NIT Rourkela, to show effectiveness of the proposed feature extraction technique. The proposed 1D Multi-Point Local Ternary Pattern (1D MP-LTP)-based classification techniques outperform traditional and state-of-the-art classification techniques using LBP. This research can be helpful for instructors to identify students who need special care for improving their learning ability. Researchers in educational technology can extend this work by adopting this methodology in other online teaching pedagogy such as Massive Open Online Courses (MOOC).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data and Code Availability

Data and Code are available upon request to the corresponding author.

Notes

  1. http://mwm2.neurosky.com

References

  1. Savvaki C, Leonidis A, Paparoulis G, Antona M, Stephanidis C. Designing a technology–augmented school desk for the future classroom. In: Proceedings of the International Conference on Human-Computer Interaction. 2013. p. 681–685.

  2. Abeysekera L, Dawson P. Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. Higher Education Research & Development. 2015;34(1):1–14.

    Article  Google Scholar 

  3. Hao Y. Exploring undergraduates’ perspectives and flipped learning readiness in their flipped classrooms. Comput Hum Behav. 2016;59:82–92.

    Article  Google Scholar 

  4. Roach T. Student perceptions toward flipped learning: New methods to increase interaction and active learning in economics. International Review of Economics Education. 2014;17:74–84.

    Article  Google Scholar 

  5. Gren L. A flipped classroom approach to teaching empirical software engineering. IEEE Trans Educ. 2020;63(3):155–63.

    Article  Google Scholar 

  6. Kar P, Chattopadhyay S, Chakraborty S. Gestatten: Estimation of User’s attention in mobile Moocs from eye gaze and gaze gesture tracking. Proc ACM Hum-Comput Interact. 2020;4(72):32–72.

    Google Scholar 

  7. Xiao X, Wang J. Towards attentive, bi-directional MOOC learning on mobile devices. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 2015. p. 163–170.

  8. Kim Y, Soyata T, Behnagh RF. Towards emotionally aware AI smart classroom: current issues and directions for engineering and education. IEEE Access. 2018;6:5308–31.

    Article  Google Scholar 

  9. Chen C-M, Wang J-Y, Yu C-M. Assessing the attention levels of students by using a novel attention aware system based on brainwave signals. Br J Edu Technol. 2017;48(2):348–69.

    Article  Google Scholar 

  10. Sinha A, Chatterjee D, Saha SK, Basu A. Validation of stimulus for EEG signal based cognitive load analysis. In: Proceedings of the 2015 Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG). 2015. p. 1–4.

  11. Lee D-Y, Lee M, Lee S-W. Classification of imagined speech using Siamese neural network. In: Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 2020. p. 2979–2984.

  12. Jiang H, Dykstra K, Whitehill J. Predicting when teachers look at their students in 1-on-1 tutoring sessions. In: Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition. 2018. p. 593–598.

  13. Wei X, Cheng I-L, Chen N-S, Yang X, Liu Y, Dong Y, Zhai X, et al. Effect of the flipped classroom on the mathematics performance of middle school students. Educ Technol Res Dev. 2020:1–24.

  14. Shaw R, Mohanty C, Pradhan A, Patra BK. Attention analysis in flipped classroom using 1d multi-point local ternary patterns. In: 2021 International Conference on Advanced Learning Technologies (ICALT). 2021. p. 4–5.

  15. Ojala T, Pietikäinen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 1996;29(1):51–9.

    Article  Google Scholar 

  16. Zhang L, Chu R, Xiang S, Liao S, Li SZ. Face detection based on multi-block lbp representation. In: Proceedings of the International conference on biometrics; 2007. p. 11–18.

  17. Lan R, Lu H, Zhou Y, Liu Z, Luo X. An LBP encoding scheme jointly using quaternionic representation and angular information. Neural Comput Appl. 2020;32(9):4317–23.

    Article  Google Scholar 

  18. Khan KA, Shanir P, Khan YU, Farooq O. A hybrid Local Binary Pattern and wavelets based approach for EEG classification for diagnosing epilepsy. Expert Syst Appl. 2020;140.

    Article  Google Scholar 

  19. Tuncer T, Dogan S, Ertam F, Subasi A. A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals. Cogn Neurodyn. 2020:1–15.

  20. Kaya Y, Uyar M, Tekin R, Yıldırım S. 1D-local binary pattern based feature extraction for classification of epileptic EEG signals. Appl Math Comput. 2014;243:209–19.

    MathSciNet  MATH  Google Scholar 

  21. Tuncer T, Dogan S, Subasi A. A new fractal pattern feature generation function based emotion recognition method using EEG. Chaos, Solitons & Fractals. 2021;144.

    Article  MathSciNet  Google Scholar 

  22. Benouis M, Mostefai L, Costen N, Regouid M. ECG based biometric identification using one-dimensional local difference pattern. Biomed Signal Process Control. 2021;64.

    Article  Google Scholar 

  23. Kuncan M, Kaplan K, Minaz MR, Kaya Y, Ertunc HM. A novel feature extraction method for bearing fault classification with one dimensional ternary patterns. ISA Trans. 2020;100:346–57.

    Article  Google Scholar 

  24. Rogaten J, Rienties B, Sharpe R, Cross S, Whitelock D, Lygo-Baker S, Little-john A. Reviewing affective, behavioural and cognitive learning gains in higher education. Assess Eval High Educ. 2019;44(3):321–37.

    Article  Google Scholar 

  25. Giacomone B, Beltrán-Pellicer P, Godino JD. Cognitive analysis on prospective mathematics teachers’ reasoning using area and tree diagrams. International Journal of Innovation in Science and Mathematics Education. 2019;27(2):18–32.

    Article  Google Scholar 

  26. Roohr KC, Liu H, Liu OL. Investigating student learning gains in college: a longitudinal study. Stud High Educ. 2017;42(12):2284–300.

    Article  Google Scholar 

  27. Emke AR, Butler AC, Larsen DP. Effects of team-based learning on short-term and long-term retention of factual knowledge. Med Teach. 2016;38(3):306–11.

    Google Scholar 

  28. Pappas IO, Giannakos MN, Mikalef P. Investigating students’ use and adoption of with-video assignments: lessons learnt for video-based open educational resources. J Comput High Educ. 2017;29(1):160–77.

    Article  Google Scholar 

  29. Papamitsiou Z, Pappas IO, Sharma K, Giannakos M. Utilizing multimodal data through fsqca to explain engagement in adaptive learning. IEEE Trans Learn Technol. 2020.

  30. Sharma K, Papamitsiou Z, Giannakos M. Building pipelines for educational data using AI and multimodal analytics: A “grey-box” approach. Br J Edu Technol. 2019;50(6):3004–31.

    Article  Google Scholar 

  31. Mangaroska K, Vesin B, Kostakos V, Brusilovsky P, Giannakos M. Architecting analytics across multiple e-learning systems to enhance learning design. IEEE Trans Learn Technol. 2021.

  32. Subramaniam SR, Muniandy B. The effect of flipped classroom on students’ engagement. Technol Knowl Learn. 2019;24(3):355–72.

    Article  Google Scholar 

  33. Sojayapan C, Khlaisang J. The effect of a flipped classroom with online group investigation on students’ team learning ability. Kasetsart Journal of Social Sciences. 2020;41(1):28–33.

    Google Scholar 

  34. Sharma K, Giannakos M, Dillenbourg P. Eye-tracking and artificial intelligence to enhance motivation and learning. Smart Learning Environments. 2020;7:1–19.

    Article  Google Scholar 

  35. Giannakos MN, Sharma K, Papavlasopoulou S, Pappas IO, Kostakos V. Fitbit for learning: Towards capturing the learning experience using wearable sensing. Int J Hum Comput Stud. 2020;136.

    Article  Google Scholar 

  36. Lin FR, Kao CM. Mental effort detection using EEG data in E-learning contexts. Comput Educ. 2018;122:63–79.

    Article  Google Scholar 

  37. Giannakos MN, Sharma K, Pappas IO, Kostakos V, Velloso E. Multimodal data as a means to understand the learning experience. Int J Inf Manage. 2019;48:108–19.

    Article  Google Scholar 

  38. Lin HCK, Su SH, Chao CJ, Hsieh CY, Tsai SC. Construction of multi-mode affective learning system: taking affective design as an example. J Educ Technol Soc. 2016;19(2):132–47.

    Google Scholar 

  39. Chen CM, Wu CH. Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance. Comput Educ. 2015;80:108–21.

    Article  Google Scholar 

  40. Huang YM, Liu MC, Lai CH, Liu CJ. Using humorous images to lighten the learning experience through questioning in class. Br J Edu Technol. 2017;48(3):878–96.

    Article  Google Scholar 

  41. Lai CH, Liu MC, Liu CJ, Huang YM. Using positive visual stimuli to lighten the online learning experience through in class questioning. International Review of Research in Open and Distributed Learning. 2016;17(1):23–41.

    Article  Google Scholar 

  42. Tan X, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process. 2010;19(6):1635–50.

    Article  MathSciNet  MATH  Google Scholar 

  43. Murala S, Maheshwari R, Balasubramanian R. Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process. 2012;21(5):2874–86.

    Article  MathSciNet  MATH  Google Scholar 

  44. Ding C, Choi J, Tao D, Davis LS. Multi-directional multi-level dual-cross patterns for robust face recognition. IEEE Trans Pattern Anal Mach Intell. 2015;38(3):518–31.

    Article  Google Scholar 

  45. Ramanna S, Tirunagari S, Windridge D. Epileptic seizure detection using constrained singular spectrum analysis and 1D-local binary patterns. Heal Technol. 2020:1–11.

  46. Sairamya N, Subathra M, Suviseshamuthu ES, George ST. A new approach for automatic detection of focal EEG signals using wavelet packet decomposition and quad binary pattern method. Biomed Signal Process Control. 2021;63.

    Article  Google Scholar 

  47. Tirunagari S, Kouchaki S, Abasolo D, Poh N. One dimensional local binary patterns of electroencephalogram signals for detecting Alzheimer’s disease. In: Proceedings of the 2017 22nd International Conference on Digital Signal Processing (DSP); 2017. p. 1–5.

  48. Tuncer T, Ozyurt F, Dogan S, Subasi A. A novel COVID-19 and pneumonia classification method based on F-transform. Chemom Intell Lab Syst. 2021:104256.

  49. Giannakos MN, Krogstie J, Aalberg T. Video-based learning ecosystem to support active learning: application to an introductory computer science course. Smart Learning Environments. 2016;3(1):1–13.

    Article  Google Scholar 

  50. Balakrishnan G, Coetzee D. Predicting student retention in massive open online courses using hidden Markov models. Electrical Engineering and Computer Sciences University of California at Berkeley. 2013;53:57–8.

    Google Scholar 

  51. Hung H-C, Liu I-F, Liang C-T, Su Y-S. Applying educational data mining to explore students’ learning patterns in the flipped learning approach for coding education. Symmetry. 2020;12(2):213.

    Article  Google Scholar 

  52. Chao C-Y, Chen Y-T, Chuang K-Y. Exploring students’ learning attitude and achievement in flipped learning supported computer aided design curriculum: A study in high school engineering education. Comput Appl Eng Educ. 2015;23(4):514–26.

    Article  Google Scholar 

  53. Hwang G-J, Lai C-L, Wang S-Y. Seamless flipped learning: a mobile technology-enhanced flipped classroom with effective learning strategies. Journal of Computers in Education. 2015;2(4):449–73.

    Article  Google Scholar 

  54. Shaw R, Patra BK. Classifying students based on cognitive state in flipped learning pedagogy. Futur Gener Comput Syst. 2022;126:305–17.

    Article  Google Scholar 

  55. Szafir D, Mutlu B. ARTFul: adaptive review technology for flipped learning. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2013. p. 1001–1010.

  56. Heil CE, Walnut DF. Continuous and discrete wavelet transforms. SIAM Rev. 1989;31(4):628–66.

    Article  MathSciNet  MATH  Google Scholar 

  57. Shensa MJ. The discrete wavelet transform: wedding the a Trous and Mallat algorithms. IEEE Trans Signal Process. 1992;40(10):2464–82.

    Article  MATH  Google Scholar 

  58. Mi J-X, Yu B-X, Liu K, Deng X. Channel binary pattern based global-local spatial information fusion for motor imagery tasks. Informatics in Medicine Unlocked. 2020;20:100352.

  59. Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell. 2002;24(7):971–87.

    Article  MATH  Google Scholar 

  60. Desai R, Porob P, Rebelo P, Edla DR, Bablani A. EEG data classification for mental state analysis using wavelet packet transform and Gaussian process classifier. Wireless Pers Commun. 2020;115(3):2149–69.

    Article  Google Scholar 

Download references

Funding

The Science and Engineering Research Board (SERB), New Delhi, Government of India supports this work with File No: EMR/2017/004357, Dated 18/06/2018.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Rourkela, Rourkela, Odisha, 769008, India

    Rabi Shaw, Chinmay Mohanty, Bidyut Kr. Patra & Animesh Pradhan

Authors
  1. Rabi Shaw
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Chinmay Mohanty
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Bidyut Kr. Patra
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Animesh Pradhan
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Rabi Shaw.

Ethics declarations

Ethics Approval

Experiments were conducted with IRB approval from NIT Rourkela (Our institute).

Consent to Participate

Consent of each participant is obtained for the experiment.

Consent for Publication

Consent for publications was obtained for publishing their data and photographs.

Conflicts of interest

The authors are declaring that there are no conflicts of interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shaw, R., Mohanty, C., Patra, B.K. et al. 1D Multi-Point Local Ternary Pattern: A Novel Feature Extraction Method for Analyzing Cognitive Engagement of students in Flipped Learning Pedagogy. Cogn Comput 15, 1243–1256 (2023). https://doi.org/10.1007/s12559-022-10023-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-022-10023-5

Keywords