Md Akib Shahriar Khan
Md Jannatul Baki Showmik
A F M Saifuddin Saif
ABSTRACT
Analysis of pedestrian activities in the video sequences is an intriguing domain that incorporates vast applications, such as autonomous driving systems, traffic control systems and interactions between people and computers. The primary focus of this research was on evaluating several strategies to analyse pedestrian activities effectively. The constructive comparison included three main steps, i.e. detection of the pedestrian, recognition of their actions and prediction about the activity of the pedestrian. Changes in activities of pedestrians, dynamic background, moving camera, view angle and processing time made it more challenging. Recent approaches were justified and compared based on precision accuracy, processing time and minimum resource allocation. The results were also compared by a series of state-of-the-art research datasets with provided significant observations in terms of greater accuracy which can lead to the construction of an extremely improvised system that would save pedestrian people from road accidents and assist autonomous driving systems. The purpose of this study is to discuss the current progress using different approaches.
- A.S. Saif, M.A.S. Khan, A.M. Hadi, R.P. Karmoker, and J.J. Gomes, Aggressive Action Estimation: A Comprehensive Review on Neural Network Based Human Segmentation and Action Recognition. International Journal of Education and Management Engineering 9(2019) 9. https://doi.org/10.5815/ijeme.2019.01.02Google Scholar
Cross Ref
- J. Hariyono, and K.-H. Jo, Centroid based pose ratio for pedestrian action recognition, 2016 IEEE 25th International Symposium on Industrial Electronics (ISIE), IEEE, 2016, pp. 895-900. https://doi.org/10.1109/isie.2016.7745009Google Scholar
- A.T. Schulz, and R. Stiefelhagen, A controlled interactive multiple model filter for combined pedestrian intention recognition and path prediction, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 173-178. https://doi.org/10.1109/itsc.2015.37Google ScholarDigital Library
- M. Raza, Z. Chen, S.-U. Rehman, P. Wang, and P. Bao, Appearance based pedestrians’ head pose and body orientation estimation using deep learning. Neurocomputing 272 (2018) 647-659. https://doi.org/10.1016/j.neucom.2017.07.029Google ScholarDigital Library
- H. Song, I.K. Choi, M.S. Ko, J. Bae, S. Kwak, and J. Yoo, Vulnerable pedestrian detection and tracking using deep learning, 2018 International Conference on Electronics, Information, and Communication (ICEIC), IEEE, 2018, pp. 1-2. https://doi.org/10.23919/elinfocom.2018.8330547Google ScholarCross Ref
- E.J. Lee, B.C. Ko, and J.-Y. Nam, Recognizing pedestrian's unsafe behaviours in far-infrared imagery at night. Infrared Physics & Technology 76 (2016) 261-270. https://doi.org/10.1016/j.infrared.2016.03.006Google ScholarCross Ref
- R. Quintero, I. Parra, D.F. Llorca, and M. Sotelo, Pedestrian intention and pose prediction through dynamical models and behaviour classification, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 83-88. https://doi.org/10.1109/itsc.2015.22Google ScholarDigital Library
- L. Zhang, L. Lin, X. Liang, and K. He, Is faster r-cnn doing well for pedestrian detection?, European conference on computer vision, Springer, 2016, pp. 443-457. https://doi.org/10.1007/978-3-319-46475-6_28Google ScholarCross Ref
- J. Hariyono, and K.-H. Jo, Pedestrian action recognition using motion type classification, 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), IEEE, 2015, pp. 129- 132. https://doi.org/10.1109/cybconf.2015.7175919Google ScholarCross Ref
- R.M. Mueid, C. Ahmed, and M.A.R. Ahad, Pedestrian activity classification using patterns of motion and histogram of oriented gradient. Journal on Multimodal User Interfaces 10 (2016) 299-305. https://doi.org/10.1007/s12193-015-0178-3Google ScholarCross Ref
- B. Hilsenbeck, D. Münch, A.-K. Grosselfinger, W. Hübner, and M. Arens, Action recognition in the longwave infrared and the visible spectrum using Hough forests, 2016 IEEE International Symposium on Multimedia (ISM), IEEE, 2016, pp. 329-332. https://doi.org/10.1109/ism.2016.0072Google ScholarCross Ref
- P. Zhang, C. Lan, J. Xing, W. Zeng, J. Xue, and N. Zheng, View adaptive recurrent neural networks for high performance human action recognition from skeleton data, Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2117-2126. https://doi.org/10.1109/iccv.2017.233Google ScholarCross Ref
- C. Li, Z. Cui, W. Zheng, C. Xu, R. Ji, and J. Yang, Action-attending graphic neural network. IEEE Transactions on Image Processing 27 (2018) 3657-3670. https://doi.org/10.1109/tip.2018.2815744Google ScholarCross Ref
- J.S. Casallas, J.H. Oliver, J.W. Kelly, F. Merienne, and S. Garbaya, Towards a model for predicting intention in 3D moving-target selection tasks, International Conference on Engineering Psychology and Cognitive Ergonomics, Springer, 2013, pp. 13-22. https://doi.org/10.1007/978-3-642-39360-0_2Google ScholarCross Ref
- A.T. Schulz, and R. Stiefelhagen, A controlled interactive multiple model filter for combined pedestrian intention recognition and path prediction, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 173-178. https://doi.org/10.1109/itsc.2015.37Google ScholarDigital Library
- M. Raza, Z. Chen, S.-U. Rehman, P. Wang, and P. Bao, Appearance based pedestrians’ head pose and body orientation estimation using deep learning. Neurocomputing 272 (2018) 647- 659. https://doi.org/10.1016/j.neucom.2017.07.029Google ScholarDigital Library
- A. Rudenko, L. Palmieri, and K.O. Arras, Joint long-term prediction of human motion using a planning-based social force approach, 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2018, pp. 1-7. https://doi.org/10.1109/icra.2018.8460527Google ScholarDigital Library
- I. Batkovic, M. Zanon, N. Lubbe, and P. Falcone, A computationally efficient model for pedestrian motion prediction, 2018 European Control Conference (ECC), IEEE, 2018, pp. 374-379. https://doi.org/10.23919/ecc.2018.8550300Google ScholarCross Ref
- D.-P. Tran, N.G. Nhu, and V.-D. Hoang, Pedestrian action prediction based on deep features extraction of human posture and traffic scene, Asian Conference on Intelligent Information and Database Systems, Springer, 2018, pp. 563-572. https://doi.org/10.1007/978-3-319-75420-8_53Google ScholarCross Ref
- H. Kataoka, Y. Satoh, Y. Aoki, S. Oikawa, and Y. Matsui, Temporal and fine-grained pedestrian action recognition on driving recorder database. Sensors 18 (2018) 627. https://doi.org/10.3390/s18020627Google ScholarCross Ref
- J.-Y. Kwak, B.C. Ko, and J.-Y. Nam, Pedestrian intention prediction based on dynamic fuzzy automata for vehicle driving at nighttime. Infrared Physics & Technology 81 (2017) 41-51. https://doi.org/10.1016/j.infrared.2016.12.014Google ScholarCross Ref
- R. Mueid, L. Christopher, and R. Tian, Vehicle-pedestrian dynamic interaction through tractography of relative movements and articulated pedestrian pose estimation, 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), IEEE, 2016, pp. 1-6. https://doi.org/10.1109/aipr.2016.8010592Google ScholarCross Ref
- O. Ghori, R. Mackowiak, M. Bautista, N. Beuter, L. Drumond, F. Diego, and B. Ommer, Learning to forecast pedestrian intention from pose dynamics, 2018 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2018, pp. 1277-1284. https://doi.org/10.1109/ivs.2018.8500657Google ScholarDigital Library
- K. Nishida, T. Kobayashi, T. Iwamoto, and S. Yamasaki, Pedestrian action prediction using static image feature, 2015 7th International Joint Conference on Computational Intelligence (IJCCI), IEEE, 2015, pp. 99-105. https://doi.org/10.5220/0005593600990105Google ScholarDigital Library
- J. Qianyin, L. Guoming, Y. Jinwei, and L. Xiying, A model based method of pedestrian abnormal behaviour detection in traffic scene, 2015 IEEE First International Smart Cities Conference (ISC2), IEEE, 2015, pp. 1-6. https://doi.org/10.1109/isc2.2015.7366164Google ScholarCross Ref
- R.Q. Mínguez, I.P. Alonso, D. Fernández-Llorca, and M.Á. Sotelo, Pedestrian path, pose, and intention prediction through gaussian process dynamical models and pedestrian activity recognition. IEEE Transactions on Intelligent Transportation Systems 20 (2018) 1803-1814. https://doi.org/10.1109/tits.2018.2836305Google ScholarCross Ref
- J. Almeida, and V. Santos, Pedestrian pose estimation using stereo perception, Robot 2015: Second Iberian Robotics Conference, Springer, 2016, pp. 491-502. https://doi.org/10.1007/978-3-319-27146-0_38Google ScholarCross Ref
- R. Quintero, I. Parra, D.F. Llorca, and M. Sotelo, Pedestrian intention and pose prediction through dynamical models and behaviour classification, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 83-88. https://doi.org/10.1109/itsc.2015.22Google ScholarDigital Library
- J. Hariyono, and K.-H. Jo, Detection of pedestrian crossing road using action classification model, 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), IEEE, 2015, pp. 21-24. https://doi.org/10.1109/aim.2015.7222502Google ScholarCross Ref
- E.J. Lee, B.C. Ko, and J.-Y. Nam, Recognizing pedestrians's unsafe behaviours in far-infrared imagery at night. Infrared Physics & Technology 76 (2016) 261-270. https://doi.org/10.1016/j.infrared.2016.03.006Google ScholarCross Ref
- Z. Fang, and A.M. López, Is the pedestrian going to cross? answering by 2d pose estimation, 2018 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2018, pp. 1271-1276. https://doi.org/10.1109/ivs.2018.8500413Google ScholarDigital Library
- J. Liu, A. Shahroudy, G. Wang, L.-Y. Duan, and A.K. Chichung, Skeleton-based online action prediction using scale selection network. IEEE transactions on pattern analysis and machine intelligence (2019). https://doi.org/10.1109/tpami.2019.2898954Google Scholar
- L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool, Temporal segment networks for action recognition in videos. IEEE transactions on pattern analysis and machine intelligence (2018). https://doi.org/10.1109/cvpr.2018.00705Google Scholar
- P. Zhang, C. Lan, J. Xing, W. Zeng, J. Xue, and N. Zheng, View adaptive neural networks for high performance skeleton-based human action recognition. IEEE transactions on pattern analysis and machine intelligence (2019). https://doi.org/10.1109/iccv.2017.233Google ScholarCross Ref
- S. Agahian, F. Negin, and C. Köse, An efficient human action recognition framework with pose-based spatiotemporal features. Engineering Science and Technology, an International Journal (2019). https://doi.org/10.1016/j.jestch.2019.04.014Google Scholar
- N. Jaouedi, N. Boujnah, and M.S. Bouhlel, A new hybrid deep learning model for human action recognition. Journal of King Saud University-Computer and Information Sciences (2019). https://doi.org/10.1016/j.jksuci.2019.09.004Google Scholar
- W. You, J. Guo, K. Shan, and Y. Dai, A Novel Trajectory-VLAD Based Action Recognition Algorithm for Video Analysis. Procedia Computer Science 147 (2019) 165-171. https://doi.org/10.1016/j.procs.2019.01.213Google ScholarDigital Library
- D. Anisuzzaman, and A.S. Saif, Efficient Framework Using Morphological Modelling for Frequent Iris Movement Investigation towards Questionable Observer Detection. International Journal of Image, Graphics and Signal Processing 10 (2018) 28. https://doi.org/10.5815/ijigsp.2018.11.04Google ScholarCross Ref
- D. Anisuzzaman, and A.S. Saif, A Study of Activity Recognition and Questionable Observer Detection. International Journal of Computer Applications 975 8887. https://doi.org/10.5120/ijca2018917855Google Scholar
- Z.R. Mahayuddin, and A.S. Saif, Fast and Effective Motion Model for Moving Object Detection Using Aerial Images. International Journal of Computer Vision and Signal Processing (IJCVSP) 1 (2018) 1-11.Google Scholar
- A. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Moving object detection using dynamic motion modelling from UAV aerial images. The Scientific World Journal 2014 (2014). https://doi.org/10.1155/2014/890619Google Scholar
- A.S. Saif, M.S. Hossain, K.T. Hasan, and M. Rahman, Measurement of Unique Pupillary Distance using Modified Circle Algorithm. International Journal of Computer Applications 975 8887. https://doi.org/10.5120/ijca2018916125Google Scholar
- A.S. Saif, and M.S. Hossain, A Study of Pupil Orientation and Detection of Pupil using Circle Algorithm: A Review. International Journal of Engineering Trends and Technology (IJETT) 54 (2017). https://doi.org/10.14445/22315381/ijett-v54p203Google Scholar
- A.S. Saif, A.G. Garba, J. Awwalu, H. Arshad, and L.Q. Zakaria, Performance Comparison of Min-Max Normalisation on Frontal Face Detection Using Haar Classifiers. Pertanika Journal of Science and Technology 25 (2017) 163-171.Google Scholar
- A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Moment feature based fast feature extraction algorithm for moving object detection using aerial images. PloS one 10 (2015) e0126212. https://doi.org/10.1371/journal.pone.0126212Google Scholar
- Z.R. Mahayuddin, A.S. Saif, and A.S. Prabuwono, Efficiency measurement of various denoise techniques for moving object detection using aerial images, 2015 International Conference on Electrical Engineering and Informatics (ICEEI), IEEE, 2015, pp. 161-165. https://doi.org/10.1109/iceei.2015.7352488Google ScholarCross Ref
- A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Motion analysis for moving object detection from UAV aerial images: A review, 2014 International Conference on Informatics, Electronics & Vision (ICIEV), IEEE, 2014, pp. 1-6. https://doi.org/10.1109/iciev.2014.6850753Google ScholarCross Ref
- A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Adaptive motion pattern analysis for machine vision based moving detection from UAV aerial images, International Visual Informatics Conference, Springer, 2013, pp. 104-114. https://doi.org/10.1007/978-3-319-02958-0_10Google ScholarDigital Library
- D. Nandi, A.S. Saif, P. Prottoy, K.M. Zubair, and S.A. Shubho, Traffic sign detection based on colour segmentation of obscure image candidates: a comprehensive study. International Journal of Modern Education and Computer Science 10 (2018) 35. https://doi.org/10.5815/ijmecs.2018.06.05Google ScholarCross Ref
- A.S. Saif, A.S. Prabuwono, Z.R. Mahayuddin, and H.T. Himawan, A review of machine vision based on moving objects: object detection from UAV aerial images. International Journal of Advancements in Computing Technology 5 (2013) 57.Google Scholar
- A.S. Saif, A.S. Prabuwono, Z.R. Mahayuddin, and T. Mantoro, Vision-based human face recognition using extended principal component analysis. International Journal of Mobile Computing and Multimedia Communications (IJMCMC) 5 (2013) 82-94. https://doi.org/10.4018/ijmcmc.2013100105Google Scholar
- A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Real time vision based object detection from UAV aerial images: a conceptual framework, FIRA RoboWorld Congress, Springer, 2013, pp. 265-274. https://doi.org/10.1007/978-3-642-40409-2_23Google ScholarCross Ref
- E.N. Kajabad, and S.V. Ivanov, People Detection and Finding Attractive Areas by the use of Movement Detection Analysis and Deep Learning Approach. Procedia Computer Science 156 (2019) 327-337. https://doi.org/10.1016/j.procs.2019.08.209Google ScholarDigital Library
- T. Wang, Z. Miao, Y. Chen, Y. Zhou, G. Shan, and H. Snoussi, AED-Net: An Abnormal Event Detection Network. Engineering (2019). https://doi.org/10.1016/j.eng.2019.02.008Google Scholar
- F. Letsch, D. Jirak, and S. Wermter, Localising salient body motion in multi-person scenes using convolutional neural networks. Neurocomputing 330 (2019) 449-464. https://doi.org/10.1016/j.neucom.2018.11.048Google ScholarCross Ref
- Z.R. Mahayuddin, and A.S. Saif, A COMPARATIVE STUDY OF THREE CORNER FEATURE BASED MOVING OBJECT DETECTION USING AERIAL IMAGES. Malaysian Journal of Computer Science S.1 (2019) 25-33. https://doi.org/10.22452/mjcs.sp2019no3.2Google ScholarCross Ref
- Z.R. Mahayuddin and A.S. Saif, Efficiency measurement of various denoise techniques for moving object detection using aerial images , International Visual Informatics Conference, Springer, 2019, pp. 227-236. https://doi.org/10.1007/978-3-030-34032-2_21Google ScholarCross Ref
- Saif, A. F., Khan, M. A., Hadi, A. M., Karmoker, R. P., & Gomes, J. J. (2021). Silhouette Pose Feature-Based Human Action Classification Using Capsule Network. Journal of Information Technology Research (JITR), 14(2), 106-124. http://doi.org/10.4018/JITR.2021040106Google Scholar
Index Terms
- A Constructive Review on Pedestrian Action Detection, Recognition and Prediction
Recommendations
Pedestrian Action Prediction using Static Image Feature
IJCCI 2015: Proceedings of the 7th International Joint Conference on Computational IntelligenceIn this study, we propose a method to predict how the target object move (run or walk) in the
future by using only appearance-based image features. Such kind of motion prediction significantly
contributes to intelligent braking system in cars; by ...
Human Action Recognition and Prediction: A Survey
AbstractDerived from rapid advances in computer vision and machine learning, video analysis tasks have been moving from inferring the present state to predicting the future state. Vision-based action recognition and prediction from videos are such tasks, ...
Detection of pedestrian crossing road
Detection of pedestrian crossing road is the objective of this work. The model incorporates the pedestrian pose recognition and lateral speed, motion direction and spatial layout of the environment. Pedestrian poses are recognized according to the ...
Comments