Skip to main content
SpringerLink
Log in

Device-Free Activity Recognition: A Survey

  • Conference paper
  • First Online:
Wireless Sensor Networks (CWSN 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1321))

Included in the following conference series:

Abstract

With the increasing number of WIFI hotspots and video surveillance equipment deployed, device-free activity recognition based on video and WIFI signals has attracted widespread attention. In order to better understand the current device-free human motion recognition work and the future development trend of device-free perception, this paper provides a detailed review of existing video-based and WIFI-based related work. Meanwhile, the principle of device-free activity recognition is deeply analyzed. We have compared the existing work in different aspects. Finally, by analyzing the shortcomings of the existing methods, the future research direction of device-free motion recognition is proposed.

This work was supported by the National Natural Science Foundation of China under Grant 51774282 and Grant 51874302.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Barr, P., Noble, J., Biddle, R.: Video game values: human-computer interaction and games. Interact. Comput. 19(2), 180–195 (2007)

    Article  Google Scholar 

  2. Foroughi, H., Aski, B.S., Pourreza, H.: Intelligent video surveillance for monitoring fall detection of elderly in home environments. In: 2008 11th International Conference on Computer and Information Technology, pp. 219–224. IEEE (2008)

    Google Scholar 

  3. Scheible, J., Ojala, T., Coulton, P.: MobiToss: a novel gesture based interface for creating and sharing mobile multimedia art on large public displays. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 957–960. ACM (2008)

    Google Scholar 

  4. Burton, A.M., Wilson, S., Cowan, M., Bruce, V.: Face recognition in poor-quality video: evidence from security surveillance. Psychol. Sci. 10(3), 243–248 (1999)

    Article  Google Scholar 

  5. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  6. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Chapter  Google Scholar 

  7. Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., Schiele, B.: DeeperCut: a deeper, stronger, and faster multi-person pose estimation model. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 34–50. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_3

    Chapter  Google Scholar 

  8. Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)

    Google Scholar 

  9. Bahl, P., Padmanabhan, V.N.: Radar: an in-building RF-based user location and tracking system, vol. 2, pp. 775–784 (2000)

    Google Scholar 

  10. Chetty, K., Smith, G.E., Woodbridge, K.: Through-the-wall sensing of personnel using passive bistatic WIFI radar at standoff distances. IEEE Trans. Geosci. Remote Sens. 50(4), 1218–1226 (2012)

    Article  Google Scholar 

  11. Adib, F., Katabi, D.: See through walls with Wi-Fi!. In: ACM SIGCOMM Conference on SIGCOMM (2013)

    Google Scholar 

  12. Pu, Q., Gupta, S., Gollakota, S., Patel, S.N.: Whole-home gesture recognition using wireless signals, pp. 27–38 (2013)

    Google Scholar 

  13. Halperin, D., Hu, W., Sheth, A., Wetherall, D.: Tool release: gathering 802.11n traces with channel state information. ACM SIGCOMM Comput. Commun. Rev. 41(1), 53 (2011)

    Article  Google Scholar 

  14. Zheng, X., Wang, J., Shangguan, L., Zhou, Z., Liu, Y.: Smokey: ubiquitous smoking detection with commercial WIFI infrastructures. In: IEEE INFOCOM - The IEEE International Conference on Computer Communications (2016)

    Google Scholar 

  15. Wang, G., Zou, Y., Zhou, Z., Wu, K., Ni, L.M.: We can hear you with Wi-Fi!. In: International Conference on Mobile Computing & Networking (2014)

    Google Scholar 

  16. Qian, H., Mao, Y., Xiang, W., Wang, Z.: Recognition of human activities using SVM multi-class classifier. Pattern Recogn. Lett. 31(2), 100–111 (2010)

    Article  Google Scholar 

  17. Kim, W., Lee, J., Kim, M., Oh, D., Kim, C.: Human action recognition using ordinal measure of accumulated motion. Eurasip J. Adv. Signal Process. 2010(1), 1–11 (2010)

    Google Scholar 

  18. Fang, C.-H., Chen, J.-C., Tseng, C.-C., Lien, J.-J.J.: Human action recognition using spatio-temporal classification. In: Zha, H., Taniguchi, R., Maybank, S. (eds.) ACCV 2009. LNCS, vol. 5995, pp. 98–109. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12304-7_10

    Chapter  Google Scholar 

  19. Ziaeefard, M., Ebrahimnezhad, H.: Hierarchical human action recognition by normalized-polar histogram. In: 2010 20th International Conference on Pattern Recognition, pp. 3720–3723. IEEE (2010)

    Google Scholar 

  20. Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3D human skeletons as points in a lie group. In: IEEE Conference on Computer Vision & Pattern Recognition (2014)

    Google Scholar 

  21. Mahbub, U., Imtiaz, H., Ahad, M.: An optical flow based approach for action recognition. In: International Conference on Computer & Information Technology (2012)

    Google Scholar 

  22. Holte, M.B., Moeslund, T.B., Nikolaidis, N., Pitas, I.: 3D human action recognition for multi-view camera systems. In: 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, pp. 342–349. IEEE (2011)

    Google Scholar 

  23. Pham, C.H., Ducournau, A., Fablet, R., Rousseau, F.: Brain MRI super-resolution using deep 3D convolutional networks. In: IEEE International Symposium on Biomedical Imaging (2017)

    Google Scholar 

  24. Rahmani, H., Mian, A.: 3D action recognition from novel viewpoints. In: IEEE Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  25. Lin, S., Jia, K., Yeung, D.Y., Shi, B.E.: Human action recognition using factorized spatio-temporal convolutional networks (FSTCN). In: IEEE International Conference on Computer Vision (2015)

    Google Scholar 

  26. Shuiwang, J., Ming, Y., Kai, Y.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)

    Article  Google Scholar 

  27. Karpathy, A., Toderici, G., Shetty, S., Leung, T., Li, F.F.: Large-scale video classification with convolutional neural networks. In: Computer Vision & Pattern Recognition (2014)

    Google Scholar 

  28. Liu, C., Wei-Sheng, X.U., Qi-Di, W.U.: Spatiotemporal convolutional neural networks and its application in action recognition. Comput. Sci. (2015)

    Google Scholar 

  29. Kim, H., Uh, Y., Ko, S., Byun, H.: Weighing classes and streams: toward better methods for two-stream convolutional networks. Opt. Eng. 55(5), 053108 (2016)

    Article  Google Scholar 

  30. Feichtenhofer, C., Pinz, A., Zisserman, A.: Convolutional two-stream network fusion for video action recognition. In: Computer Vision & Pattern Recognition (2016)

    Google Scholar 

  31. Wu, Z., Jiang, Y.G., Xi, W., Hao, Y., Xue, X.: Multi-stream multi-class fusion of deep networks for video classification. In: ACM on Multimedia Conference (2016)

    Google Scholar 

  32. Doretto, G., Chiuso, A., Ying, N.W., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003)

    Article  MATH  Google Scholar 

  33. Yan, X., Chang, H., Shan, S., Chen, X.: Modeling video dynamics with deep dynencoder. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 215–230. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_15

    Chapter  Google Scholar 

  34. Srivastava, N., Mansimov, E., Salakhudinov, R.: Unsupervised learning of video representations using LSTMs. In: International Conference on Machine Learning (2015)

    Google Scholar 

  35. Goodfellow, I.J., et al.: Generative adversarial nets. In: International Conference on Neural Information Processing Systems (2014)

    Google Scholar 

  36. Duarte, M., Bonaventura, Z., Massot, M., Bourdon, A., Dumont, T.: A new numerical strategy with space-time adaptivity and error control for multi-scale streamer discharge simulations. J. Comput. Phys. 231(3), 1002–1019 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  37. Zhang, J., Han, Y., Tang, J., Hu, Q., Jiang, J.: Semi-supervised image-to-video adaptation for video action recognition. IEEE Trans. Cybern. 47(4), 960–973 (2016)

    Article  Google Scholar 

  38. Fernando, B., Anderson, P., Hutter, M., Gould, S.: Discriminative hierarchical rank pooling for activity recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1924–1932 (2016)

    Google Scholar 

  39. Jones, S., Shao, L., Zhang, J., Liu, Y.: Relevance feedback for real-world human action retrieval. Pattern Recogn. Lett. 33(4), 446–452 (2012)

    Article  Google Scholar 

  40. Sadek, S., Al-Hamadi, A., Michaelis, B., Sayed, U.: An action recognition scheme using fuzzy log-polar histogram and temporal self-similarity. Eurasip J. Adv. Signal Process. 2011(1), 540375 (2011)

    Article  Google Scholar 

  41. Ikizler-Cinbis, N., Sclaroff, S.: Object, scene and actions: combining multiple features for human action recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 494–507. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_36

    Chapter  Google Scholar 

  42. Lui, Y.M., Beveridge, J.R.: Tangent bundle for human action recognition. In: IEEE International Conference on Automatic Face & Gesture Recognition (2013)

    Google Scholar 

  43. Wang, H., Kläser, A., Schmid, C., Liu, C.L.: Action recognition by dense trajectories. In: Computer Vision & Pattern Recognition (2011)

    Google Scholar 

  44. Yu-Gang, J., Qi, D., Wei, L., Xiangyang, X., Chong-Wah, N.: Human action recognition in unconstrained videos by explicit motion modeling. IEEE Trans. Image Process. 2(11), 3781–3795 (2015)

    MathSciNet  MATH  Google Scholar 

  45. Cohen, N., Sharir, O., Shashua, A.: Deep SimNets. In: Computer Vision & Pattern Recognition (2016)

    Google Scholar 

  46. Wang, X., Farhadi, A., Gupta, A.: Actions transformations. In: IEEE Computer Vision & Pattern Recognition (2016)

    Google Scholar 

  47. Wang, H., Schmid, C.: Action recognition with improved trajectories. In: IEEE International Conference on Computer Vision (2014)

    Google Scholar 

  48. Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description (2015)

    Google Scholar 

  49. Wang, L., Yu, Q., Tang, X.: Action recognition with trajectory-pooled deep-convolutional descriptors. In: Computer Vision & Pattern Recognition (2015)

    Google Scholar 

  50. Wang, L., et al.: Temporal segment networks: towards good practices for deep action recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 20–36. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_2

    Chapter  Google Scholar 

  51. Varior, R.R., Shuai, B., Lu, J., Xu, D., Wang, G.: A Siamese long short-term memory architecture for human re-identification. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 135–153. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_9

    Chapter  Google Scholar 

  52. Dan, W., Zhang, D., Xu, C., Hao, W., Xiang, L.: Device-free WIFI human sensing: from pattern-based to model-based approaches. IEEE Commun. Mag. 55(10), 91–97 (2017)

    Article  Google Scholar 

  53. Ali, K., Liu, A.X., Wang, W., Shahzad, M.: Keystroke recognition using WIFI signals. In: International Conference on Mobile Computing & Networking (2015)

    Google Scholar 

  54. Dou, F., et al.: Breathing rhythm analysis in body centric networks. IEEE Access 6, 1 (2018)

    Article  Google Scholar 

  55. Gu, Y., Zhang, Y., Li, J., Ji, Y., An, X., Ren, F.: Sleepy: wireless channel data driven sleep monitoring via commodity WIFI devices. IEEE Trans. Big Data 1 (2019)

    Google Scholar 

  56. Villamizar, M., Suarez, J., Villanueva, J., Borja, G., Rios, E.D.L.: Design and implementation of sleep monitoring system using electrooculographs signals. In: Health Care Exchanges (2014)

    Google Scholar 

  57. Hong, L., Wei, Y., Wang, J., Yang, X., Huang, L.: WIFInger: talk to your smart devices with finger-grained gesture. In: ACM International Joint Conference on Pervasive & Ubiquitous Computing (2016)

    Google Scholar 

  58. Tan, S., Yang, J.: WIFInger: leveraging commodity WIFI for fine-grained finger gesture recognition. In: ACM International Symposium on Mobile Ad Hoc Networking & Computing (2016)

    Google Scholar 

  59. Wu, X., Chu, Z., Yang, P., Xiang, C., Zheng, X., Huang, W.: TW-See: human activity recognition through the wall with commodity Wi-Fi devices. IEEE Trans. Veh. Technol. 68(1), 306–319 (2019)

    Article  Google Scholar 

  60. Wei, X., Zhao, J., Li, X.Y., Zhao, K., Jiang, Z.: Electronic frog eye: counting crowd using WIFI. In: INFOCOM. IEEE (2015)

    Google Scholar 

  61. Yu, G., Zhan, J., Zhi, L., Jie, L., Ji, Y., Wang, X.: Sleepy: adaptive sleep monitoring from afar with commodity WIFI infrastructures. In: 2018 IEEE Wireless Communications and Networking Conference (WCNC) (2018)

    Google Scholar 

  62. He, L., Ota, K., Dong, M., Guo, M.: Learning human activities through Wi-Fi channel state information with multiple access points. IEEE Commun. Mag. 56(5), 124–129 (2018)

    Article  Google Scholar 

  63. Gao, Q., et al.: CSI-based device-free wireless localization and activity recognition using radio image features. IEEE Trans. Veh. Technol. 66(11), 10346–10356 (2017)

    Article  Google Scholar 

  64. Yan, W., Jian, L., Chen, Y., Gruteser, M., Liu, H.: E-eyes: device-free location-oriented activity identification using fine-grained WIFI signatures (2014)

    Google Scholar 

  65. Fu, X., Jing, C., Xiao, H.X., Gui, L., Sun, J.L., Wang, N.R.: SEARE: a system for exercise activity recognition and quality evaluation based on green sensing. IEEE Trans. Emerg. Top. Comput. 1 (2018)

    Google Scholar 

  66. Arshad, S., et al.: Wi-chase: a WIFI based human activity recognition system for sensorless environments. In: IEEE International Symposium on a World of Wireless (2017)

    Google Scholar 

  67. Chang, J.Y., Lee, K.Y., Lin, C.J., Hsu, W.: WIFI action recognition via vision-based methods. In: IEEE International Conference on Acoustics (2016)

    Google Scholar 

  68. Wang, W., Liu, A.X., Shahzad, M., Ling, K., Lu, S.: Device-free human activity recognition using commercial WIFI devices. IEEE J. Sel. Areas Commun. 35(5), 1118–1131 (2017)

    Article  Google Scholar 

  69. Zou, Y., Wang, Y., Ye, S., Wu, K., Ni, L.M.: TagFree: passive object differentiation via physical layer radiometric signatures. In: IEEE International Conference on Pervasive Computing & Communications (2017)

    Google Scholar 

  70. Tian, Z., Wang, J., Yang, X., Mu, Z.: WiCatch: a Wi-Fi based hand gesture recognition system. IEEE Access 6(99), 16911–16923 (2018)

    Article  Google Scholar 

  71. Qian, K., Wu, C., Zhang, Y., Zhang, G., Yang, Z., Liu, Y.: Widar2.0: passive human tracking with a single Wi-Fi link. In: Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, pp. 350–361 (2018)

    Google Scholar 

  72. Duan, S., Yu, T., Jie, H.: WiDriver: driver activity recognition system based on WIFI CSI. Int. J. Wirel. Inf. Netw. 25(3), 1–11 (2018)

    Google Scholar 

  73. Guo, X., Liu, B., Shi, C., Liu, H., Chen, Y., Chuah, M.C.: WIFI-enabled smart human dynamics monitoring. In: Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, pp. 1–13 (2017)

    Google Scholar 

  74. Peng, H., Jia, W.: WiFind: driver fatigue detection with fine-grained Wi-Fi signal features. In: Globecom IEEE Global Communications Conference (2018)

    Google Scholar 

  75. Zhang, L., Liu, M., Lu, L., Gong, L.: Wi-run: multi-runner step estimation using commodity Wi-Fi. In: Annual IEEE International Conference on Sensing, Communication, and Networking, SECON (2018)

    Google Scholar 

  76. Qian, K., Wu, C., Zheng, Y., Yang, C., Liu, Y.: Decimeter level passive tracking with WIFI. In: Workshop on Hot Topics in Wireless (2016)

    Google Scholar 

  77. Wang, Y., Wu, K., Ni, L.M.: WiFall: device-free fall detection by wireless networks. IEEE Trans. Mob. Comput. 16(2), 581–594 (2016)

    Article  Google Scholar 

  78. Qian, K., Wu, C., Zhou, Z., Yue, Z., Liu, Y.: Inferring motion direction using commodity Wi-Fi for interactive exergames (2017)

    Google Scholar 

  79. Zhao, M., et al.: Through-wall human pose estimation using radio signals. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7356–7365 (2018)

    Google Scholar 

  80. Wang, X., Chao, Y., Mao, S.: TensorBeat: tensor decomposition for monitoring multi-person breathing beats with commodity WIFI. ACM Trans. Intell. Syst. Technol. 9(1), 1–27 (2017)

    Article  Google Scholar 

  81. Turaga, P., Chellappa, R., Subrahmanian, V.S., Udrea, O.: Machine recognition of human activities: a survey. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1473–1488 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Mingzhi Pang, Xu Yang, Jing Liu, Peihao Li, Faren Yan & Pengpeng Chen

  2. China Mine Digitization Engineering Research Center, Ministry of Education, Xuzhou, 221116, China

    Pengpeng Chen

Authors
  1. Mingzhi Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peihao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Faren Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pengpeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pengpeng Chen .

Editor information

Editors and Affiliations

  1. Computer Science and Engineering Department, Northwest Normal University, Lanzhou, China

    Zhanjun Hao

  2. Computer Science and Engineering Department, Northwest Normal University, Lanzhou, China

    Xiaochao Dang

  3. Computer Science and Engineering Department, Northwest Normal University, Lanzhou, China

    Honghong Chen

  4. Computer Science and Engineering Department, Northwest Normal University, Lanzhou, China

    Fenfang Li

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pang, M., Yang, X., Liu, J., Li, P., Yan, F., Chen, P. (2020). Device-Free Activity Recognition: A Survey. In: Hao, Z., Dang, X., Chen, H., Li, F. (eds) Wireless Sensor Networks. CWSN 2020. Communications in Computer and Information Science, vol 1321. Springer, Singapore. https://doi.org/10.1007/978-981-33-4214-9_17

Download citation

  • DOI: https://doi.org/10.1007/978-981-33-4214-9_17

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-33-4213-2

  • Online ISBN: 978-981-33-4214-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation