Skip to main content
SpringerLink
Log in

Landmark Detection Based on Human Activity Recognition for Automatic Floor Plan Construction

  • Conference paper
  • First Online:
Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom 2022)

  • 595 Accesses

Abstract

Landmark detection technology has a wide range of applications in people's lives, including map correcting, localization and navigation, etc. Besides, landmarks are also utilized to label different areas for automatic floor plan construction. Currently, vision-based landmark detection methods have some limitations, such as light, camera shaking, and privacy-invasive. In addition, deep learning-based methods increase the time consumption of marking labels due to the huge requirement for data. Targeting the above challenges, our work first proposes a landmark detection approach based on Human Activity Recognition (HAR) for automatic floor plan construction, which introduces a self-attention model to recognize various landmarks by walker's daily activities due to their strong correlation. First, the accelerometer and gyroscope sensor data are extracted and eliminated by a Gaussian filter and are divided into the same length segments by slide window. Next, it is input into the self-attention network to train a human activity recognition model. Finally, the corresponding relationship between human activities and landmarks is created to detect landmarks through the trained HAR model. Empirical results on two publicly available USC-HAD and OPPORTUNITY datasets show our proposed approach can recognize landmarks effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Yanying, G., Lo, A., Niemegeers, I.: A survey of indoor positioning systems for wireless personal networks. IEEE Commun. Surv. Tutorials 11, 13–32 (2009)

    Google Scholar 

  2. Forman, G.H., Zahorjan, J.: The challenges of mobile computing. Computer 27, 38–47 (1994)

    Google Scholar 

  3. Barry, B., et al.: Educating for mobile computing: addressing the new challenges. In: Proceedings of the Final Reports on Innovation and Technology in Computer Science Education 2012 Working Groups Haifa, Israel: ACM, pp. 51–63 (2012)

    Google Scholar 

  4. Kakousis, K., Paspallis, N., Papadopoulos, G.A.: A survey of software adaptation in mobile and ubiquitous computing. Enterp. Inf. Syst. 4, 355–389 (2010)

    Google Scholar 

  5. Ladd, D., Alan, D., Avimanyu, S., et al.: Trends in mobile computing with in the is discipline: a ten-year retrospective. Commun. Assoc. Inf. Syst. 27, 285–316 (2010)

    Google Scholar 

  6. Gay, G.: Context-aware mobile computing: affordances of space, social awareness, and social influence. Synthesis Lectures on Human-Centered Informatics. Morgan and Claypool Publishers, San Rafael. vol. 2, pp. 1–62 (2009)

    Google Scholar 

  7. Sana.: A survey of indoor localization techniques. IOSR J. Electr. Electron. Eng. (IOSR-JEEE). 6, 69–76 (2013)

    Google Scholar 

  8. Alzantot, M.: Youssef, M.: Crowdinside: automatic construction of indoor floorplans. In: Proceedings of the 20th International Conference on Advances in Geographic Information Systems, New York, United States, pp. 99–108 (2012)

    Google Scholar 

  9. X. Zhang, Y. Jin, et al. CIMLoc: A crowdsourcing indoor digital map construction system for localization. In 2014 IEEE Ninth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP),  Singapore, pp. 1–6, IEEE (2014)

    Google Scholar 

  10. Elhamshary, M., Alzantot, M., Youssef, M.: JustWalk: a crowdsourcing approach for the automatic construction of indoor floorplans. IEEE Trans. Mob. Comput. 18(10), 2358–2371 (2018)

    Article  Google Scholar 

  11. Zhou, B., Li, Q., Mao, Q., Tu, W., et al.: ALIMC: activity landmark-based indoor mapping via crowdsourcing. IEEE Trans. Intell. Transp. Syst. 16(5), 2774–2785 (2015)

    Article  Google Scholar 

  12. Amarasinghe, D., Mann, G.K., Gosine, R.G.: Landmark detection and localization for mobile robot applications: a multisensor approach. Robotica 28(5), 663–673 (2010)

    Article  Google Scholar 

  13. Alansary, A., Oktay, O., et al.: Evaluating reinforcement learning agents for anatomical landmark detection. Med. Image Anal. 53, 156–164 (2019)

    Google Scholar 

  14. Nilwong, S., Hossain, D., et al.: Deep learning-based landmark detection for mobile robot outdoor localization. Machines 7(2), 25 (2019)

    Article  Google Scholar 

  15. Wang, Z., Vandersteen, C., Raffaelli, C., Guevara, N., Patou, F., Delingette, H.: One-shot learning for landmarks detection. In: Engelhardt, S., et al. (eds.) Deep Generative Models, and Data Augmentation, Labelling, and Imperfections. Lecture Notes in Computer Science, vol. 13003, pp. 163–172. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88210-5_15

    Chapter  Google Scholar 

  16. Weishaupt, F., Will, P.S., et al.: Robust point-shaped landmark detection using polarimetric radar. In: 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 859–865, IEEE (2021) 

    Google Scholar 

  17. Narayana, K., Goulette, F., Steux, B.: Planar landmark detection using a specific arrangement of LIDAR scanners. In: IEEE/ION Position, Location and Navigation Symposium, pp. 1057–1069, IEEE, May 2010

    Google Scholar 

  18. Ravankar, A., Hoshino, Y., Kobayashi, Y.: Robust landmark detection in vineyards using laser range sensor. In: The Proceedings of JSME annual Conference on Robotics and Mechatronics (Robomec), pp. 1A1-E03 (2019)

    Google Scholar 

  19. Sun, S., Yin, Y., et al.: D. Robust landmark detection and position measurement based on monocular vision for autonomous aerial refueling of UAVs. IEEE Trans. Cybern. 49(12), 4167–4179 (2018)

    Google Scholar 

  20. Rous, M., Lupschen, H., et al.: Vision-based indoor scene analysis for natural landmark detection. In: Proceedings of the 2005 IEEE International conference on Robotics and Automation, Barcelona, Spain, pp. 4642–4647 (2005)

    Google Scholar 

  21. Sun, S., Yin, Y., Wang, X., Xu, D.: Robust landmark detection and position measurement based on monocular vision for autonomous aerial refueling of UAVs. IEEE Trans. Cybern. 49(12), 4167–4179 (2018)

    Article  Google Scholar 

  22. Zheng, Y., Liu, D., Georgescu, B., Nguyen, H., Comaniciu, D.: 3D deep learning for efficient and robust landmark detection in volumetric data. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science, vol. 9349, pp. 565–572. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_69

    Chapter  Google Scholar 

  23. Schwendicke, F., et al.: Deep learning for cephalometric landmark detection: systematic review and meta-analysis. Clin. Oral Invest. 25(7), 4299–4309 (2021). https://doi.org/10.1007/s00784-021-03990-w

    Article  Google Scholar 

  24. Han, D., Gao, Y., Wu, G., Yap, P.-T., Shen, D.: Robust anatomical landmark detection for MR brain image registration. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014. Lecture Notes in Computer Science, vol. 8673, pp. 186–193. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10404-1_24

    Chapter  Google Scholar 

  25. Zhang, J., Liu, M., Shen, D.: Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks. IEEE Trans. Image Process. 26(10), 4753–4764 (2017)

    Google Scholar 

  26. Jheng, Y.-C., et al.: A novel machine learning-based algorithm to identify and classify lesions and anatomical landmarks in colonoscopy images. Surg. Endosc. 36(1), 640–650 (2021). https://doi.org/10.1007/s00464-021-08331-2

    Article  Google Scholar 

  27. Zhang, Z., Luo, P., et al.: Facial landmark detection by deep multi-task learning. In: European Conference on Computer Vision, Part II, Zurich, Switzerland, pp. 94–108, 6–12 Sep 2014

    Google Scholar 

  28. Liu, Z., et al.: Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions. IEEE Trans. Intell. Transp. Syst. 23(7) 6640–653 (2021)

    Google Scholar 

  29. Wang, C., Liu, J., Chen, Y., et al.: Towards in-baggage suspicious object detection using commodity wifi. In: 2018 IEEE Conference on Communications and Network Security (CNS), pp. 1–9. IEEE, May 2018

    Google Scholar 

  30. Beltrán, J., Guindel, C., Moreno, F.M., et al.: BirdNet: a 3D object detection framework from lidar information. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 3517–3523. IEEE, November 2018

    Google Scholar 

  31. Zhou, B., Elbadry, M., Gao, R., Ye, F.: Towards scalable indoor map construction and refinement using acoustics on smartphones. IEEE Trans. Mob. Comput. 19(1), 217–230 (2019)

    Article  Google Scholar 

  32. Dubois, A., François, C.: Human activities recognition with RGB-Depth camera using HMM. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, Osaka, Japan, 3–7 Jul 2013

    Google Scholar 

  33. Wang, K., He, J., Zhang, L.: Attention-based convolutional neural network for weakly labeled human activities’ recognition with wearable sensors. IEEE Sens. J. 19(17), 7598–7604 (2019)

    Google Scholar 

  34. Varshney, N., Bakariya, B.: Deep convolutional neural model for human activities recognition in a sequence of video by combining multiple CNN streams. Multimedia Tools Appl. 81, 1–13 (2021). https://doi.org/10.1007/s11042-021-11220-4

    Article  Google Scholar 

  35. Liu, Z., Han, Y., Chen, Z., Fang, Y., Qian, H., Zhou, J.: Human activities recognition from videos based on compound deep neural network. In: Liu, Qi., Liu, X., Shen, T., Qiu, X. (eds.) The 10th International Conference on Computer Engineering and Networks. Advances in Intelligent Systems and Computing, vol. 1274, pp. 314–326. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-8462-6_37

    Chapter  Google Scholar 

  36. Gnouma, M., Ladjailia, A., Ejbali, R., Zaied, M.: Stacked sparse autoencoder and history of binary motion image for human activity recognition. Multimedia Tools Appl. 78(2), 2157–2179 (2018). https://doi.org/10.1007/s11042-018-6273-1

    Article  Google Scholar 

  37. Snoun, A., Jlidi, N., Bouchrika, T., Jemai, O., Zaied, M.: Towards a deep human activity recognition approach based on video to image transformation with skeleton data. Multimedia Tools Appl. 80(19), 29675–29698 (2021). https://doi.org/10.1007/s11042-021-11188-1

    Article  Google Scholar 

  38. Murad, A., Pyun, J.Y.: Deep recurrent neural networks for human activity recognition. Sensors 17(11), 2556 (2017)

    Article  Google Scholar 

  39. Xu, C., et al.: InnoHAR: a deep neural network for complex human activity recognition. IEEE Access 7, 9893–9902 (2019)

    Article  Google Scholar 

  40. Zhang, F., et al.: Towards a diffraction-based sensing approach on human activity recognition. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3(1), 1–25 (2019)

    Google Scholar 

  41. Yan, H., et al.: WiAct: a passive WiFi-based human activity recognition system. IEEE Sens. J. 20(1), 296–305 (2019)

    Article  Google Scholar 

  42. Bashar, S.K., Abdullah, A.F., Ki, H.C.: Smartphone based human activity recognition with feature selection and dense neural network. In: 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Montreal, Canada, pp. 20–24 (2020)

    Google Scholar 

  43. Mahmud, S., Tonmoy, M.: et al.: Human activity recognition from wearable sensor data using self-attention. arXiv preprint arXiv:2003.09018. (2020)

  44. Zhang, M., Sawchuk, A.A.: USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, USA, pp. 1036–1043 (2012)

    Google Scholar 

  45. Roggen. D., Calatroni, A., et al:. Collecting complex activity datasets in highly rich networked sensor environments. In: 2010 Seventh International Conference on Networked Sensing Systems (INSS), Kassel, Germany, pp. 233–240, IEEE (2010)

    Google Scholar 

  46. Thakur, D., Biswas, S., Ho., et al.: ConvAE-LSTM: convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition. IEEE Access 10, 4137–4156 (2022)

    Google Scholar 

  47. Lim, X.Y., Gan, K.B., et al.: Deep ConvLSTM network with dataset resampling for upper body activity recognition using minimal number of IMU sensors. Appl. Sci. 11(8), 3543 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Engineering and Computer Science, Queen Mary University of London, London, E1 4NS, UK

    Zhao Huang & Stefan Poslad

  2. School of Architecture and Urban Planning, Shenzhen University, Shenzhen, 58000, China

    Qingquan Li

  3. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, 430072, China

    Jianping Li & Chi Chen

Authors
  1. Zhao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Poslad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingquan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chi Chen .

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Honghao Gao

  2. Xi’an Jiaotong-Liverpool University, Suzhou, China

    Xinheng Wang

  3. Zhejiang University City College, Hangzhou, China

    Wei Wei

  4. London South Bank University, London, UK

    Tasos Dagiuklas

Rights and permissions

Reprints and permissions

Copyright information

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, Z., Poslad, S., Li, Q., Li, J., Chen, C. (2022). Landmark Detection Based on Human Activity Recognition for Automatic Floor Plan Construction. In: Gao, H., Wang, X., Wei, W., Dagiuklas, T. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 461. Springer, Cham. https://doi.org/10.1007/978-3-031-24386-8_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-24386-8_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-24385-1

  • Online ISBN: 978-3-031-24386-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation