Skip to main content
SpringerLink
Log in

Intended human object detection for automatically protecting privacy in mobile video surveillance

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

With the recent popularization of mobile video cameras including camera phones, a new technology, mobile video surveillance, which uses mobile video cameras for video surveillance has been emerging. Such videos, however, may infringe upon the privacy of others by disclosing privacy sensitive information (PSI), i.e., their appearances. To prevent videos from infringing on the right to privacy, new techniques are required that automatically obscure PSI regions. The problem is how to determine the PSI regions to be obscured while maintaining enough video content to present the camera persons’ capture-intentions, i.e., what they want to record in their videos to achieve their surveillance tasks. To this end, we introduce a new concept called intended human objects that are defined as human objects essential for capture-intentions, and develop a new method called intended human object detection that automatically detects the intended human objects in videos taken by different camera persons. Through the process of intended human object detection, we develop a system for automatically obscuring PSI regions. We experimentally show the performance of intended human object detection and the contributions of the features used. Our user study shows the potential applicability of our proposed system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Babaguchi, N., Koshimizu, T., Umata, I., Toriyama, T.: Psychological study for designing privacy protected video surveillance system: PriSurv. In: Protecting Privacy in Video Surveillance, pp. 147–164. Springer, Berlin (2009)

  2. Bergstrand, F., Landgren, J.: Information sharing using live video in emergency response work. In: Proceedings of the 6th International ISCRAM Conference, pp. 1–5 (2009)

  3. Boyle, M., Edwards, C., Greenberg, S.: The effects of filtered video on awareness and privacy. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, pp. 1–10 (2000)

  4. Brassil, J.: Technical challenges in location-aware video surveillance privacy. In: Protecting Privacy in Video Surveillance, pp. 91–113. Springer, Berlin (2009)

  5. Chaudhari, J., Cheung, S.C.S., Venkatesh, M.V.: Privacy protection for life-log video. In: Proceedings of IEEE Workshop on Signal Processing Applications for Public Security and Forensics, pp. 1–5 (2007)

  6. Chen D., Chang Y., Yan R., Yang, J.: Tools for protecting the privacy of specific individuals in video. EURASIP J. Appl. Signal Process. 1, 1–9 (2007)

    Google Scholar 

  7. Cucchiara, R., Gualdi, G.: Mobile video surveillance systems: An architectural overview. In: Jiang, X., Ma, M., Chen, C. (eds.) Mobile Multimedia Processing, Lecture Notes in Computer Science, vol. 5960, pp. 89–109. Springer, Berlin (2010)

  8. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR 2005), pp. 886–893 (2005)

  9. Dufaux F., Ebrahimi, T.: Scrambling for privacy protection in video surveillance systems. IEEE Trans. Circuits Syst. Video Technol. 18(8), 1168–1174 (2008)

    Article  Google Scholar 

  10. Dufaux F., Konrad, J.: Efficient, robust, and fast global motion estimation for video coding. IEEE Trans. Image Process. 9(3), 497–501 (2000)

    Article  Google Scholar 

  11. Elazary, L., Itti, L.: Interesting objects are visually salient. J. Vis. 8(3), 1–15 (2008)

    Article  Google Scholar 

  12. Fan, J., Luo, H., Hacid, M.S., Bertin, E.: A novel approach for privacy-preserving video sharing. In: Proceedings of ACM Fourteenth Conference Information and Knowledge Management (CIKM2005), pp. 609–616 (2005)

  13. Flores, A., Belongie, S.: Removing pedestrians from google street view images. In: Proceedings of IEEE International Workshop on Mobile Vision, pp. 1–6 (2010)

  14. Frome, A., Cheung, G., Abdulkader, A., Zennaro, M., Wu, B., Bissacco, A., Adam, H., Neven, H., Vincent, L.: Large-scale privacy protection in google street view. In: Proceedings of IEEE 12th International Conference on Computer Vision (ICCV2009), pp. 2373–2380 (2009)

  15. Gross R., Sweeney L., De la Torre F., Baker, S.: Semi-supervised learning of multi-factor models for face de-identification. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR2008), pp. 1–8 (2008)

  16. Hu, Y., Rajan, D., Chia, L.T.: Attention-from-motion: A factorization approach for detecting attention objects in motion. Comput. Vis. Image Underst. 113(3), 319–331 (2009)

    Article  Google Scholar 

  17. Hua, X.S., Lu, L., Zhang, H.J.: Optimization-based automated home video editing system. IEEE Trans. Circuits Syst. Video Technol. 14(5), 572–583 (2004)

    Article  Google Scholar 

  18. Itti, L., Baldi, P.: A principled approach to detecting surprising events in videos. In: Proceedings of the IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR 2005), pp. 631–637 (2005)

  19. Itti L., Baldi, P.: Bayesian surprise attracts human attention. Vis. Res. 49(10), 1295–1306 (2009)

    Article  Google Scholar 

  20. Itti, L., Koch, C., Niebur, E.: A model of saliency based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  21. Kitahara, I., Kogure, K., Hagita, N.: Stealth vision for protecting privacy. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), vol 4, pp. 404–407 (2004)

  22. Landgren, J., Bergstrand, F.: Mobile live video in emergency response: its use and consequences. Bull. Am. Soc. Inf. Sci. Technol. 36(5), 27–19 (2010)

    Article  Google Scholar 

  23. Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum H.Y.: Learning to detect a salient object. In: Proceedings of the IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR 2007), pp. 1–8 (2007)

  24. Ma, Y.F., Zhang, H.J.: Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of the 11th ACM International Conference on Multimedia, pp. 374–381 (2003)

  25. Ma, Y.F., Hua, X.S., Lu, L., Zhang, H.J.: A generic framework of user attention model and its application in video summarization. IEEE Trans. Multimedia 7(5), 907–919 (2005)

    Article  Google Scholar 

  26. Matsushitam, Y., Ofek, E., Ge, W., Tang, X., Shum, H.Y.: Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell. 28(7), 1150–1163 (2006)

    Article  Google Scholar 

  27. Mei, T., Hua, X.S., Zhou, H.Q., Li, S.: Modeling and mining of users’ capture intention for home video. IEEE Trans. Multimedia 9(1), 66–77 (2007)

    Article  Google Scholar 

  28. Murase, H., Vinod, V.V.: Fast visual search using focused color matching—active search. Syst. Comput. Jpn. 31(9), 81–88 (2000)

    Article  Google Scholar 

  29. Nakashima, Y., Babaguchi, N., Fan, J.: Automatically protecting privacy in consumer generated videos using intended human object detector. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 1135–1138 (2010a)

  30. Nakashima, Y., Babaguchi, N., Fan, J.: Detecting intended human objects in human-captured videos. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2010 (CVPRW2010), pp. 1–8 (2010b)

  31. Neustaedter, C., Greenberg, S., Boyle, M.: Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput. Hum. Interact. 13(1), 1–36 (2006)

    Article  Google Scholar 

  32. Park, S., Trivedi, M.M.: A track-based human movement analysis and privacy protection system adaptive to environmental contexts. In: Proceedings IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS 2005), pp. 171–176 (2005)

  33. Peng, J., Babaguchi, N., Luo, H., Gao, Y., Fan, J.: Constructing distributed hippocratic video databases for privacy-preserving online patient training and counseling. IEEE Trans. Inf. Technol. Biomed. 14(4), 1014–1026 (2010)

    Article  Google Scholar 

  34. Platt, J.C.: Probabilities for SV machines. In: Advances in Large Margin Classifiers, MIT Press, Cambridge (1999)

  35. Tansuriyavong, S, Hanaki, S.: Privacy protection by concealing persons in circumstantial video image. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces, pp. 1–4 (2001)

  36. Wang, T., Mei, T., Hua, X.S., Liu, X.L., Zhou, H.Q.: Video collage: a novel presentation of video sequence. In: Proceedings of International Conference on Multimedia and Expo (ICME 2007), pp. 1479–1482 (2007)

  37. Yesil, B.: Recording and reporting: camera phones, user-generated images and surveillance. In: ICTs for Mobile and Ubiquitous Urban Infrastructures: Surveillance, Locative Media and Global Networks, IGI Global, pp. 272–293 (2011)

  38. Yu, X., Chinomi, K., Koshimizu, T., Nitta, N., Ito, Y., Babaguchi, N.: PriSurv: Privacy protecting visual processing for secure video surveillance. In: Proceedings of International Conference on Image Processing (ICIP 2008), pp. 1672–1675 (2008)

  39. Zhang, H.J., Hua, X.S., Lu, L.: AVE—automated home video editing. In: Proceedings of the 11th ACM International Conference on Multimedia, pp. 490–497 (2003)

Download references

Acknowledgments

This work was partly supported by a Grant-in-Aid for Scientific Research from the Japan Society for the Promotion of Science.

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Osaka University, 2-1 Yamadaoka, Suita, Osaka, Japan

    Yuta Nakashima & Noboru Babaguchi

  2. Department of Computer Science, UNC-Charlotte, Charlotte, NC, 28223, USA

    Jianping Fan

Authors
  1. Yuta Nakashima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Noboru Babaguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianping Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuta Nakashima.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nakashima, Y., Babaguchi, N. & Fan, J. Intended human object detection for automatically protecting privacy in mobile video surveillance. Multimedia Systems 18, 157–173 (2012). https://doi.org/10.1007/s00530-011-0244-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-011-0244-y

Keywords

Search

Navigation