Abstract
With the recent popularization of mobile video cameras including camera phones, a new technology, mobile video surveillance, which uses mobile video cameras for video surveillance has been emerging. Such videos, however, may infringe upon the privacy of others by disclosing privacy sensitive information (PSI), i.e., their appearances. To prevent videos from infringing on the right to privacy, new techniques are required that automatically obscure PSI regions. The problem is how to determine the PSI regions to be obscured while maintaining enough video content to present the camera persons’ capture-intentions, i.e., what they want to record in their videos to achieve their surveillance tasks. To this end, we introduce a new concept called intended human objects that are defined as human objects essential for capture-intentions, and develop a new method called intended human object detection that automatically detects the intended human objects in videos taken by different camera persons. Through the process of intended human object detection, we develop a system for automatically obscuring PSI regions. We experimentally show the performance of intended human object detection and the contributions of the features used. Our user study shows the potential applicability of our proposed system.
Similar content being viewed by others
References
Babaguchi, N., Koshimizu, T., Umata, I., Toriyama, T.: Psychological study for designing privacy protected video surveillance system: PriSurv. In: Protecting Privacy in Video Surveillance, pp. 147–164. Springer, Berlin (2009)
Bergstrand, F., Landgren, J.: Information sharing using live video in emergency response work. In: Proceedings of the 6th International ISCRAM Conference, pp. 1–5 (2009)
Boyle, M., Edwards, C., Greenberg, S.: The effects of filtered video on awareness and privacy. In: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, pp. 1–10 (2000)
Brassil, J.: Technical challenges in location-aware video surveillance privacy. In: Protecting Privacy in Video Surveillance, pp. 91–113. Springer, Berlin (2009)
Chaudhari, J., Cheung, S.C.S., Venkatesh, M.V.: Privacy protection for life-log video. In: Proceedings of IEEE Workshop on Signal Processing Applications for Public Security and Forensics, pp. 1–5 (2007)
Chen D., Chang Y., Yan R., Yang, J.: Tools for protecting the privacy of specific individuals in video. EURASIP J. Appl. Signal Process. 1, 1–9 (2007)
Cucchiara, R., Gualdi, G.: Mobile video surveillance systems: An architectural overview. In: Jiang, X., Ma, M., Chen, C. (eds.) Mobile Multimedia Processing, Lecture Notes in Computer Science, vol. 5960, pp. 89–109. Springer, Berlin (2010)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR 2005), pp. 886–893 (2005)
Dufaux F., Ebrahimi, T.: Scrambling for privacy protection in video surveillance systems. IEEE Trans. Circuits Syst. Video Technol. 18(8), 1168–1174 (2008)
Dufaux F., Konrad, J.: Efficient, robust, and fast global motion estimation for video coding. IEEE Trans. Image Process. 9(3), 497–501 (2000)
Elazary, L., Itti, L.: Interesting objects are visually salient. J. Vis. 8(3), 1–15 (2008)
Fan, J., Luo, H., Hacid, M.S., Bertin, E.: A novel approach for privacy-preserving video sharing. In: Proceedings of ACM Fourteenth Conference Information and Knowledge Management (CIKM2005), pp. 609–616 (2005)
Flores, A., Belongie, S.: Removing pedestrians from google street view images. In: Proceedings of IEEE International Workshop on Mobile Vision, pp. 1–6 (2010)
Frome, A., Cheung, G., Abdulkader, A., Zennaro, M., Wu, B., Bissacco, A., Adam, H., Neven, H., Vincent, L.: Large-scale privacy protection in google street view. In: Proceedings of IEEE 12th International Conference on Computer Vision (ICCV2009), pp. 2373–2380 (2009)
Gross R., Sweeney L., De la Torre F., Baker, S.: Semi-supervised learning of multi-factor models for face de-identification. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR2008), pp. 1–8 (2008)
Hu, Y., Rajan, D., Chia, L.T.: Attention-from-motion: A factorization approach for detecting attention objects in motion. Comput. Vis. Image Underst. 113(3), 319–331 (2009)
Hua, X.S., Lu, L., Zhang, H.J.: Optimization-based automated home video editing system. IEEE Trans. Circuits Syst. Video Technol. 14(5), 572–583 (2004)
Itti, L., Baldi, P.: A principled approach to detecting surprising events in videos. In: Proceedings of the IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR 2005), pp. 631–637 (2005)
Itti L., Baldi, P.: Bayesian surprise attracts human attention. Vis. Res. 49(10), 1295–1306 (2009)
Itti, L., Koch, C., Niebur, E.: A model of saliency based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)
Kitahara, I., Kogure, K., Hagita, N.: Stealth vision for protecting privacy. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), vol 4, pp. 404–407 (2004)
Landgren, J., Bergstrand, F.: Mobile live video in emergency response: its use and consequences. Bull. Am. Soc. Inf. Sci. Technol. 36(5), 27–19 (2010)
Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum H.Y.: Learning to detect a salient object. In: Proceedings of the IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR 2007), pp. 1–8 (2007)
Ma, Y.F., Zhang, H.J.: Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of the 11th ACM International Conference on Multimedia, pp. 374–381 (2003)
Ma, Y.F., Hua, X.S., Lu, L., Zhang, H.J.: A generic framework of user attention model and its application in video summarization. IEEE Trans. Multimedia 7(5), 907–919 (2005)
Matsushitam, Y., Ofek, E., Ge, W., Tang, X., Shum, H.Y.: Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell. 28(7), 1150–1163 (2006)
Mei, T., Hua, X.S., Zhou, H.Q., Li, S.: Modeling and mining of users’ capture intention for home video. IEEE Trans. Multimedia 9(1), 66–77 (2007)
Murase, H., Vinod, V.V.: Fast visual search using focused color matching—active search. Syst. Comput. Jpn. 31(9), 81–88 (2000)
Nakashima, Y., Babaguchi, N., Fan, J.: Automatically protecting privacy in consumer generated videos using intended human object detector. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 1135–1138 (2010a)
Nakashima, Y., Babaguchi, N., Fan, J.: Detecting intended human objects in human-captured videos. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2010 (CVPRW2010), pp. 1–8 (2010b)
Neustaedter, C., Greenberg, S., Boyle, M.: Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput. Hum. Interact. 13(1), 1–36 (2006)
Park, S., Trivedi, M.M.: A track-based human movement analysis and privacy protection system adaptive to environmental contexts. In: Proceedings IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS 2005), pp. 171–176 (2005)
Peng, J., Babaguchi, N., Luo, H., Gao, Y., Fan, J.: Constructing distributed hippocratic video databases for privacy-preserving online patient training and counseling. IEEE Trans. Inf. Technol. Biomed. 14(4), 1014–1026 (2010)
Platt, J.C.: Probabilities for SV machines. In: Advances in Large Margin Classifiers, MIT Press, Cambridge (1999)
Tansuriyavong, S, Hanaki, S.: Privacy protection by concealing persons in circumstantial video image. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces, pp. 1–4 (2001)
Wang, T., Mei, T., Hua, X.S., Liu, X.L., Zhou, H.Q.: Video collage: a novel presentation of video sequence. In: Proceedings of International Conference on Multimedia and Expo (ICME 2007), pp. 1479–1482 (2007)
Yesil, B.: Recording and reporting: camera phones, user-generated images and surveillance. In: ICTs for Mobile and Ubiquitous Urban Infrastructures: Surveillance, Locative Media and Global Networks, IGI Global, pp. 272–293 (2011)
Yu, X., Chinomi, K., Koshimizu, T., Nitta, N., Ito, Y., Babaguchi, N.: PriSurv: Privacy protecting visual processing for secure video surveillance. In: Proceedings of International Conference on Image Processing (ICIP 2008), pp. 1672–1675 (2008)
Zhang, H.J., Hua, X.S., Lu, L.: AVE—automated home video editing. In: Proceedings of the 11th ACM International Conference on Multimedia, pp. 490–497 (2003)
Acknowledgments
This work was partly supported by a Grant-in-Aid for Scientific Research from the Japan Society for the Promotion of Science.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nakashima, Y., Babaguchi, N. & Fan, J. Intended human object detection for automatically protecting privacy in mobile video surveillance. Multimedia Systems 18, 157–173 (2012). https://doi.org/10.1007/s00530-011-0244-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00530-011-0244-y