skip to main content
research-article

Enabling Public Cameras to Talk to the Public

Published:05 July 2018Publication History
Skip Abstract Section

Abstract

This paper asks: Is it possible for cameras in public areas, say ceiling cameras in a museum, to send personalized messages to people without knowing any addresses of their phones? We define this kind of problem as Private Human Addressing and develop a real-time end-to-end system called PHADE to solve it. Unlike traditional data transmission protocols that need to first learn the destination's address, our cameras rely on viewing user's motion patterns, and use the uniqueness of these patterns as the address for communication. Once receiving the wireless broadcast from the cameras, the user's phone can locally compare the "motion address" of the packet against its own motion sensor data, and accept the packet upon a "good" match.

In addition to requiring no data from users, our system transforms the motion patterns into low-dimensional codes to prevent leakage of user's walking behaviors. Thus, a hacker who collects all the broadcast messages would still not be able to infer the motion patterns of users. Real-world experiments show that PHADE discriminates 2, 4, 6, 8, 10 people with 98%, 95%, 90%, 90%, 87% correctness and about 3 seconds constant delay. Since abundant and accurate information can be extracted from videos, PHADE would find applications in various contexts. Extended to localization system and audio guide, PHADE achieves a median error of 0.19m and 99.7% matching correctness, respectively. PHADE can also deliver messages based on human gestures. There is no need to deploy any extra infrastructures or to require users to rent any dedicated device.

References

  1. Edoardo Amaldi and Viggo Kann. 1998. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science 209, 1--2 (1998), 237--260. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Anton Andriyenko and Konrad Schindler. 2011. Multi-target tracking by continuous energy minimization. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 1265--1272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Ashwin Ashok, Shubham Jain, Marco Gruteser, Narayan Mandayam, Wenjia Yuan, and Kristin Dana. 2014. Capacity of pervasive camera based communication under perspective distortions. In Pervasive Computing and Communications (PerCom), 2014 IEEE International Conference on. IEEE, 112--120.Google ScholarGoogle ScholarCross RefCross Ref
  4. Hristo Bojinov, Yan Michalevsky, Gabi Nakibly, and Dan Boneh. 2014. Mobile device identification via sensor fingerprinting. arXiv preprint arXiv:1408.1416 (2014).Google ScholarGoogle Scholar
  5. Raymond C Browning, Emily A Baker, Jessica A Herron, and Rodger Kram. 2006. Effects of obesity and sex on the energetic cost and preferred speed of walking. Journal of Applied Physiology 100, 2 (2006), 390--398.Google ScholarGoogle ScholarCross RefCross Ref
  6. Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In CVPR.Google ScholarGoogle Scholar
  7. Shaxun Chen, Amit Pande, and Prasant Mohapatra. 2014. Sensor-assisted Facial Recognition: An Enhanced Biometric Authentication System for Smartphones. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '14). ACM, New York, NY, USA, 109--122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 1. IEEE, 886--893. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Yves-Alexandre De Montjoye, César A Hidalgo, Michel Verleysen, and Vincent D Blondel. 2013. Unique in the crowd: The privacy bounds of human mobility. Scientific reports 3 (2013), 1376.Google ScholarGoogle Scholar
  10. Sanorita Dey, Nirupam Roy, Wenyuan Xu, Romit Roy Choudhury, and Srihari Nelakuditi. 2014. AccelPrint: Imperfections of Accelerometers Make Smartphones Trackable.. In NDSS.Google ScholarGoogle Scholar
  11. Piotr Dollár. {n. d.}. Piotr's Computer Vision Matlab Toolbox (PMT). https://github.com/pdollar/toolbox.Google ScholarGoogle Scholar
  12. P. Dollár, R. Appel, and W. Kienzle. 2012. Crosstalk Cascades for Frame-Rate Pedestrian Detection. In ECCV. 645--659.Google ScholarGoogle Scholar
  13. Piotr Dollár, Ron Appel, and Wolf Kienzle. 2012. Crosstalk cascades for frame-rate pedestrian detection. In Computer Vision--ECCV 2012. Springer, 645--659.Google ScholarGoogle Scholar
  14. Piotr Dollár, Vincent Rabaud, Garrison Cottrell, and Serge Belongie. 2005. Behavior recognition via sparse spatio-temporal features. In Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005. 2nd Joint IEEE International Workshop on. IEEE, 65--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Howard Whitley Eves. 1972. A survey of geometry. Vol. 1. Allyn and Bacon.Google ScholarGoogle Scholar
  16. Olivier Faugeras. 1993. Three-dimensional computer vision: a geometric viewpoint. MIT press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2001. The elements of statistical learning. Vol. 1. Springer series in statistics Springer, Berlin.Google ScholarGoogle Scholar
  18. Ardeshir Goshtasby. 1986. Piecewise linear mapping functions for image registration. Pattern Recognition 19, 6 (1986), 459--466.Google ScholarGoogle ScholarCross RefCross Ref
  19. Ardeshir Goshtasby. 1988. Image registration by local approximation methods. Image and Vision Computing 6, 4 (1988), 255--261. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Helmut Grabner, Jiri Matas, Luc Van Gool, and Philippe Cattin. 2010. Tracking the invisible: Learning where the object might be. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 1285--1292.Google ScholarGoogle ScholarCross RefCross Ref
  21. Isabelle Guyon and André Elisseeff. 2003. An introduction to variable and feature selection. Journal of machine learning research 3, Mar (2003), 1157--1182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Kiryong Ha, Zhuo Chen, Wenlu Hu, Wolfgang Richter, Padmanabhan Pillai, and Mahadev Satyanarayanan. 2014. Towards wearable cognitive assistance. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, 68--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Puneet Jain, Justin Manweiler, and Romit Roy Choudhury. 2015. Overlay: Practical mobile augmented reality. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 331--344. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Niall Jenkins. {n. d.}. 245 million video surveillance cameras installed globally in 2014. IHS Technology ({n. d.}).Google ScholarGoogle Scholar
  25. Haojian Jin, Christian Holz, and Kasper Hornbñk. 2015. Tracko: Ad-hoc mobile 3d tracking using bluetooth low energy and inaudible signals for cross-device interaction. In Proceedings of the 28th Annual ACM Symposium on User Interface Software 8 Technology. ACM, 147--156. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Deokwoo Jung, Thiago Teixeira, and Andreas Savvides. 2010. Towards cooperative localization of wearable sensors using accelerometers and cameras. In INFOCOM, 2010 Proceedings IEEE. IEEE, 1--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Iris A Junglas and Richard T Watson. 2008. Location-based services. Commun. ACM 51, 3 (2008), 65--69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Rudolph Emil Kalman et al. 1960. A new approach to linear filtering and prediction problems. Journal of basic Engineering 82, 1 (1960), 35--45.Google ScholarGoogle ScholarCross RefCross Ref
  29. Daniel Kelly, Seamas Donnelly, and Brian Caulfield. 2015. Smartphone derived movement profiles to detect changes in health status in COPD patients-A preliminary investigation. In Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. IEEE, 462--465.Google ScholarGoogle ScholarCross RefCross Ref
  30. Cheng-Hao Kuo and Ram Nevatia. 2011. How does person identity recognition help multi-person tracking?. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 1217--1224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Hanchuan Li, Peijin Zhang, Samer Al Moubayed, Shwetak N Patel, and Alanson P Sample. 2016. Id-match: a hybrid computer vision and rid system for recognizing individuals in groups. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 4933--4944. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Tianxing Li, Chuankai An, Xinran Xiao, Andrew T Campbell, and Xia Zhou. 2015. Real-time screen-camera communication behind any scene. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 197--211. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Baiyang Liu, Junzhou Huang, Lin Yang, and Casimir Kulikowsk. 2011. Robust tracking using local sparse appearance model and k-selection. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 1313--1320. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. David Fernández Llorca, Ricardo Quintero, I Parra, and MA Sotelo. 2017. Recognizing individuals in groups in outdoor environments combining stereo vision, RFID and BLE. Cluster Computing 20, 1 (2017), 769--779. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Yan Michalevsky, Aaron Schulman, Gunaa Arumugam Veerapandian, Dan Boneh, and Gabi Nakibly. {n. d.}. PowerSpy: Location Tracking Using Mobile Device Power Analysis.Google ScholarGoogle Scholar
  36. AG Amitha Perera, Chukka Srinivas, Anthony Hoogs, Glen Brooksby, and Wensheng Hu. 2006. Multi-object tracking through simultaneous long occlusions and split-merge conditions. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, Vol. 1. IEEE, 666--673. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Hamed Pirsiavash, Deva Ramanan, and Charless C Fowlkes. 2011. Globally-optimal greedy algorithms for tracking a variable number of objects. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 1201--1208. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Nirupam Roy, He Wang, and Romit Roy Choudhury. 2014. I am a smartphone and i can tell my user's walking direction. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, 329--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Jochen Schiller and Agnès Voisard. 2004. Location-based services. Elsevier. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Matthias Scholz, Fatma Kaplan, Charles L Guy, Joachim Kopka, and Joachim Selbig. 2005. Non-linear PCA: a missing data approach. Bioinformatics 21, 20 (2005), 3887--3895. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Tomas Simon, Hanbyul Joo, Iain Matthews, and Yaser Sheikh. 2017. Hand Keypoint Detection in Single Images using Multiview Bootstrapping. In CVPR.Google ScholarGoogle Scholar
  42. Oncel Tuzel, Fatih Porikli, and Peter Meer. 2006. Region covariance: A fast descriptor for detection and classification. In European conference on computer vision. Springer, 589--600. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Anran Wang, Zhuoran Li, Chunyi Peng, Guobin Shen, Gan Fang, and Bing Zeng. 2015. Inframe++: Achieve simultaneous screen-human viewing and hidden screen-camera communication. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 181--195. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Chen Wang, Xiaonan Guo, Yan Wang, Yingying Chen, and Bo Liu. 2016. Friend or foe?: Your wearable devices reveal your personal pin. In Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. ACM, 189--200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. He Wang, Xuan Bao, Romit Roy Choudhury, and Srihari Nelakuditi. 2013. InSight: recognizing humans without face recognition. In Proceedings of the 14th Workshop on Mobile Computing Systems and Applications. ACM, 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. He Wang, Xuan Bao, Romit Roy Choudhury, and Srihari Nelakuditi. 2015. Visually fingerprinting humans without face recognition. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 345--358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. He Wang, Ted Tsung-Te Lai, and Romit Roy Choudhury. 2015. Mole: Motion leaks through smartwatch sensors. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 155--166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Shu Wang, Huchuan Lu, Fan Yang, and Ming-Hsuan Yang. 2011. Superpixel tracking. In Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 1323--1330. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Shih-En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. 2016. Convolutional pose machines. In CVPR.Google ScholarGoogle Scholar
  50. Junliang Xing, Haizhou Ai, and Shihong Lao. 2009. Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 1200--1207.Google ScholarGoogle Scholar
  51. Bo Yang and Ram Nevatia. 2012. Online learned discriminative part-based appearance models for multi-human tracking. In European Conference on Computer Vision. Springer, 484--498. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Bo Yang and Ramakant Nevatia. 2014. Multi-target tracking by online learning a CRF model of appearance and motion patterns. International Journal of Computer Vision 107, 2 (2014), 203--217. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Zhe Yang, Yuting Bao, Chuhao Luo, Xingya Zhao, Siyu Zhu, Chunyi Peng, Yunxin Liu, and Xinbing Wang. 2016. ARTcode: preserve art and code in any image. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 904--915. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Zengbin Zhang, David Chu, Xiaomeng Chen, and Thomas Moscibroda. 2012. Swordfight: Enabling a new class of phone-to-phone action games on commodity phones. In Proceedings of the 10th international conference on Mobile systems, applications, and services. ACM, 1--14. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Enabling Public Cameras to Talk to the Public

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
            Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 2, Issue 2
            June 2018
            741 pages
            EISSN:2474-9567
            DOI:10.1145/3236498
            Issue’s Table of Contents

            Copyright © 2018 ACM

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 5 July 2018
            • Accepted: 1 April 2018
            • Revised: 1 February 2018
            • Received: 1 August 2017
            Published in imwut Volume 2, Issue 2

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader