skip to main content
10.1145/2663204.2663270acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
poster

CrossMotion: Fusing Device and Image Motion for User Identification, Tracking and Device Association

Published: 12 November 2014 Publication History

Abstract

Identifying and tracking people and mobile devices indoors has many applications, but is still a challenging problem. We introduce a cross-modal sensor fusion approach to track mobile devices and the users carrying them. The CrossMotion technique matches the acceleration of a mobile device, as measured by an onboard internal measurement unit, to similar acceleration observed in the infrared and depth images of a Microsoft Kinect v2 camera. This matching process is conceptually simple and avoids many of the difficulties typical of more common appearance-based approaches. In particular, CrossMotion does not require a model of the appearance of either the user or the device, nor in many cases a direct line of sight to the device. We demonstrate a real time implementation that can be applied to many ubiquitous computing scenarios. In our experiments, CrossMotion found the person's body 99% of the time, on average within 7cm of a reference device position.

Supplementary Material

MP4 File (icmi2561-file3.mp4)

References

[1]
Brox, T., Bruhn, A., Papenberg, N., and Weikert, J. High accuracy optical flow estimation based on a theory for warping. In Proc. 8th European Conference on Computer Vision, vol. 4. 2004. 25--36.
[2]
Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P., and Smith, M. Synchronous gestures for multiple persons and computers. In Proc. UIST 2003. 149--158.
[3]
Holmquist, L. E., Mattern, F., Schiele, B., Alahuhta, P., Beigl, M., and Gellersen, H. Smart-Its friends: A technique for users to easily establish connections between smart artefacts. In Proc. Ubicomp 2001. 116--122.
[4]
Kawai, J., Shintani, K., Haga, H., and Kaneda, S. Identification and positioning based on motion sensors and a video camera. In Proc. 4th IASTED Int. Conf. on Web-Based Education, 2005.
[5]
Maki, Y., Kagami, S., and Hashimoto, K. Accelerometer detection in camera view based on feature point tracking. In Proc. IEEE/SICE Int'l Symp. On System Integration. 2010. 448--453.
[6]
Marquardt, N., Hinckley, K., and Greenberg, S. Cross-device interaction via micro-mobility and F-formations. In Proc. UIST 2012. 13--22.
[7]
Mayrhofer, R., and Gellersen, H. Shake well before use: intuitive and secure pairing of mobile devices. IEEE Transactions on Mobile Computing, 8(6). 2009. 792--806.
[8]
Olwal, A., and Wilson, A. D. SurfaceFusion: Unobtrusive tracking of everyday objects in tangible interfaces. In Proc. Graphics Interface 2008. 235--242.
[9]
Plötz, T., Chen, C., Hammerla, N. Y., and Abowd, G. A. Automatic synchronization of wearable sensors and videocameras for ground truth annotation - a practical approach. In Proc. 16th Int. Symp. on Wearable Computers, 2012.
[10]
Rekimoto, J., Ayatsuka, Y., and Kohno, M. SyncTap: An interaction technique for mobile networking. In Proc. Mobile CHI 2003. 104--115.
[11]
Rofouei, M., Wilson, A. D., and Brush, A. J. Your phone or mine?: Fusing body, touch, and device sensing for multi-user device-display interaction. In Proc. ACM SIGCHI. 2012. 1915--1918.
[12]
Schmidt, D., Chehimi, F., Rukzio, E., and Gellersen, H. PhoneTouch: A technique for direct phone interaction on surfaces. In Proc. UIST 2010. 13--16.
[13]
Shigeta, O., Kagami, S., and Hashimoto, K. Identifying a moving object with an accelerometer in a camera view. In Proc. Int. Conf. Intelligent Robots and Systems, 2008.
[14]
Stein, S., and McKenna, S. J. Accelerometer localization in the view of a stationary camera. In Proc. Ninth Conference on Computer and Robot Vision, 2012. 109--116.
[15]
Teixeira, T., Jung, D., and Savvides, A. Tasking networked CCTV cameras and mobile phones to identify and localize multiple people. In Proc. Ubicomp, 2010. 213--222.
[16]
Want, R., Hopper, A., Falcão, V., and Gibbons, J. The active badge system. ACM Transactions on Information Systems, 10(1), 1992. 91--102.
[17]
Welch, G., and Bishop, G. An introduction to the Kalman filter. Technical Report TR 95-041. University of North Carolina at Chapel Hill Dept. of Computer Science. 1995.
[18]
Wilson, A. D., and Sarin, R. BlueTable: connecting wireless mobile devices on interactive surfaces using vision-based handshaking. In Proc. Graphics Interface 2007. 119--1.
[19]
Wolfe, J. M. Guided search 2.0: a revised model of visual search. Psychonomic Bulletin & Review 1, 2 (1994), 202--238.

Cited By

View all
  • (2022)Formalizing Digital Proprioception for Devices, Environments, and UsersAmbient Intelligence – Software and Applications – 12th International Symposium on Ambient Intelligence10.1007/978-3-031-06894-2_1(1-10)Online publication date: 1-Sep-2022
  • (2020)Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring ApplicationsSensors10.3390/s2009257620:9(2576)Online publication date: 1-May-2020
  • (2020)Motion Coupling of Earable Devices in Camera ViewProceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia10.1145/3428361.3428470(13-17)Online publication date: 22-Nov-2020
  • Show More Cited By

Index Terms

  1. CrossMotion: Fusing Device and Image Motion for User Identification, Tracking and Device Association

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction
      November 2014
      558 pages
      ISBN:9781450328852
      DOI:10.1145/2663204
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 12 November 2014

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. depth cameras
      2. inertial measurement units
      3. sensor fusion

      Qualifiers

      • Poster

      Conference

      ICMI '14
      Sponsor:

      Acceptance Rates

      ICMI '14 Paper Acceptance Rate 51 of 127 submissions, 40%;
      Overall Acceptance Rate 453 of 1,080 submissions, 42%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)4
      • Downloads (Last 6 weeks)2
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)Formalizing Digital Proprioception for Devices, Environments, and UsersAmbient Intelligence – Software and Applications – 12th International Symposium on Ambient Intelligence10.1007/978-3-031-06894-2_1(1-10)Online publication date: 1-Sep-2022
      • (2020)Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring ApplicationsSensors10.3390/s2009257620:9(2576)Online publication date: 1-May-2020
      • (2020)Motion Coupling of Earable Devices in Camera ViewProceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia10.1145/3428361.3428470(13-17)Online publication date: 22-Nov-2020
      • (2020)Gravity-Direction-Aware Joint Inter-Device Matching and Temporal Alignment between Camera and Wearable SensorsCompanion Publication of the 2020 International Conference on Multimodal Interaction10.1145/3395035.3425968(433-441)Online publication date: 25-Oct-2020
      • (2020)Facilitating Temporal Synchronous Target Selection through User Behavior ModelingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33698393:4(1-24)Online publication date: 14-Sep-2020
      • (2020)CoSafe: Securing Mobile Devices through Mutual Mobility Consistency VerificationIEEE Transactions on Mobile Computing10.1109/TMC.2020.2974222(1-1)Online publication date: 2020
      • (2019)Evaluation of Ad-hoc Secure Device Pairing Method with Accelerometer and Camera Using MarkerInternational Journal of Networking and Computing10.15803/ijnc.9.2_3189:2(318-338)Online publication date: 2019
      • (2019)Acceptance and perceptions of interactive location-tracking displaysProceedings of the 8th ACM International Symposium on Pervasive Displays10.1145/3321335.3324931(1-7)Online publication date: 12-Jun-2019
      • (2019)EyeFlowProceedings of the 11th ACM Symposium on Eye Tracking Research & Applications10.1145/3314111.3319820(1-10)Online publication date: 25-Jun-2019
      • (2019)Cross-Device TaxonomyProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300792(1-28)Online publication date: 2-May-2019
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media