skip to main content
10.1145/2933242.2933255acmconferencesArticle/Chapter ViewAbstractPublication PageseicsConference Proceedingsconference-collections
research-article

Statistical analysis and improvement of the repeatability and reproducibility of an evaluation method for IMUs on a smartphone

Published: 21 June 2016 Publication History

Abstract

An increasing number of mobile applications, like indoor-navigation or augmented reality, rely on highly accurate inertial sensor systems. However, there exists no standardized test for common inertial sensors, like accelerometers and gyroscopes to assess their accuracy. To easily determine the sensor system quality, a mobile sensor test application for smartphones was published recently. Although this test has several advantages, it lacks reproducibility and repeatability. As a remedy, we designed a special user interface that guides the operator through the test and helps judging the quality of the test conductance. While poorly conducted tests are neglected, only the meaningful ones are evaluated, so that our technology enables a more detailed analysis of the sensor systems. In this paper, we propose a statistical approach capable to distinguish between the four most important systematic errors we observed. Experiments with five state-of-the-art smartphones show that reproducibility and repeatability were significantly improved, enabling a fair comparison of the smartphone sensor systems.

References

[1]
2015. Repeatability and reproducibility in measurement systems. (2015). http://support.minitab.com/en-us/minitab/17/topic-library/quality-tools/measurement-system-analysis/gage-r-r-analyses/repeatability-and-reproducibility/.
[2]
2016. What is a gage R&R (repeatability and reproducibility) study? (2016). http://support.minitab.com/en-us/minitab/17/topic-library/quality-tools/measurement-system-analysis/gage-r-r-analyses/what-is-a-gage-r-r-study/.
[3]
R. Azuma. 1997. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 6, 4 (1997), 355--385.
[4]
N. Barbour and G. Schmidt. 2001. Inertial sensor technology trends. IEEE Sensors Journal 1, 4 (2001), 332--339.
[5]
R. Bogue. 2013. Recent developments in MEMS sensors: a review of applications, markets and technologies. Sensor Review 33, 4 (2013), 300--304.
[6]
P. L. Gribble, S. Everling, K. Ford, and A. Mattar. 2002. Hand-eye coordination for rapid pointing movements. Exp Brain Res (2002).
[7]
K. Hinckley, M. Sinclair, E. Hanson, R. Szeliski, and M. Conway. 1999. The videomouse: A camera-based multidegree-of-freedom input device. In 12th annual ACM symposium on User interface software and technology. ACM Press, 103--112.
[8]
Google Inc. 2015. Using CTS Verifier. (2015). https://source.android.com/compatibility/cts/verifier.html.
[9]
Google Inc. 2016. Game rotation vector. (2016). https://source.android.com/devices/sensors/sensor-types.html#game_rotation_vector.
[10]
F. Khoshnoud and C. de Silva. 2012. Recent advances in mems sensor technology-mechanical applications. In 6th IEEE and ACM International Symposium. IEEE, 14--24.
[11]
O. Korn, M. Funk, and A. Schmidt. 2015. Towards a Gamification of Industrial Production. A Comparative Study in Sheltered Work Environments. In 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems. ACM Press, 84--93.
[12]
F. Koushanfar, M. Potkonjak, and A. Sangiovanni-Vincentelli. 2004. Error models for light sensors by statistical analysis of raw sensor measurements. In Proceedings of IEEE Sensors. IEEE, 1472--1475.
[13]
F. Li, C. Zhao, G. Ding, J. Gong, C. Liu, and F. Zhao. 2012. A Reliable and Accurate Indoor Localization Method UsingPhone Inertial Sensors. In Conference on Ubiquitous Computing. ACM, 421--430.
[14]
L. Middendorf, R. Dorsch, R. Bichler, C. Strohrmann, and C. Haubelt. 2015. A Mobile Camera-Based Evaluation Method of Inertial Measurement Units on Smartphones. In International Conference on Sensor Systems and Software. EAI.
[15]
Q. Mourcou, A. Fleury, C Franco, F. Klopcic, and N. Vuillerme. 2015. Performance Evaluation of Smartphone Inertial Sensors Measurement for Range of Motion. Sensors (2015).
[16]
D. Schmalstieg and D. Wagner. 2007. Experiences with handheld augmented reality. In 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, 3--18.
[17]
A. Serra, D. Carboni, and V. Marotto. 2010. Indoor pedestrian navigation system using a modern smartphone. In 12th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM Press, 397--398.
[18]
W. Shi, J. Yang, Y. Jiang, F. Yang, and Y. Xiong. 2011. SenGuard: Passive User Identification on Smartphones Using Multiple Sensors. In 7th International Conference on Wireless and Mobile Computing, Networking and Communications. IEEE, 141--148.
[19]
L. Wang, Y. Hao, Z. Wei, and F. Wang. 2010. A Calibration Procedure and Testing of MEMS Inertial Sensors for an FPGA-based GPS/INS System. In International Conference on Mechatronics and Automation. IEEE, 1431--1436.
[20]
Z. Zhang. 2000. A flexible new technique for camera calibration. IEEE Pattern Analysis and Machine Intelligence 22, 11 (2000), 1330--1334.

Cited By

View all
  • (2021)On the Functional and Extra-Functional Properties of IMU Fusion Algorithms for Body-Worn Smart SensorsSensors10.3390/s2108274721:8(2747)Online publication date: 13-Apr-2021
  • (2019)Evaluation Method for the Absolute Orientation from the Rotation Vector on Mobile Devices2019 IEEE International Symposium on Inertial Sensors and Systems (INERTIAL)10.1109/ISISS.2019.8739713(1-4)Online publication date: Apr-2019

Index Terms

  1. Statistical analysis and improvement of the repeatability and reproducibility of an evaluation method for IMUs on a smartphone

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        EICS '16: Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems
        June 2016
        321 pages
        ISBN:9781450343220
        DOI:10.1145/2933242
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 21 June 2016

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. error classification
        2. guided movement
        3. inertial sensors
        4. repeatability and reproducibility
        5. sensor evaluation
        6. smartphone

        Qualifiers

        • Research-article

        Conference

        EICS'16
        Sponsor:

        Acceptance Rates

        Overall Acceptance Rate 73 of 299 submissions, 24%

        Upcoming Conference

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)6
        • Downloads (Last 6 weeks)2
        Reflects downloads up to 17 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2021)On the Functional and Extra-Functional Properties of IMU Fusion Algorithms for Body-Worn Smart SensorsSensors10.3390/s2108274721:8(2747)Online publication date: 13-Apr-2021
        • (2019)Evaluation Method for the Absolute Orientation from the Rotation Vector on Mobile Devices2019 IEEE International Symposium on Inertial Sensors and Systems (INERTIAL)10.1109/ISISS.2019.8739713(1-4)Online publication date: Apr-2019

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media