Abstract
In this paper, we address the problem of estimating the 3-DOF attitude of a mobile device using measurements from a camera and an accelerometer; two sensors that are typically found on most mobile devices. In contrast to previous approaches that combine measurements from both the gyroscopes and the accelerometers of an inertial measurement unit (IMU) to estimate attitude, we restrict ourselves to the case of gyro-less devices, of which over one billion are currently in use mostly in developing countries. Furthermore, and in order to support virtual and augmented reality (VR/AR) applications on low-cost devices with limiting processing, we introduce an efficient and robust algorithm where attitude is first estimated locally over a sliding window of three keyframes and subsequently is integrated within an extended Kalman filter (EKF) to track the device’s global orientation. Additionally, gravity direction estimates, which are extracted from the accelerometer measurements when the device is not in motion, are used to update the roll and pitch attitude estimates, as well as the accelerometer’s biases, all of which, as we prove, are observable. The accuracy of the proposed method is assessed using the MAV EuRoC datasets [3], and it is shown to outperform alternative approaches relying on only visual data or the IMU, over a wide range of motions and conditions. Lastly, the efficiency of our algorithm is demonstrated on a Huawei 7A cell phone where it is able to run at 20 Hz on a single 1.4 GHz ARM Cortex A53 processor core.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Although many of these devices may be equipped with magnetometers, we do not consider them since they are susceptible to disturbances and bias from local and global magnetic distortion.
- 2.
In special cases, e.g., blurry images or dramatic changes in illumination that cause all feature tracks to drift, we extract ORB features on both images and perform brute-force matching [18].
- 3.
If the inlier size returned by the 2pt RANSAC is less than \(80\%\) of the input size, the static condition on the rotation angle is not considered satisfied.
- 4.
We employ the visual constraint (with \(\theta _s = 0.05\) deg) due to the fact that the accelerometer’s measurements are contaminated by noise and bias, which can make the static detection unreliable.
- 5.
For simplicity, we consider the accelerometer and camera frames to coincide. In practice, we determine the relative orientation, \({}^{A}\mathbf {q}_{\scriptscriptstyle {C_{}}}\), offline.
- 6.
In contrast to other sliding-window algorithms, in which the oldest keyframe is removed whenever a new keyframe is available, our method keeps accumulating keyframes and only uses the oldest 3 to create the structure.
- 7.
Even though our actual state comprises the global-in-local attitude, we chose to perform the analysis using the local-in-global attitude due to its simpler form of time derivatives. Note that this is without loss of generality, since there exists a bijective mapping \({}^{G}\mathbf {s}_C = -{}^{C}\mathbf {s}_G\).
- 8.
Note that we also compared our algorithm against the 2pt method of [11]. Its median error, however, even for the MH_02 dataset (easy) was over 75 deg; hence, we did not consider it further.
References
Bazin, J.-C., et al.: Globally optimal line clustering and vanishing point estimation in Manhattan world. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 638–645, Providence, RI, 16–21 June 2012
Bouguet, J.-Y., et al.: Pyramidal implementation of the affine Lucas Kanade feature tracker description of the algorithm. Intel Corporation 5(1–10), 4 (2000)
Burri, M., et al.: The EuRoC micro aerial vehicle datasets. Int. J. Robot. Res. 35(10), 1157–1163 (2016)
Craig, J.J.: Introduction to Robotics: Mechanics and Control. Pearson Prentice-Hall, Upper Saddle River (2005)
Do, T., Carrillo-Arce, L.C., Roumeliotis, S.I.: High-speed autonomous quadrotor navigation through visual and inertial paths. Int. J. Robot. Res. 38(4), 486–504 (2019)
Do, T., Neira, L., Roumeliotis, S.I.: Attitude tracking from a camera and an accelerometer on gyro-less devices (2019). https://mars.cs.umn.edu/tr/attitude_gyroless.pdf
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the KITTI vision benchmark suite. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361, Providence, RI, 16–21 June 2012
Hartmann, W., Havlena, M., Schindler, K.: Visual gyroscope for accurate orientation estimation. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 286–293, Waikoloa, HI, 5–9 January 2015
Hesch, J.A., Kottas, D.G., Bowman, S.L., Roumeliotis, S.I.: Camera-IMU-based localization: observability analysis and consistency improvement. Int. J. Robot. Res. 33(1), 182–201 (2014)
Horn, B.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)
Kamran, D., Karimian, M., Nazemipour, A., Manzuri, M.T.: Online visual gyroscope for autonomous cars. In: Proceedings of the 24th Iranian Conference on Electrical Engineering (ICEE), pp. 113–118, Shiraz, Iran, 10–12 May 2016
Ke, T., Roumeliotis, S.: An efficient algebraic solution to the perspective-three-point problem. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7225–7233, Honolulu, HI, 21–26 July 2017
Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR 2007, pp. 1–10, Nara, Japan, 13–16 November 2007
Lee, J.-K., Yoon, K.-J.: Real-time joint estimation of camera orientation and vanishing points. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1866–1874, Boston, 7–12 June 2015
Lefferts, E.J., Markley, F.L., Shuster, M.D.: Kalman filtering for spacecraft attitude estimation. J. Guidance Control Dyn. 5(5), 417–429 (1982)
Mahony, R., Hamel, T., Pflimlin, J.: Nonlinear complementary filters on the special orthogonal group. IEEE Trans. Autom. Control 53(5), 1203–1218 (2008)
Mirzaei, F.M., Roumeliotis, S.I.: Optimal estimation of vanishing points in a Manhattan world. In: Proceedings of the 2011 International Conference on Computer Vision, pp. 2454–2461, Barcelona, 6–13 November 2011
Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)
Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 430–443. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_34
Schmidt, S.F.: Application of state-space methods to navigation problems. In: Advances in Control System, vol. 3, pp. 293–340 (1966)
Stewenius, H., Engels, C., Nistér, D.: Recent developments on direct relative orientation. ISPRS J. Photogrammetry Remote Sens. 60(4), 284–294 (2006)
Trawny, N., Roumeliotis, S.I.: Indirect Kalman filter for 3D attitude estimation. Technical Report, January 2005
Acknowledgements
This work was supported by the University of Minnesota and the National Science Foundation (IIS-1328722).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Do, T., Neira, L., Yang, Y., Roumeliotis, S.I. (2022). Attitude Tracking from a Camera and an Accelerometer on Gyro-Less Devices. In: Asfour, T., Yoshida, E., Park, J., Christensen, H., Khatib, O. (eds) Robotics Research. ISRR 2019. Springer Proceedings in Advanced Robotics, vol 20. Springer, Cham. https://doi.org/10.1007/978-3-030-95459-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-95459-8_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95458-1
Online ISBN: 978-3-030-95459-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)