Skip to main content
SpringerLink
Log in

Robust outdoor stereo vision SLAM for heavy machine rotation sensing

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The paper presents a robust outdoor stereo vision simultaneous localization and mapping (SLAM) algorithm. It estimates camera pose reliably in outdoor environments with directional sunlight illumination causing shadows and non-uniform scene lighting. The algorithm has been developed to measure a mining rope shovel’s rotation angle about its vertical axis (“swing” axis). A stereo camera is mounted externally to the shovel house (upper revolvable portion of the shovel), with a clear view of the shovel’s lower carbody. As the shovel house swings, the camera revolves with the shovel house in a planar circular orbit, seeing differing views of the carbody top. During the swing, the SLAM algorithm builds a map of observed 3D features on the carbody and simultaneously using these landmarks to estimate the camera position. This estimated camera position is then used to compute the shovel swing angle. Two novel techniques are employed to improve the SLAM algorithm’s robustness in outdoor environments. First, a “Locally Maximal” feature selection technique for Harris corners is used to select features more consistently in non-uniformly illuminated scenes. Another novel technique is the use of 3D “Feature Clusters” as SLAM landmarks rather than individual single features. The Feature Cluster landmarks improve the robustness of the landmark matching and allow significant reduction of the SLAM filter computational cost. This approach of estimating the shovel swing angle has a maximum error of ±1° upon SLAM map convergence. Results demonstrate the improvements of using the novel techniques compared to previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Borthwick, J., Lawrence, P., Hall, R.: Mining haul truck localization using stereo vision. In: Proceedings of the 14th IASTED International Conference on Robotics and Applications. Cambridge, Massachusetts (2009)

  2. Bradski G., Kaehler A.: Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, Inc., California (2008)

    Google Scholar 

  3. Davison A., Reid I., Molton N., Stasse O.: MonoSLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007). doi:10.1109/TPAMI.2007.1049

    Article  Google Scholar 

  4. Dissanayake M., Newman P., Clark S., Durrant-Whyte H., Csorba M.: A solution to the simultaneous localization and map building (SLAM) problem. IEEE Trans. Robotics Autom. 17(3), 229–241 (2001). doi:10.1109/70.938381

    Article  Google Scholar 

  5. Fua P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Mach. Vis. Appl. 6(1), 35–49 (1993). doi:10.1007/BF01212430

    Article  Google Scholar 

  6. Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, vol. 15, p. 50 (1988)

  7. Himmelman, N.P., Borthwick, J.R., Kashani, A.H., Lin, L., Poon, A., Hall, R.A., Lawrence, P.D., Salcudean, S.E., Owen, W.S.: Rope shovel collision avoidance system. In: Application of Computers and Operations Research in the Mineral Industry (2009)

  8. Jensfelt, P., Folkesson, J., Kragic, D., Christensen, H.: Exploiting distinguishable image features in robotic mapping and localization. In: European Robotics Symposium 2006, pp. 143–157. Springer, Berlin (2006)

  9. Johnson, A.E., Goldberg, S.B., Cheng, Y., Matthies, L.H.: Robust and efficient stereo feature tracking for visual odometry. In: IEEE International Conference on Robotics and Automation, 2008. ICRA 2008, pp. 39–46 (2008)

  10. Kaess, M.: Incremental smoothing and mapping (2008)

  11. Kashani, A., Owen, W., Lawrence, P., Hall, R.: Real-Time robot joint variable extraction from a laser scanner. In: IEEE International Conference on Automation and Logistics, pp. 2994–2999 (2007). doi:10.1109/ICAL.2007.4339095

  12. Konolige K., Agrawal M.: Frameslam: from bundle adjustment to real-time visual mapping. IEEE Trans. Robotics 24(5), 1066–1077 (2008)

    Article  Google Scholar 

  13. Lewis, J.P.: Fast normalized cross-correlation. In: Vision Interface, vol. 10, pp. 120–123 (1995)

  14. Lin, L.H., Lawrence, P.D., Hall, R.: Stereo vision based swing angle sensor for mining rope shovel. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1714–1721 (2010)

  15. Lowe D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  16. Mikolajczyk K., Schmid C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005). doi:10.1109/TPAMI.2005.188

    Article  Google Scholar 

  17. Montemerlo, M., Thrun, S., Koller, D., Wegbreit, B.: FastSLAM: a factored solution to the simultaneous localization and mapping problem. In: Proceedings of the National conference on Artificial Intelligence, pp. 593–598 (2002)

  18. Mozos, s., Gil, A., Ballesta, M., Reinoso, O.: Interest point detectors for visual SLAM. In: Current Topics in Artificial Intelligence, pp. 170–179. Springer, Berlin (2007)

  19. Saeedi P., Lawrence P., Lowe D.: Vision-based 3-D trajectory tracking for unknown environments. IEEE Trans. Robotics 22(1), 119–136 (2006). doi:10.1109/TRO.2005.858856

    Article  Google Scholar 

  20. Schmid C., Mohr R., Bauckhage C.: Evaluation of interest point detectors. Int. J. Comput. Vis. 37(2), 151–172 (2000). doi:10.1023/A:1008199403446

    Article  MATH  Google Scholar 

  21. Se S., Lowe D., Little J.: Mobile robot localization and mapping with uncertainty using Scale-Invariant visual landmarks. Int. J. Robotics Res. 21(8), 735–758 (2002). doi:10.1177/027836402761412467

    Article  Google Scholar 

  22. Smith R., Self M., Cheeseman P.: Estimating uncertain spatial relationships in robotics. Auton. Robot Veh. 1, 167–193 (1990)

    Article  Google Scholar 

  23. Smith S.M., Brady J.M.: SUSANAnew approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997). doi:10.1023/A:1007963824710

    Article  Google Scholar 

  24. Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, Massachusetts (2005). http://portal.acm.org/citation.cfm?id=1121596

  25. Zhu, Z., Oskiper, T., Samarasekera, S., Kumar, R., Sawhney, H.S.: Real-time global localization with a pre-built visual landmark database. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR, pp. 1–8 (2008)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada

    Li-Heng Lin, Peter D. Lawrence & Robert Hall

Authors
  1. Li-Heng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter D. Lawrence
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert Hall
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li-Heng Lin.

Electronic Supplementary Material

The Below is the Electronic Supplementary Material.

ESM 1 (AVI 4,116 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lin, LH., Lawrence, P.D. & Hall, R. Robust outdoor stereo vision SLAM for heavy machine rotation sensing. Machine Vision and Applications 24, 205–226 (2013). https://doi.org/10.1007/s00138-011-0380-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-011-0380-6

Keywords

Search

Navigation