Skip to main content
SpringerLink
Log in

Stereo vision-based vehicle localization in point cloud maps using multiswarm particle swarm optimization

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

We propose a vision-based localization algorithm with multiswarm particle swarm optimization for driving an autonomous vehicle. With stereo vision, the vehicle can be localized within a 3D point cloud map using the particle swarm optimization. For vehicle localization, the GPS (global positioning system)-based algorithms are often affected by the certain conditions resulting in intermittent missing signal. We address this issue in vehicle localization by using stereo vision in addition to the GPS information. The depth-based localization is formulated as an optimization-based tracking problem. Virtual depth images generated from the point cloud are matched with the online stereo depth images using the particle swarm optimization. The virtual depth images are generated from the point cloud using a series of coordinate transforms. We propose a novel computationally efficient tracker, i.e., a multiswarm particle swarm optimization-based algorithm. The tracker is initialized with GPS information and employs a Kalman filter in the bootstrapping phase. The Kalman filter stabilizes the GPS information in this phase and, subsequently, initializes the online tracker. The proposed localization algorithm is validated with acquired datasets from driving tests. A detailed comparative and parametric analysis is conducted in the experiments. The experimental results demonstrate the effectiveness and robustness of the proposed algorithm for vehicle localization, which advances the state of the art for autonomous driving.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Aisan Technology Co Ltd: What’s MMS. http://www.whatmms.com/ (2013). Accessed 7 Nov 2018

  2. Aulinas, J., Petillot, Y., Salvi, J., Lladó, X.: The SLAM problem: a survey. In: Proceedings of the 11th International Conference of the Catalan Association for Artificial Intelligence, IOS Press, Amsterdam, The Netherlands, pp. 363–371 (2008)

  3. Bazeille, S., Battesti, E., Filliat, D.: Qualitative localization using vision and odometry for path following in topo-metric maps. In: Proceedings of the 5th European Conference on Mobile Robots, Örebro, Sweden, pp. 303–308 (2011)

  4. Cappelle, C., Pomorski, D., Yang, Y.: Gps/ins data fusion for land vehicle localization. In: The Proceedings of the Multiconference on “Computational Engineering in Systems Applications”, pp. 21–27 (2006)

  5. Colombo, O.: Ephemeris errors of GPS satellites. Bull. Godsique 60(1), 64–84 (1986)

    Article  Google Scholar 

  6. Cunha, J., Pedrosa, E., Cruz, C., Neves, A.J.R., Lau, N.: Using a depth camera for indoor robot localization and navigation. In: In Robotics Science and Systems (RSS) RGB-D Workshop, pp. 1–6 (2011)

  7. Dailey, M.N., Parnichkun, M.: Simultaneous localization and mapping with stereo vision. In: Ninth International Conference on Control, Automation, Robotics and Vision, Singapore, pp. 1–6 (2006)

  8. Deutscher, J., Blake, A., Reid, I.D.: Articulated body motion capture by annealed particle filtering. In: Conference on Computer Vision and Pattern Recognition, Hilton Head, USA, pp. 2126–2133 (2000)

  9. El Jaafari, I., El Ansari, M., Koutti, L.: Fast edge-based stereo matching approach for road applications. Signal Image Video Process. 11(2), 267–274 (2017)

    Article  Google Scholar 

  10. Franconi, L., Jennison, C.: Comparison of a genetic algorithm and simulated annealing in an application to statistical image reconstruction. Stat. Comput. 7(3), 193–207 (1997)

    Article  Google Scholar 

  11. Grewal, M.S., Weill, L.R., Andrews, A.P.: Global Positioning Systems, Inertial Navigation, and Integration. Wiley, New York (2007)

    Book  Google Scholar 

  12. Jia, B., Feng, W., Zhu, M.: Obstacle detection in single images with deep neural networks. Signal Image Video Process. 10(6), 1033–1040 (2016)

  13. John, V., Trucco, E., Ivekovic, S.: Markerless human articulated tracking using hierarchical particle swarm optimisation. Image Vis. Comput. 28(11), 1530–1547 (2010)

    Article  Google Scholar 

  14. John, V., Long, Q., Liu, Z., Mita, S.: Automatic calibration and registration of lidar and stereo camera without calibration objects. In: IEEE International Conference on Vehicular Electronics and Safety, ICVES 2015, Yokohama, Japan, November 5–7, 2015, pp. 231–237 (2015)

  15. John, V., Long, Q., Xu, Y., Liu, Z., Mita, S.: Registration of GPS and stereo vision for point cloud localization in intelligent vehicles using particle swarm optimization. In: Advances in Swarm Intelligence—8th International Conference, ICSI, Fukuoka, Japan, Proceedings, Part I, pp. 209–217 (2017a)

  16. John, V., Tsuchizawa, S., Liu, Z., Mita, S.: Fusion of thermal and visible cameras for the application of pedestrian detection. Signal Image Video Process. 11(3), 517–524 (2017b)

    Article  Google Scholar 

  17. John, V., Liu, Z., Mita, S., Guo, C., Kidono, K.: Real-time road surface and semantic lane estimation using deep features. Signal Image Video Process. 12(6), 1133–1140 (2018)

    Article  Google Scholar 

  18. Long, Q., Xie, Q., Mita, S., Tehrani, H., Ishimaru, K., Guo, C.: Real-time dense disparity estimation based on multi-path viterbi for intelligent vehicle applications. In: British Machine Vision Conference, BMVC 2014, Nottingham, UK, pp. 1–8 (2014)

  19. Maier, D., Hornung, A., Bennewitz, M.: Real-time navigation in 3d environments based on depth camera data. In: 12th IEEE-RAS International Conference on Humanoid Robots, Osaka, Japan, pp. 692–697 (2012)

  20. Mattern, N., Schubert, R., Wanielik, G.: High-accurate vehicle localization using digital maps and coherency images. In: IEEE Intelligent Vehicles Symposium, La Jolla, CA, USA, pp. 462–469 (2010)

  21. Niknejad, H.T., Takeuchi, A., Mita, S., McAllester, D.: On-road multivehicle tracking using deformable object model and particle filter with improved likelihood estimation. IEEE Trans. Intell. Transp. Syst. 13, 748–758 (2012)

    Article  Google Scholar 

  22. Nistér, D., Naroditsky, O., Bergen, J.R.: Visual odometry. In: 2004 Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp. 652–659 (2004)

  23. Oh, S.M., Tariq, S., Walker, B.N., Dellaert, F.: Map-based priors for localization. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, Japan, pp. 2179–2184 (2004)

  24. Panzieri, S., Pascucci, F., Ulivi, G.: An outdoor navigation system using GPS and inertial platform. IEEE/ASME Trans. Mechatron. 7(2), 134–142 (2002)

    Article  Google Scholar 

  25. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: International Conference on Evolutionary Computation, pp. 69–73 (1998)

  26. Yoneda, K., Tehrani, H., Ogawa, T., Hukuyama, N., Mita, S.: Lidar scan feature for localization with highly precise 3-d map. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, June 8–11, 2014, pp. 1345–1350 (2014)

Download references

Author information

Authors and Affiliations

  1. Toyota Technological Institute, Nagoya, Japan

    V. John, S. Mita & Y. Xu

  2. University of British Columbia, Vancouver, Canada

    Z. Liu

Authors
  1. V. John
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Z. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Mita
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Y. Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. John.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

John, V., Liu, Z., Mita, S. et al. Stereo vision-based vehicle localization in point cloud maps using multiswarm particle swarm optimization. SIViP 13, 805–812 (2019). https://doi.org/10.1007/s11760-019-01416-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01416-5

Keywords

Search

Navigation