Skip to main content
SpringerLink
Log in

A flexible vehicle surround view camera system by central-around coordinate mapping model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The surround view camera system is an emerging driving assistant technology that can assist drivers in parking by providing top-down view of surrounding situations. Such a system usually consists of four wide-angle or fish-eye cameras that mounted around the vehicle, and a bird-eye view is synthesized from images of these cameras. Commonly there are two fundamental problems for the surround view synthesis, geometric alignment and image synthesis. Geometric alignment performs fish-eye calibration and computes the image perspective transformation between the bird-eye view and images from the surrounding cameras. Image synthesis technique dedicates to seamless stitch between adjacent views and color balancing. In this paper, we propose a flexible central-around coordinate mapping (CACM) model for vehicle surround view synthesis. The CACM model calculates perspective transformation between a top-view central camera coordinate and the around camera coordinates by a marker point based method. With the transformation matrices, we could generate the pixel point mapping relationship between the bird-eye view and images of the surrounding cameras. After geometric alignment, an image fusion method based on distance weighting is adopted for seamless stitch, and an effective overlapping region brightness optimization method is proposed for color balancing. Both the seamless stitch and color balancing can be easily operated by using two types of weight coefficient under the framework of the CACM model. Experimental results show that the proposed approaches could provide a high-performance surround view camera system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Baftiu I, Pajaziti A, Cheok KC (2016) Multi-mode surround view for ADAS vehicles. In: IEEE International symposium on robotics and intelligent sensors, pp 190–193

  2. Basu A, Licardie S (1995) Alternative models for fish-eye lenses. Pattern Recogn Lett 16:433–441

    Article  Google Scholar 

  3. Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/index.html

  4. Devernay F, Faugeras O (2010) Straight lines have to be straight. Mach Vis Appl 13(1):14–24

    Article  Google Scholar 

  5. Formentin S, Novara C, Savaresi SM (2015) Active braking control system design: the D2-IDC approach. IEEE/ASME Transactions on Mechatronics 20(4):1573–1584

    Article  Google Scholar 

  6. Gao Y, Liu C, Zhao Y, Wang X, Wei S, Huang Q (2017) 3-D surround view for advanced driver assistance systems. IEEE Transactions on Intelligent Transportations Systems 19(1):320–328

    Article  Google Scholar 

  7. Hedi A, Loncaric S (2012) A system for vehicle surround view. In: International Federation of Automatic Control Symposium on Robot Control, pp 120–125

    Article  Google Scholar 

  8. Kannala J, Brandt SS (2006) A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Trans Pattern Anal Mach Intell 28(8):1335–1340

    Article  Google Scholar 

  9. Lin CC, Wang MS (2012) A vision based top-view transformation model for a vehicle parking assistant. Sensors 12(4):4431–4446

    Article  Google Scholar 

  10. Liu Y, Zhang B (2014) Photometric alignment for surround view camera system. In: IEEE International Conference on Image Processing, pp 1827–1831

  11. Liu YC, Lin KY, Chen YS (2008) Bird’s-eye view vision system for vehicle surrounding monitoring. In: International Conference on Robot Vision, pp 207–218

  12. Lo WJ, Lin DT (2015) Embedded system implementation for vehicle around view monitoring. Advanced Concepts for Intelligent Vision Systems. Lecture Notes in Computer Science, pp 181–192

    Chapter  Google Scholar 

  13. Luo L, Chen J, An S, Wang C, Park J, Chong J (2012) Look-up table-based hardware implementation of bird’s-eye view system for camera mounted on vehicle. International Journal of Digital Content Technology and its application 6(15):143–154

    Article  Google Scholar 

  14. Natroshivili K, Scholl KU (2017) Automatic extrinsic calibration methods for surround view systems. IEEE Intelligent Vehicles Symposium. https://doi.org/10.1109/IVS.2017.7995702

  15. Szeliski R, Shum HY (1997) Creating full view panoramic image mosaics and environment maps. In: ACM SIGGRAPH, pp 251–258

  16. Szeliski R (2006) Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics and Computer Vision 2(1):1–104

    Article  MathSciNet  Google Scholar 

  17. Shum HY, Szeliski R (2000) Systems and experiment paper: construction of panoramic image mosaics with global and local alignment. Int J Comput Vis 36(2):101–130

    Article  Google Scholar 

  18. Tao D, Guo Y, Li Y, Gao X (2018) Tensor rank preserving discriminant analysis for facial recognition. IEEE Trans Image Process 27(1):325–334

    Article  MathSciNet  Google Scholar 

  19. Tao D, Guo Y, Yu B, Pang J, Yu Z (2017) Deep multi-view feature learning for person re-identification. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2017.2726580

    Article  Google Scholar 

  20. Tseng DC, Lin YC, Chao TW (2015) Wide-scoped surrounding top-view monitor for advanced driver assistance systems. International Conference on Mechatronics, Materials, Chemistry and Computer Engineering. https://doi.org/10.2991/icmmcce-15.2015.90

  21. Yeh YT, Peng CK, Chen KW, Chen YS, Hung YP (2014) Driver assistance system providing an intuitive perspective view of vehicle surrounding. In: Asian Conference on Computer Vision, pp 403–417

    Google Scholar 

  22. Yu M, Ma G (2014) 360° surround view system with parking guidance. Driver Assistance Systems 7(1):19–24

    Google Scholar 

  23. Zhang B, Appia V, Pekkucuksen I, Liu Y (2014) A surround view camera solution for embedded systems. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, pp 676–681

  24. Zhang Z (1999) Flexible camera calibration by view a plane from unknown orientations. In: IEEE International Conference on Computer Vision, pp 666–673

  25. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by NSFC (No. 61501177, No. 61772455, No. U1713213, No. 61762090), Guangzhou Key Laboratory (No. 201605030014), Guangzhou University’s Training Program for Excellent New-recruited Doctors (No. YB201712), the Yunnan Natural Science Funds under Grant 2016FB105 and 2016FA026, the Program for Excellent Young Talents of Yunnan University under Grant WX069051, and the Project of Innovative Research Team of Yunnan Province.

Author information

Authors and Affiliations

  1. School of Mechanical and Electric Engineering, Guangzhou University, Guangzhou, 510006, People’s Republic of China

    Zhao Yang & Xiao Hu

  2. School of Media & Information Engineering, Yunnan Open University, Kunming, 650023, People’s Republic of China

    Yang Zhao

  3. Kunming Shipborne Equipment Research and Test Center, Kunming, 650051, People’s Republic of China

    Yi Yin

  4. School of Information Science and Engineering, Yunnan University, Kunming, 650091, People’s Republic of China

    Lihua Zhou & Dapeng Tao

Authors
  1. Zhao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lihua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dapeng Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dapeng Tao.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

The expressions and corresponding descriptions that used in the CACM model.

Expression

Descriptions

Size

Unit

M, N

rectangular region around the vehicle

scalar

mm

m, n

resolution of the corresponding composite image

scalar

pixel

fCx, fCy

focal length of the central camera in pixels

scalar

pixel

uC0, vC0

principal point of the central camera

scalar

pixel

K C

internal parameter matrix of the central camera

3 × 3 matrix

–

fAx, fAy

focal length of a fish-eye camera camera in pixels

scalar

pixel

uC0, vC0

principal point of a fish-eye camera

scalar

pixel

[k1, k2, k3, k4]

distortion parameter of a fish-eye camera

4 × 1 vector

–

Rt

extrinsic parameter of a fish-eye camera

3 × 3 matrix

–

m = [i, j, 1]T

2D pixel point at location (i, j) in the composite image with homogeneous coordinate

3 × 1 vector

pixel

v

coordinate of m in the central camera coordinates system

3 × 1 vector

mm

f

coordinate of m in the right camera coordinates system

3 × 1 vector

mm

a, b

normalized coordinate of f

scalar

–

θ

incidence angle in the right camera

scalar

radian

r, r'

the distance between the image point and the principal point in the right camera.

scalar

mm

u,v

corresponding pixel coordinate of m in the original distorted image of the right fish-eye camera

scalar

pixel

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Z., Zhao, Y., Hu, X. et al. A flexible vehicle surround view camera system by central-around coordinate mapping model. Multimed Tools Appl 78, 11983–12006 (2019). https://doi.org/10.1007/s11042-018-6744-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6744-4

Keywords

Search

Navigation