Skip to main content
SpringerLink
Log in

CCTV-Calib: a toolbox to calibrate surveillance cameras around the globe

  • Research
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

CCTV camera calibration and geolocalization are essential for the deployment of smart-city applications. Despite the pressing need for practical and accurate surveillance camera calibration, most existing techniques are cumbersome and rely on restrictive assumptions on the camera’s or the scene’s geometry. In this context, we propose CCTV-Calib, a light and user-friendly toolbox to calibrate traffic cameras using satellite views. Unlike other surveillance camera calibration techniques, CCTV-Calib can estimate the intrinsic and extrinsic parameters as well as the GPS location of one or multiple CCTV cameras in a few clicks. For this purpose, we propose a simple yet effective two-stage pipeline to ensure a robust, repeatable, and easy calibration process. In the first stage, we perform automated keypoint matching between the CCTV image and the satellite views to compute an initial estimation of the camera’s parameters. In the second stage, this rough initialization guides a fine matching strategy, further improving calibration accuracy. This novel calibration pipeline is integrated into an easy-to-use GUI, making traffic camera calibration accessible to non-computer vision experts. To qualitatively and quantitatively evaluate the accuracy and relevance of our technique, we propose a novel dataset composed of synthetic and real images captured around the globe. Finally, in order to illustrate the pertinence of our calibration strategy, we demonstrate its applicability to 3D vehicle geolocalization. We made this toolbox and datasets freely available: https://github.com/rameau-fr/CCTV-Calib.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://www.google.com/earth/studio/.

  2. https://map.kakao.com/.

  3. www.youtube.com.

  4. Time square 3D vehicle detection video link: https://www.youtube.com/watch?v=KcbXZZcCmag.

  5. 3D car geolocalization KAIST video demo: https://www.youtube.com/watch?v=24Iw_wvUmjE &t=572s.

  6. 3D car geolocalization Netherland video demo: https://www.youtube.com/watch?v=V1n9MUZP7Ec.

References

  1. Agapito, L., Hayman, E., Reid, I.: Self-calibration of rotating and zooming cameras. Int. J. Comput. Vis. (IJCV) 45, 107–127 (2001)

    Article  MATH  Google Scholar 

  2. Alemán-Flores, M., Alvarez, L., Gomez, L., Santana-Cedrés, D.: Automatic lens distortion correction using one-parameter division models. Image Process. Line 4, 327–343 (2014)

    Article  Google Scholar 

  3. Altekar, N., Como, S., Lu, D., Wishart, J., Bruyere, D., Saleem, F., Head, K.L.: Infrastructure-based sensor data capture systems for measurement of operational safety assessment metrics. SAE Technical Papers (2021)

  4. Antunes, M., P Barreto, J., Aouada, D., Ottersten, B.: Unsupervised vanishing point detection and camera calibration from a single manhattan image with radial distortion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  5. Barreto, J.P.: A unifying geometric representation for central projection systems. Comput. Vis. Image Underst. (CVIU) 103, 208–217 (2006)

    Article  Google Scholar 

  6. Bartl, V., Juránek, R., Špaňhel, J., Herout, A.: PlaneCalib: automatic camera calibration by multiple observations of rigid objects on plane. In: Digital Image Computing: Techniques and Applications (DICTA). IEEE (2020)

  7. Bartl, V., Špaňhel, J., Dobeš, P., Juranek, R., Herout, A.: Automatic camera calibration by landmarks on rigid objects. Mach. Vis. Appl. (MVA) 32(1), 1–13 (2021)

    Google Scholar 

  8. Bhardwaj, R., Tummala, G.K., Ramalingam, G., Ramjee, R., Sinha, P.: AutocAlib: automatic traffic camera calibration at scale. ACM Trans. Sens. Netw. (TOSN) 14(3–4), 1–27 (2018)

    Google Scholar 

  9. Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., Cheng, M.M.: GMS: grid-based motion statistics for fast, ultra-robust feature correspondence (2017)

  10. Bogdan, O., Eckstein, V., Rameau, F., Bazin, J.C.: DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras. In: ACM SIGGRAPH European Conference on Visual Media Production (CVMP) (2018)

  11. Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2005)

  12. Bujnak, M., Kukelova, Z., Pajdla, T.: A general solution to the p4p problem for camera with unknown focal length. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)

  13. DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperPoint: self-supervised interest point detection and description. In: IEEE Conference on Computer Vision and Pattern Recognition—Workshop (CVPR-W) (2018)

  14. Deutscher, J., Isard, M., MacCormick, J.: Automatic camera calibration from a single manhattan image. In: European Conference on Computer Vision (ECCV) (2002)

  15. Duane, C.B.: Close-range camera calibration. Photogramm. Eng. 37(8), 855–866 (1971)

    Google Scholar 

  16. Dubská, M., Herout, A., Sochor, J.: Automatic camera calibration for traffic understanding. In: British Machine Vision Conference (BMVC), vol. 4. p. 8 (2014)

  17. Evangelidis, G.D., Psarakis, E.Z.: Parametric image alignment using enhanced correlation coefficient maximization. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 30(10), 1858–1865 (2008)

    Article  Google Scholar 

  18. Fitzgibbon, A.W.: Simultaneous linear estimation of multiple view geometry and lens distortion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2001)

  19. Gao, X.S., Hou, X.R., Tang, J., Cheng, H.F.: Complete solution classification for the perspective-three-point problem. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 25(8), 930–943 (2003)

    Article  Google Scholar 

  20. Ha, H., Rameau, F., Kweon, I.S.: 6-DOF direct homography tracking with extended Kalman filter. In: Pacific-Rim Symposium on Image and Video Technology (PSIVT) (2015)

  21. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision (2003)

  22. Hu, F., Ren, Y., Avadhanam, N., Pashiney, A.: System and method for optimal camera calibration, May 23. US Patent 11,657,535 (2023)

  23. Huang, S., Ying, X., Rong, J., Shang, Z., Zha, H.: Camera calibration from periodic motion of a pedestrian. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  24. Jeon, H.G., Im, S., Lee, B.U., Rameau, F., Choi, D.G., Oh, J., Kweon, I.S., Hebert, M.: A large-scale virtual dataset and egocentric localization for disaster responses. IEEE Tran. Pattern Anal. Mach. Intell. TPAMI 855, 87 (2021). https://doi.org/10.1109/TPAMI.2021.3094531

    Article  Google Scholar 

  25. Jocher, G., Stoken, A., Chaurasia, A., Borovec, J., TaoXie, NanoCode012, Kwon, Y., Michael, K., Changyu, L., Fang, J., Laughing, Abhiram V., yxNONG, tkianai, Skalski, P., Hogan, A., Nadar, J., Mammana, imyhxy, L., Fati, AlexWang1900, C., Montes, D., Hajek, J., Diaconu, L., Minh, M.T.: Albinxavi, Marc, Oleg, Fatih, Wanghaoyang0106.: ultralytics/yolov5: v6.0 - YOLOv5n ’Nano’ models, Roboflow integration, TensorFlow export, OpenCV DNN support (October 2021)

  26. Junejo, I., Foroosh, H.: Robust auto-calibration from pedestrians. In: IEEE International Conference on Video and Signal Based Surveillance (2006)

  27. Kannala, J., Brandt, S.S.: A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 28(8), 1335–1340 (2006)

    Article  Google Scholar 

  28. Krahnstoever, N., Mendonca, P.: Bayesian autocalibration for surveillance. In: IEEE International Conference on Computer Vision (ICCV) (2005)

  29. Kukelova, Z., Bujnak, M., Pajdla, T.: Real-time solution to the absolute pose problem with unknown radial distortion and focal length. In: IEEE International Conference on Computer Vision (ICCV) (2013)

  30. Kukelova, Z., Heller, J., Bujnak, M., Pajdla, T.: Radial distortion homography. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

  31. Kukelova, Z., Pajdla, T.: A minimal solution to the autocalibration of radial distortion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007)

  32. Kurdi, H.: Review of closed circuit television techniques for vehicles traffic management. Int. J. Comput. Sci. Inf. Technol. 6(2), 199 (2014)

    Google Scholar 

  33. Laugraud, B., Piérard, S., Braham, M., Van Droogenbroeck, M.: Simple median-based method for stationary background generation using background subtraction algorithms. In: International Conference on Image Analysis and Processing (2015)

  34. Li, C., Zeeshan Z., M, Tran, Q.H., Yu, X., D Hager, G., Chandraker, M.: Deep supervision with shape concepts for occlusion-aware 3D object parsing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  35. Li, C., Zia, M.Z., Tran, Q.H., Yu, X., Hager, G.D., Chandraker, M.: Deep supervision with intermediate concepts. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 41, 1828–1843 (2018)

    Article  Google Scholar 

  36. Li, S., Nguyen, V.H., Ma, M., Jin, C.B., Do, T.D., Kim, H.: A simplified nonlinear regression method for human height estimation in video surveillance. EURASIP J. Image Video Process. 2015(1), 1–9 (2015)

    Article  Google Scholar 

  37. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: European Conference on Computer Vision (ECCV) (2014)

  38. Liu, J., Collins, R.T., Liu, Y.: Surveillance camera autocalibration based on pedestrian height distributions. In: British Machine Vision Conference (BMVC) (2011)

  39. Liu, X., Liu, W., Ma, H., Fu, H.: Large-scale vehicle re-identification in urban surveillance videos. In: IEEE International Conference on Multimedia and Expo (ICME) (2016)

  40. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. (IJCV) 60(2), 91–110 (2004)

    Article  Google Scholar 

  41. Lu, D., C Jammula, V., Como, S., Wishart, J., Chen, Y., Yang, Y.: Carom–vehicle localization and traffic scene reconstruction from monocular cameras on road infrastructures. In: IEEE International Conference on Robotics and Automation (ICRA) (2021)

  42. Lv, F., Zhao, T., Nevatia, R.: Self-calibration of a camera from video of a walking human. In: IEEE International Conference on Pattern Recognition (ICPR) (2002)

  43. Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: MOT16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)

  44. Milosavljević, A., Rančić, D., Dimitrijević, A., Predić, B., Mihajlović, V.: A method for estimating surveillance video georeferences. ISPRS Int. J. Geo Inf. 6(7), 211 (2017)

    Article  Google Scholar 

  45. Morel, J.M., Asift, GYu.: A new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2(2), 438–469 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  46. Naphade, M., Wang, S., Anastasiu, D., Tang, Z., Chang, M.C., Yang, X., Yao, Y., Zheng, L., Chakraborty, P., Lopez, C., et al.: The 5th AI city challenge. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

  47. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: European Conference on Computer Vision (ECCV) (2016)

  48. Oskarsson, M.: A fast minimal solver for absolute camera pose with unknown focal length and radial distortion from four planar points. arXiv preprint arXiv:1805.10705 (2018)

  49. Pritts, J., Kukelova, Z., Larsson, V., Chum, O.: Radially-distorted conjugate translations. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

  50. Radosavovic, I., Kosaraju, R.P., Girshick, R., He, K., Dollár, P.: Designing network design spaces. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

  51. Rameau, F., Bailo, O., Park, J., Joo, K., Kweon, I.S.: Real-time multi-car localization and see-through system. Int. J. Comput. Vis. (IJCV) 130, 384–404 (2022)

    Article  Google Scholar 

  52. Rameau, F., Ha, H., Joo, K., Choi, J., Park, K., Kweon, I.S.: A real-time augmented reality system to see-through cars. IEEE Trans. Vis. Comput. Graph. (TVCG) 22(11), 2395–2404 (2016)

    Article  Google Scholar 

  53. Rameau, F., Habed, A., Demonceaux, C., Sidibé, D., Fofi, D.: Self-calibration of a PTZ camera using new LMI constraints. In: Asian Conference on Computer Vision (ACCV) (2013)

  54. Rameau, F., Park, J., Bailo, O., Kweon, I.S.: MC-Calib: a generic and robust calibration toolbox for multi-camera systems. Comput. Vis. Image Underst. (CVIU) 217, 103353 (2022)

    Article  Google Scholar 

  55. Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)

    Article  Google Scholar 

  56. Rezaei, M., Azarmi, M., Mohammad P.F.: Mir. Traffic-Net: 3D traffic monitoring using a single camera. arXiv preprint arXiv:2109.09165 (2021)

  57. Robusto, C.: The cosine-haversine formula. Am. Math. Mon. 64(1), 38–40 (1957)

    Article  MathSciNet  Google Scholar 

  58. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: IEEE International Conference on Computer Vision (ICCV) (2011)

  59. Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperGlue: learning feature matching with graph neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

  60. Schindler, G., Dellaert, F.: Atlanta world: an expectation maximization framework for simultaneous low-level edge grouping and camera calibration in complex man-made environments. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2004)

  61. Schoepflin, T.N., Dailey, D.J.: Dynamic camera calibration of roadside traffic management cameras for vehicle speed estimation. IEEE Trans. Intell. Transp. Syst. (TITS) 4(2), 90–98 (2003)

    Article  Google Scholar 

  62. Shao, Z., Li, C., Li, D., Altan, O., Zhang, L., Ding, L.: An accurate matching method for projecting vector data into surveillance video to monitor and protect cultivated land. ISPRS Int. J. Geo Inf. 9(7), 448 (2020)

    Article  Google Scholar 

  63. Shi, J., Tomasi, C.: Good features to track. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (1994)

  64. Sochor, J., Juránek, R., Herout, A.: Traffic surveillance camera calibration by 3D model bounding box alignment for accurate vehicle speed measurement. Comput. Vis. Image Underst. (CVIU) 161, 87–98 (2017)

    Article  Google Scholar 

  65. Sochor, J., Juránek, R., Španhel, J., Maršık, L., Širokỳ, A., Herout, A., Zemcık, P.: BrnoCompSpeed: review of traffic camera calibration and comprehensive dataset for monocular speed measurement. arXiv preprint arXiv:1702.06441, 3(5):6, (2017)

  66. Tang, Z., Lin, Y.S., Lee, K.H., Hwang, J.N., Chuang, J.H.: ESTHER: joint camera self-calibration and automatic radial distortion correction from tracking of walking humans. IEEE Access 7, 10754–10766 (2019)

    Article  Google Scholar 

  67. Tang, Z., Lin, Y.S., Lee, K.H., Hwang, J.N., Chuang, J.H., Fang, Z.: Camera self-calibration from tracking of moving persons. In: IEEE International Conference on Pattern Recognition (ICPR) (2016)

  68. Tang, Z., Wang, G., Xiao, H., Zheng, A., Hwang, J.N.: Single-camera and inter-camera vehicle tracking and 3D speed estimation based on fusion of visual and semantic features. In: IEEE Conference on Computer Vision and Pattern Recognition—Workshop (CVPR-W) (2018)

  69. Vanhoey, K., de Oliveira, C.E.P., Riemenschneider, H., Bódis-Szomorú, A., Manén, S., Paudel, D.P., Gygli, M., Kobyshev, N., Kroeger, T., Dai, D., et al.: Varcity-the video: the struggles and triumphs of leveraging fundamental research results in a graphics video production. In: ACM Transactions on Graphics (SIGGRAPH) (2017)

  70. Yang, M.Y., Cao, Y., Förstner, W., McDonald, J.: Robust wide baseline scene alignment based on 3D viewpoint normalization. In: International Symposium on Visual Computing (2010)

  71. Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J.: POI: multiple object tracking with high performance detection and appearance feature. In: European Conference on Computer Vision (ECCV) (2016)

  72. Zhang, C., Rameau, F., Kim, J., Argaw, D.M., Bazin, J.C., Kweon, I.S.: DeepPTZ: deep Self-Calibration for PTZ cameras. In: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1041–1049 (2020)

  73. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 22(11), 1330 (2000)

    Article  Google Scholar 

  74. Zhang, Z., Tang, J., Wu, G.: Simple and lightweight human pose estimation. arXiv preprint arXiv:1911.10346, (2019)

  75. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision (ICCV) (2017)

Download references

Acknowledgements

This research was supported by multiple grants. It was funded under the framework of the international cooperation program managed by the National Research Foundation of Korea (NRF-2020M3H8A1115028, FY2022). Additionally, it received support from the MSIT (Ministry of Science and ICT), Korea, under the Innovative Human Resource Development for Local Intellectualization support program (IITP-2023-RS-2022-00156287), supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation).

Author information

Authors and Affiliations

  1. Department of Computer Science, SUNY Korea, Songdo, Korea

    Francois Rameau

  2. Department of Electrical Engineering, KAIST, Daejeon, Korea

    Jaesung Choe, Fei Pan & In So Kweon

  3. School of Energy Engineering, KENTECH, Naju, Korea

    Seokju Lee

Authors
  1. Francois Rameau
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaesung Choe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fei Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seokju Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. In So Kweon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francois Rameau.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rameau, F., Choe, J., Pan, F. et al. CCTV-Calib: a toolbox to calibrate surveillance cameras around the globe. Machine Vision and Applications 34, 125 (2023). https://doi.org/10.1007/s00138-023-01476-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01476-1

Keywords

Search

Navigation