Skip to main content

Advertisement

SpringerLink
Log in

A Smart multi-view panoramic imaging integrating stitching with geometric matrix relations among surveillance cameras (SMPI)

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Reducing data stored and transferred is a critical topic in the modern era, particularly after the evolution in multimedia applications and surveillance systems worldwide. Motivated by the massive amount of data generated by surveillance cameras and the enormous number of redundant pixels produced among them, this paper introduces a novel model entitled: “A Smart Multi-View Panoramic Imaging integrating stitching with geometric matrix relations among surveillance cameras (SMPI).” The introduced model aims to create a novel feedback real-time stitching system to reduce the storing and transferring of redundant data generated by neighboring surveillance cameras for an extra level of compression. Moreover, the panoramic view is mostly a better monitoring option rather than multiple monitors in complicated surveillance cameras’ control rooms. The proposed system, in this paper, merges feature extraction stitching techniques with geometric relational matrix calculations to reduce the time complexity limitations of traditional mosaicking. Additionally, the proposed work introduces a real-time algorithm to reconstruct images of each camera from the panoramic view, and a novel algorithm for ordering cameras’ frames before stitching is recommended for producing a panoramic view without any human interference. The experimental work tests numerous state of the art feature extraction algorithms for stitching, Scale Invariant Feature Transform (SIFT), Speed Up Robust Feature (SURF) and Oriented FAST and Rotated BRIEF (ORB) with different orders of stitching. The amount of compression per image after reconstruction is also analyzed. The suggested model was implemented and tested using a vast number of benchmark datasets. Evaluation measures have been used to indicate the efficiency of the recommended system. The proposed model’s algorithm has recorded a low time processing per frame while keeping high accurate results. It was found that the recommended Efficient Stitching Algorithm (ESA) produced an average of 46 panoramas per second, and the reconstruction phase could reach a rate of 90 frames per second, which is significantly higher than the 30 frames per second standard video format system. These results give our model an excellent advantage for the effective processing of more scalable systems with a higher number of frames per second. The proposed system created panoramas with an average of 99% similarities with the traditional mosaicking systems while being highly faster than these conventional methods. Compression ratios and data rate savings, reflecting the gain in data stored and transferred, were calculated, reporting an average of 2.66 and 0.62 per frame, respectively, when applied to standard datasets. The results illustrated that the proposed system gives a dramatic reduction in the volume of data stored/transferred and showed that the creating of mosaics and the reconstruction was made in proper processing time. Experimental outcomes also showed that, for the suggested methods, the produced frames after reconstruction have a high similarity percentage compared with original ones before stitching, which indicates that the proposed approach is efficient enough to preserve the essential features of cameras’ frames without significant information loss.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37
Fig. 38
Fig. 39
Fig. 40
Fig. 41
Fig. 42
Fig. 43
Fig. 44
Fig. 45
Fig. 46
Fig. 47
Fig. 48
Fig. 49
Fig. 50
Fig. 51
Fig. 52
Fig. 53
Fig. 54
Fig. 55
Fig. 56
Fig. 57
Fig. 58
Fig. 59
Fig. 60
Fig. 61
Fig. 62
Fig. 63
Fig. 64
Fig. 65
Fig. 66
Fig. 67
Fig. 68
Fig. 69
Fig. 70
Fig. 71
Fig. 72
Fig. 73
Fig. 74
Fig. 75
Fig. 76
Fig. 77
Fig. 78
Fig. 79
Fig. 80
Fig. 81
Fig. 82
Fig. 83
Fig. 84
Fig. 85
Fig. 86
Fig. 87
Fig. 88
Fig. 89
Fig. 90

Similar content being viewed by others

References

  1. Adobeadnin, https://sourceforge.net/adobe/adobedatasets/panoramas/home/Home/. Accessed Jan 2019

  2. Andrea V, https://andreame.com/2019/11/12/stitch.html. Accessed Apr 2020

  3. Bajpai P, Upadhyay A, Jana S, Kim J, Bandlamudi V (2018) High quality real-time panorama on mobile devices. In: IEEE international conference on multimedia & expo workshops (ICMEW)

  4. Bay H, Ess A, Tuytelaars T, Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359

    Google Scholar 

  5. Bergler M, Weiherer M, Bergen T, Avenhaus M, Rauber D, Wittenberg T, Benz M (2018) Stitching pathological tissue images using DOP feature tracking. Informatik Aktuell:322–327

  6. Bo X, Junwen W, Guangjie L, Yuewei D (2010) Image copy-move forgery detection based on SURF. In: International conference on multimedia information networking and security

  7. Brown M, Lowe DG (2003) Recognising panoramas. In: '03 proceedings of the ninth IEEE international conference on computer vision (ICCV) - volume 2, pp 1218–1225

  8. Brown M, Lowe D (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73

    Google Scholar 

  9. Burt P, Irani M, Hsu M, Anandan P, Hansen M (1999) Method and apparatus for performing mosaic based image compression. Sarnoff Corp. U.S. Patent 5, pp. 991,444

  10. Dissanayake V, Herath S, Rasnayaka S, Seneviratne S, Vidanaarachchi R, Gamage C (2015) Quantitative and qualitative evaluation of performance and robustness of image stitching algorithms. In: International conference on digital image computing: techniques and applications (DICTA), pp 1–6

  11. Ebtsam A, Elmogy M, El-Bakry H (2014) Image stitching based on feature extraction techniques: a survey. Int J Comput Appl 99:1–8

    Google Scholar 

  12. Fischler M, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    MathSciNet  Google Scholar 

  13. Girod B (1993) What’s wrong with mean-squared error. In: Digital images and human vision. the MIT press, pp 207–220

  14. Goncalves H, Corte-Real L, Goncalves J (2011) Automatic image registration through image segmentation and SIFT. IEEE Trans Geosci Remote Sens 49:2589–2600

    MATH  Google Scholar 

  15. Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: 20th international conference on pattern recognition, pp 2366–2369

  16. Juan L, Oubong G (2010) SURF applied in panorama image stitching. In: 2nd international conference on image processing theory, tools and applications

  17. Kale P, Singh K (2015) A technical analysis of image stitching algoritm. International Journal of Computer Science and Information Technologies 6(1):284–288

    Google Scholar 

  18. kushalvyas, https://github.com/kushalvyas/Python-Multiple-Image-Stitching. Accessed Mar 2019

  19. Li A, Zhou S, Wang R (2017) An improved method for eliminating ghosting in image stitching. In: 9th international conference on intelligent human-machine systems and cybernetics (IHMSC), pp 415–418

  20. Lin C, Tsai Y, Wang W, Chen L (2018) GPU-accelerated high-resolution image stitching with better initial guess. In: IEEE international conference on consumer electronics (ICCE)

  21. Liu L, Li Z, Delp EJ (2009) Efficient and low-complexity surveillance video compression using backward-channel aware Wyner–Ziv video coding. IEEE Trans Circuits Syst Video Technol 19(4):452–465

    Google Scholar 

  22. Liu Y, Yao J, Liu K, Lu X, Xia M (2016) Optimal image stitching for concrete bridge bottom surfaces aided by 3d structure lines. In: International archives of the photogrammetry, remote sensing & spatial information sciences, pp 527–534

  23. Luo B, Xu F, Richardt C, Yong J (2018) Parallax360: stereoscopic 360° scene representation for head-motion parallax. IEEE Trans Vis Comput Graph 24:1545–1553

    Google Scholar 

  24. Ma B, Ban X, Huang H, Liu W, Liu C, Wu D, Zhi Y (2019) A fast algorithm for material image sequential stitching. Computational Materials Science 158:1–13

    Google Scholar 

  25. Marr B (2015) Big data: using SMART big data, analytics and metrics to make better decisions and improve performance. Wiley, Chichester

    Google Scholar 

  26. Martens J-B, Meesters L (1998) Image dissimilarity. Signal Process 70:155–176

    MATH  Google Scholar 

  27. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Analysis Machine Intelligence 27:1615–1630

    Google Scholar 

  28. Natthavut V, Jareonpon C (2015) Multi image stitching with cylindrical surface base on local feature matching for solving the distortion problem. In: Proceedings of the 3rd IIAE international conference on intelligent systems and image processing, pp 106–113

  29. Outride.rs International Journalism Portal, https://outride.rs/en/temat/170-million-surveillance/. Accessed Apr 2019

  30. Pacot M, Marcos N (2018) Feature-based stitching algorithm of multiple overlapping images from unmanned aerial vehicle system. Asian Journal of Business and Technology

  31. Pan X, Lyu S (2010) Detecting image region duplication using SIFT features. In: IEEE international conference on acoustics, speech and signal processing, New York

  32. Pang Y, Li W, Yuan Y, Pan J (2011) Fully affine invariant SURF for image matching. Neurocomputing Journal 85:6–10

    Google Scholar 

  33. Pappas T, Safranek R (2000) Perceptual criteria for image quality evaluation. In: Bovik A (ed) Handbook of image and video proc. Academic, Orlando

    Google Scholar 

  34. Pedro L, Gilson G, Neves L (2010) Segmentation and feature extraction of panoramic dental X-Ray images, pp 1–15

  35. Pons A, Malo J, Artigas JM, Capilla P (1999) Image quality metric based on multidimensional contrast perception models. Displays 20:93–110

    Google Scholar 

  36. Qin Y, Xu H, Chen H (2014) Image feature points matching via improved ORB. In: IEEE international conference on progress in informatics and computing

  37. Qinglin H, Huaqiang W, Xuguang W, Nanyun L Stitching video streams captured by multi-UAVs with stabilization. In: Proceedings tenth international conference on graphics and image processing

  38. Ramaswamy A, Gubbi J, Raj R, Purushothaman B (2018) Frame stitching in indoor environment using drone captured images. In: 25th IEEE international conference on image processing (ICIP)

  39. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: IEEE international conference on computer vision, Barcelona (ICCV), pp 2564–2571

  40. Sampat M, Wang Z, Gupta S, Bovik A, Markey MK (2009) Complex wavelet structural similarity: a new image similarity index. IEEE Trans Image Process 18(11):2385–2401

    MathSciNet  MATH  Google Scholar 

  41. Sengupta A, Roy D (2018) Intellectual property-based lossless image compression for camera systems. IEEE Consumer Electronics Magazine 7(1):119–124

    Google Scholar 

  42. Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15:430–444

    Google Scholar 

  43. Su J, Cheng H, Yang L, Luo A (2016) Bayesian view synthesis for video stitching. In: IEEE international conference on Multimedia & Expo Workshops (ICMEW)

  44. Tareen S, Saleem Z (2018) A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. In: Computing, international conference on mathematics and engineering technologies (iCoMET), pp 1–10

  45. Tong Y, Zhang Q, Qi Y (2006) Image quality assessing by combining PSNR with SSIM. J Image and Graphics:1758–1763

  46. Wang Z, Bovik AC (2006) Modern image quality assessment. Morgan & Claypool, San Rafael

    Google Scholar 

  47. Wang Z, Simoncelli EP, Bovik AC (2003) Multi-scale structural similarity for image quality assessment. In: Proc. IEEE asilomar conf. signals, syst. comput., pp 1398–1402

  48. Wang B, Wei Z, Li Z, Hu W, Wang S, Zhang S, Du C, Shi W (2017) Region-based template matching method for multiview coastline image stitching. In: Proc 2017 8th IEEE Int. Conf. Software Eng. Service Sci. (ICSESS), pp 680–683

  49. Xiang T, Xia G, Bai X, Zhang L (2018) Image stitching by line-guided local warping with global similarity constraint. Pattern Recogn 83:481–497

    Google Scholar 

  50. Xiaoyong Y, Hao L, Yunfei D, Baopeng L, Xiaobin Y, Wei G, Zongxi S, Peiyun Z (2018) Image matching algorithm with color information based on SIFT. In: Proceedings volume 10806, tenth international conference on digital Image processing (ICDIP)

  51. Xie R, Yao J, Liu K, Lu X, Liu Y, Xia M, Zeng Q (2018) Automatic multi-image stitching for concrete bridge inspection by combining point and line features. Automation in Construction Journal 90:265–280

    Google Scholar 

  52. Yan W, Liu C, Luo W (2015) Fast and low complexity image stitching method on mobile phones. In: International conference on control, automation and information sciences (ICCAIS)

  53. Zaragoza J, Chin T, Brown M, Suter D (2013) As-projective-as-possible image stitching with moving DLT. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 2339–2346

  54. Zhuo L, Geng Z, Zhang J, Li X (2016) ORB feature based web pornographic image recognition. Neurocomputing Magazine 173:511–517

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Engineering, Computer Engineering Department, Arab Academy for Science, Technology and Maritime Transport, Alexandria, Egypt

    Sherin Youssef, Mohamed el Shehaby & Salema Fayed

Authors
  1. Sherin Youssef
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed el Shehaby
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salema Fayed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed el Shehaby.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Youssef, S., el Shehaby, M. & Fayed, S. A Smart multi-view panoramic imaging integrating stitching with geometric matrix relations among surveillance cameras (SMPI). Multimed Tools Appl 79, 30917–30981 (2020). https://doi.org/10.1007/s11042-020-09432-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09432-1

Keywords

Search

Navigation