Skip to main content
SpringerLink
Log in

Anomaly detection with a moving camera using multiscale video analysis

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

This paper addresses the problem of abandoned object detection in a cluttered environment using a camera moving along a straight track. The developed system compares captured images to a previously recorded reference video, thus requiring proper temporal alignment and geometric registration between the two signals. A real-time constraint is imposed onto the system to allow an effective surveillance capability in practical situations. In this paper, we propose to deal with the simultaneous detection of objects of different sizes using a multiresolution approach together with normalized cross-correlation and a voting step. In order to develop and properly assess the proposed method we designed a database recorded in a real surveillance scenario, consisting of an industrial plant containing a large number of pipes and rotating machines. Also, we have devised a systematic parameter tuning routine that allows the system to be adapted to different scenarios. We have validated it using the designed database. The obtained results are quite effective, achieving real-time, robust abandoned object detection in an industrial plant scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  • Chang, L., Zhao, H., Zhai, S., Ma, Y., & Liu, H. (2013). Robust abandoned object detection and analysis based on online learning. In International conference on robotics and biomimetics, Shenzhen, China.

  • Chen, Y. M., & Bajie, I. V. (2011). A joint approach to global motion estimation and motion segmentation from a coarsely sampled motion vector field. IEEE Transactions on Circuits and Systems for Video Technology, 21(9), 1316–1328.

    Article  Google Scholar 

  • Cheng, L., Gong, M., Schuurmans, D., & Caelli, T. (2011). Real-time discriminative background subtraction. IEEE Transactions on Image Processing, 20(5), 1401–1414.

    Article  MathSciNet  MATH  Google Scholar 

  • Choi, W., Pantofaru, C., & Savarese, S. (2013). A general framework for tracking multiple people from a moving camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1577–1591.

    Article  Google Scholar 

  • da Silva, A. F., Thomaz, L. A., Carvalho, G., Nakahata, M. T., Jardim, E., de Oliveira, J. F. L., et al. (2014). An annotated video database for abandoned-object detection in a cluttered environment. In International telecommunications symposium, Sao Paulo, Brazil.

  • da Silva, A. F., Thomaz, L. A., Netto, S. L., & da Silva, E. A. B. (2017). Online video-based sequence synchronization for moving camera object detection. In IEEE international workshop on multimedia signal processing (pp. 1–6). Luton, UK.

  • Davis, J. W., Morison, A. M., & Woods, D. D. (2007). An adaptive focus-of-attention model for video surveillance and monitoring. Machine Vision and Applications, 18(1), 41–64.

    Article  Google Scholar 

  • DeSouza, G. N., & Kak, A. C. (2002). Vision for mobile robot navigation: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2), 237–267.

    Article  Google Scholar 

  • Dore, A., Soto, M., & Regazzoni, C. S. (2010). Bayesian tracking for video analytics. IEEE Signal Processing Magazine, 27(5), 46–55.

    Article  Google Scholar 

  • Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010). Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1627–1645.

    Article  Google Scholar 

  • FLIR. (2016). Flir GF320. http://www.flir.com/ogi/display/?id=55671. Accessed January 14, 2016.

  • Ghosh, A., NSubudhi, B., & Ghosh, S. (2012). Object detection from videos captured by moving camera by fuzzy edge incorporated Markov random field and local histogram matching. IEEE Transactions on Circuits and Systems for Video Technology, 22(8), 1127–1135.

    Article  Google Scholar 

  • Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Hu, W. C., Chen, C. H., Chen, T. Y., Huang, D. Y., & Wu, Z. C. (2015). Moving object detection and tracking from video captured by moving camera. Journal of Visual Communication and Image Representation, 30, 164–180.

    Article  Google Scholar 

  • iRobot. (2016). iRobot Roomba vacuum cleaning robot. http://www.irobot.com/For-the-Home/Vacuuming/Roomba.aspx. Accessed February 26, 2018.

  • Jodoin, P. M., Saligrama, V., & Konrad, J. (2012). Behavior subtraction. IEEE Transactions on Image Processing, 21(9), 4244–4255.

    Article  MathSciNet  MATH  Google Scholar 

  • Kim, J., Ye, G., & Kim, D. (2010). Moving object detection under free-moving camera. In IEEE international conference on image processing, Hong Kong.

  • Kim, C., & Hwang, J. N. (2002). Fast and automatic video object segmentation and tracking for content-based applications. IEEE Transactions on Circuits and Systems for Video Technology, 12(2), 122–129.

    Article  Google Scholar 

  • Kim, S., Yun, K., Yi, K., Kim, S., & Choi, J. (2013). Detection of moving objects with a moving camera using non-panoramic background model. Machine Vision and Applications, 24(5), 1015–1028.

    Article  Google Scholar 

  • Kong, H., Audibert, J. Y., & Ponce, J. (2010). Detecting abandoned objects with a moving camera. IEEE Transactions on Image Processing, 19(8), 2201–2210.

    Article  MathSciNet  MATH  Google Scholar 

  • Kucharczak, F., da Silva, A. F., Thomaz, L. A., Carvalho, G., da Silva, E. A. B., & Netto, S. L. (2014). Comparison and optimization of image descriptors for real-time detection of abandoned objects. In Simpósio de Processamento de Sinais da UNICAMP, Campinas, Brazil. http://www.sps.fee.unicamp.br/anais/vol01/VSPS_a24_LThomaz.pdf.

  • Kundu, A., Jawahar, C. V., & Krishna, K. M. (2010). Realtime moving object detection from a freely moving monocular camera. In IEEE international conference on robotics and biomimetics (pp. 1635–1640). Tianjin, China.

  • Lee, S., Yun, I. D., & Lee, S. U. (2010). Robust bilayer video segmentation by adaptive propagation of global shape and local appearance. Journal of Visual Communication and Image Representation, 21(7), 665–676.

    Article  Google Scholar 

  • Lin, Y., Tong, Y., Cao, Y., Zhou, Y., & Wang, S. (2017). Visual-attention based background modeling for detecting infrequently moving objects. IEEE Transactions on Circuits and Systems for Video Technology, 27(6), 1208–1221.

    Article  Google Scholar 

  • Li, H., Tang, J., Wu, S., Zhang, Y., & Lin, S. (2010). Automatic detection and analysis of player action in moving background sports video sequences. IEEE Transactions on Circuits and Systems for Video Technology, 20(3), 351–364.

    Article  Google Scholar 

  • Menezes, P., Lerasle, F., & Dias, J. (2011). Towards human motion capture from a camera mounted on a mobile robot. Image and Vision Computing, 29(6), 382–393.

    Article  Google Scholar 

  • Micheloni, C., & Foresti, G. L. (2006). Real-time image processing for active monitoring of wide areas. Journal of Visual Communication and Image Representation, 17(3), 589–604.

    Article  Google Scholar 

  • Mukojima, H., Deguchi, D., Kawanishi, Y., & Ide, I. (2016). Moving camera background subtraction for obstacle detection on railway tracks. In International conference on image processing, Arizona, USA.

  • Nakahata, M. T., Thomaz, L. A., da Silva, A. F., da Silva, E. A. B., & Netto, S. L. (2017). Anomaly detection with a moving camera using spatio-temporal codebooks. Multidimensional Systems and Signal Processing. https://doi.org/10.1007/s11045-017-0486-8.

  • Nawaz, T., Poiesi, F., & Cavallaro, A. (2014). Measures of effective video tracking. IEEE Transactions on Image Processing, 23(1), 376–388.

    Article  MathSciNet  MATH  Google Scholar 

  • Nordlund, P., & Uhlin, T. (1996). Closing the loop: Detection and pursuit of a moving object by a moving observer. Image and Vision Computing, 14(4), 265–275.

    Article  Google Scholar 

  • Pinto, A. M., Correia, M. V., Moreira, A. P., & Costa, P. G. (2014). Unsupervised flow-based motion analysis for an autonomous moving system. Image and Vision Computing, 32(6–7), 391–404.

    Article  Google Scholar 

  • Romanoni, A., Matteucci, M., & Sorrenti, D. G. (2014). Background subtraction by combining temporal and spatio-temporal histograms in the presence of camera movement. Machine Vision and Applications, 25(6), 1573–1584.

    Article  Google Scholar 

  • Saligrama, V., Konrad, J., & Jodoin, P. M. (2010). Video anomaly identification. IEEE Signal Processing Magazine, 27(5), 18–33.

    Article  Google Scholar 

  • Soille, P. (2003). Morphological image analysis: Principles and applications (2nd ed.). Berlin: Springer.

    MATH  Google Scholar 

  • Subudhi, B. N., Nanda, P. K., & Ghosh, A. (2011). A change information based fast algorithm for video object detection and tracking. IEEE Transactions on Circuits and Systems for Video Technology, 21(7), 993–1004.

    Article  Google Scholar 

  • Suhr, J. K., Jung, H. G., Li, G., Noh, S. I., & Kim, J. (2011). Background compensation for pan-tilt-zoom cameras using 1-D feature matching and outlier rejection. IEEE Transactions on Circuits and Systems for Video Technology, 21(3), 371–377.

    Article  Google Scholar 

  • Sun, S. W., Wang, Y. C. F., Huang, F., & Liao, H. Y. M. (2013). Moving foreground object detection via robust SIFT trajectories. Journal of Visual Communication and Image Representation, 24(3), 232–243.

    Article  Google Scholar 

  • Taneja, A., Ballan, L., & Pollefeys, M. (2015). Geometric change detection in urban environments using images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(11), 2193–2206.

    Article  Google Scholar 

  • Tian, Y., Feris, R., Liu, H., Humpapur, A., & Sun, M. T. (2011). Robust detection of abandoned and removed objects in complex surveillance videos. IEEE Transactions on Systems, Man, and Cybernetics, 41(5), 565–576.

    Article  Google Scholar 

  • Tomioka, Y., Takara, A., & Kitazawa, H. (2012). Generation of an optimum patrol course for mobile surveillance camera. IEEE Transactions on Circuits and Systems for Video Technology, 22(2), 216–224.

    Article  Google Scholar 

  • VDAO. (2014). VDAO—Video database of abandoned objects in a cluttered industrial environment. http://www.smt.ufrj.br/~tvdigital/database/objects. Accessed February 26, 2018.

  • VDAO-200. (2017). 200-frame excerpts form VDAO database. http://www02.smt.ufrj.br/~tvdigital/database/research/. Accessed February 26, 2018.

  • Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013). Deppflow: Large displacement optical flow with deep matching. In International conference on computer vision, Sydney, Australia.

  • Xie, Y., Lin, L., & Jia, Y. (2010). Tracking objects with adaptive feature patches for PTZ camera visual surveillance. In International conference on pattern recognition (pp. 1739–1742). Istanbul, Turkey.

  • Xie, C., Tan, J., Chen, P., Zhang, J., & He, L. (2014). Multi-scale patch-based sparse appearance model for robust object tracking. Machine Vision and Applications, 25(7), 1859–1876.

    Article  Google Scholar 

  • Xue, K., Ogunmakin, G., Liu, Y., Vela, P. A., & Wang, Y. (2011). PTZ camera-based adaptive panoramic and multi-layered background model. In IEEE international conference on image processing, Brussels, Belgium.

  • Xue, K., Liu, Y., Ogunmakin, G., Chen, J., & Zhang, J. (2013). Panoramic Gaussian mixture model and large-scale range background substraction method for PTZ camera-based surveillance systems. Machine Vision and Applications, 24(3), 477–492.

    Article  Google Scholar 

  • Yilmaz, A. (2011). Kernel-based object tracking using asymmetric kernels with adaptive scale and orientation selection. Machine Vision and Applications, 22(2), 255–268.

    Article  Google Scholar 

Download references

Acknowledgements

This work was developed with the partial support of Statoil Brazil, Petrobras, ANP, and CAPES and CNPq funding agencies.

Funding

Funding was provided by Conselho Nacional de Desenvolvimento Científico e Tecnológico (Grant No. 203876/2017-2), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Grant No. 88881.135449/2016-01), Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro.

Author information

Authors and Affiliations

  1. Electrical Engineering Program, Federal University of Rio de Janeiro, PO Box 68504, Rio de Janeiro, RJ, 21941-970, Brazil

    Gustavo H. F. de Carvalho, Lucas A. Thomaz, Allan F. da Silva, Eduardo A. B. da Silva & Sergio L. Netto

Authors
  1. Gustavo H. F. de Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lucas A. Thomaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Allan F. da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eduardo A. B. da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sergio L. Netto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lucas A. Thomaz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Carvalho, G.H.F., Thomaz, L.A., da Silva, A.F. et al. Anomaly detection with a moving camera using multiscale video analysis. Multidim Syst Sign Process 30, 311–342 (2019). https://doi.org/10.1007/s11045-018-0558-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-018-0558-4

Keywords

Search

Navigation