Skip to main content
SpringerLink
Log in

An Embedded Real-Time Surveillance System: Implementation and Evaluation

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

This paper presents the design of an embedded automated digital video surveillance system with real-time performance. Hardware accelerators for video segmentation, morphological operations, labeling and feature extraction are required to achieve the real-time performance while tracking will be handled in software in an embedded processor. By implementing a complete embedded system, bottlenecks in computational complexity and memory requirements can be identified and addressed. Accordingly, a memory reduction scheme for the video segmentation unit, reducing bandwidth with more than 70%, and a low complexity morphology architecture that only requires memory proportional to the input image width, have been developed. On a system level, it is shown that a labeling unit based on a contour tracing technique does not require unique labels, resulting in more than 50% memory reduction. The hardware accelerators provide the tracking software with image objects properties, i.e. features, thereby decoupling the tracking algorithm from the image stream. A prototype of the embedded system is running in real-time, 25 fps, on a field programmable gate array development board. Furthermore, the system scalability for higher image resolution is evaluated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. AXIS Communications, “Official AXIS Communications Homepage,” 2007, http://www.axis.com.

  2. Sony, “Official Sony Homepage,” 2007, http://bssc.sel.sony.com/.

  3. IBM, “Official IBM homepage,” 2007, http://www.research.ibm.com/peoplevision/.

  4. I. Haritaoglu, D. Harwood, and L. Davis, “W4: Real-time Surveillance of People and their Activities,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, 2000, pp. 809–830.

    Article  Google Scholar 

  5. C. Stauffer and W. E. L. Grimson, “Learning Patterns of Activity Using Real-time Tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, 2000, pp. 747–758.

    Article  Google Scholar 

  6. R. Collins, A. Lipton, H. Fujiyoshi, and T. Kanade, “Algorithms for Cooperative Multisensor Surveillance,” Proc. I.E.E.E., vol. 89, no. 10, 2001, pp. 1456–1477.

    Google Scholar 

  7. T. Zhao and R. Nevatia, “Tracking Multiple Humans in Complex Situations,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, 2004, pp. 1208–1222.

    Article  Google Scholar 

  8. W. Hu, T. Tan, L. Wang, and S. Maybank, “A Survey on Visual Surveillance of Object Motion and Behaviors,” IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., vol. 34, no. 3, 2004, pp. 334–353.

    Article  Google Scholar 

  9. R. Aguilar-Ponce, J. Tessier, A. Baker, C. Emmela, J. Das, J. Tecpanecatl-Xihuitl, A. Kumar, and M. Bayoumi, “VLSI Architecture for an Object Change Detector for Visual Sensors,” in Proceeding of IEEE Workshop on Signal Processing Systems Design and Implementation, SIPS, Athens, 2005, pp. 290–295.

  10. S. Fahmy, P. Cheung, and W. Luk, “Novel FPGA-based Implementation of Median and Weighted Median Filters for Image Processing,” in Field Programmable Logic and Applications, 2005. International Conference, 2005, pp. 142–147.

  11. P. Schumacher, K. Denolf, A. Chilira-RUs, R. Turney, N. Fedele, K. Vissers, and J. Bormans, “A Scalable, Multi-stream MPEG-4 Video Decoder for Conferencing and Surveillance Applications,” in Image Processing, 2005. ICIP 2005. IEEE International Conference, vol. II, 2006, pp. 886–889.

  12. R. Kordasiewicz and S. Shirani, “ASIC and FPGA Implementations of H.264 DCT and Quantization Blocks,” in Image Processing, 2005. ICIP 2005. IEEE International Conference, vol. III, 2006, pp. 1020–1023.

  13. R. Kleihorst, B. Schueler, A. Danilin, and M. Heijligers, “Smart Cameras Mote with High Performance Vision System,” in Workshop on Distributed Smart Cameras (DSC 06), Boulder, 2006.

  14. E. Ljung, E. Simmons, and R. Kleihorst, “Distributed Vision with Multiple Uncalibrated Smart Cameras,” in Workshop on Distributed Smart Cameras (DSC 06), Boulder, 2006.

  15. E. Ljung, E. Simmons, A. Danilin, R. Kleihorst, and B. Schueler, “802.15.4 Powered Distributed Wireless Smart Cameras Network,” in Workshop on Distributed Smart Cameras (DSC 06), Boulder, 2006.

  16. R. Gonzalez and R. Woods, Digital Image Processing, 2nd ed. Prentice-Hall, 2002.

  17. J. Toyama, B. Brumitt, and B. Meyers, “Wallflower: Principles and Practice of Background Maintenance,” in Proc. IEEE International Conference on Computer Vision and Pattern Recognition, 1999.

  18. C. Stauffer and W. E. L. Grimson, “Adaptive Background Mixture Models for Real-time Tracking,” in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’99), Ft. Collins, 1999.

  19. G. Russo and M. Russo, “A Novel Class of Sorting Networks,” IEEE Trans. Circuits Syst. 1 Fundam. Theory Appl., 1996.

  20. H. Jiang, H. Ardö, and V. Öwall, “Real-time Video Segmentation with VGA Resolution and Memory Bandwidth Reduction,” in Proc. of AVSS, 2006.

  21. F. Kristensen, P. Nilsson, and V. Öwall, “Background Segmentation Beyond RGB,” in Proc. of Asien Conference on Computer Vision (ACCV’06), Hyderabad, 2006.

  22. J. Serra, Image Analysis and Mathematical Morpohology, Vol 1. Academic Press, 1982.

  23. M. von Herk, “A Fast Algorithm for Local Minimum and Maximum Filters on Rectangular and Octagonal Kernels,” Pattern Recogn. Lett. vol. 13, no. 7, 1992, pp. 517–521.

  24. H. Hedberg, F. Kristensen, P. Nilsson, and V. Öwall, “A Low Complexity Architecture for Binary Image Erosion and Dilation Structuring Element Decomposition,” in Proc. of IEEE International Symposium on Circuits and Systems (ISCAS’05), Kobe, 2005.

  25. H. Park and R. Chin, “Decomposition of Arbitrarily Shaped Morphological Structuring Elements,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, 1995, pp. 2–15.

    Article  Google Scholar 

  26. G. Anelli and A. Broggi, “Decomposition of Arbitrarily Shaped Binary Morphological Structuring Elements Using Genetic Algorithms,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, 1998, pp. 217–224.

    Article  Google Scholar 

  27. S. Fejes and F. Vajda, “A Data-driven Algorithm and Systolic Architecture for Image Morphology,” in Proc. of IEEE Image Processing (ICIP ’94), Austin, 1994, pp. 550–554.

  28. J. Velten and A. Kummert, “FPGA-based implementation of variable sized structuring elements for {2D} binary morphological operations,” in Proc. of IEEE International Symposium on Circuits and Systems (ISCAS’03), Bangkok, 2003, pp. 706–709.

  29. A. Rosenfeld and J. Pfaltz, “Sequential Operations in Digital Picture Processing,” J. ACM, vol. 13, no. 1, 1966, pp. 471–494.

    MATH  Google Scholar 

  30. M. B. Dillencourt, H. Samet, and M. Tamminen, “A General Approach to Connected-component Labeling for Aritrary Image Representations,” J. ACM, vol. 39, no. 2, 1992, pp. 253–280.

    Article  MATH  MathSciNet  Google Scholar 

  31. W. Kesheng, O. Ekow, and S. Arie, “Optimizing Connected Components Labeling Algorithms,” in SPIE International Symposium on Medical Imaging, San Diego, 2005.

  32. K. Suzuki, H. Isao, and S. Noboru, “Linear-time Connected-component Labeling Based on Sequential Local Operations,” Comput. Vis. Image Underst., vol. 89, 2003, pp. 1–23.

    Article  MATH  Google Scholar 

  33. F. Chang, C. J. Chen, and C. J. Lu, “A Linear-time Component-labeling Algorithm Using Contour Tracing Technique,” Comput. Vis. Image Underst., vol. 93, 2004, pp. 206–220.

    Article  Google Scholar 

  34. PETS, “The PETS 2001 Data Set, Sequence from Camera 1,” 2001, http://www.cvg.cs.rdg.ac.uk/cgi-bin/PETSMETRICS/page.cgi?dataset.

  35. L. Yang and F. Algbregtsen, “Discrete Green’s theorem and its application in moment computation,” in Int. Conf. on Electronics and Information Technology, Beijing, 1994.

  36. H. Hedberg, F. Kristensen, and V. Öwall, “Implementation of Labeling Algorithm Based on Contour Tracing with Feature Extraction,” in Proc. of IEEE ISCAS’07, New Orleans, 2007.

  37. E. Brookner, Tracking and Kalman Filtering Made Easy, 1st ed. Wiley-Interscience, 1998.

  38. Lund University, “Project homepage,” 2007, http://www.es.lth.se/Digital_Surveillance.

  39. D. Magee, “Tracking Multiple Vehicles Using Foreground, Background and Motion Models,” Image Vis. Comput., vol. 22, 2004, pp. 143–155.

    Article  Google Scholar 

  40. JBIG Committee, “Official JBIG homepage,” 2007, http://www.jpeg.org/jbig/index.html.

Download references

Author information

Authors and Affiliations

  1. CCCD, Department of Electrical and Information Technology, Lund University, Box 118, 221 00, Lund, Sweden

    Fredrik Kristensen, Hugo Hedberg, Hongtu Jiang, Peter Nilsson & Viktor Öwall

Authors
  1. Fredrik Kristensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hugo Hedberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongtu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Nilsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Viktor Öwall
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fredrik Kristensen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kristensen, F., Hedberg, H., Jiang, H. et al. An Embedded Real-Time Surveillance System: Implementation and Evaluation. J Sign Process Syst Sign Image Video Technol 52, 75–94 (2008). https://doi.org/10.1007/s11265-007-0100-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-007-0100-7

Keywords

Search

Navigation