Skip to main content
Springer Nature Link
Log in

Adequation and hardware implementation of the color structure descriptor for real-time temporal video segmentation

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In this paper, we focused on the study of the MPEG-7 color structure descriptor (CSD) for real-time temporal video segmentation. A new hardware architecture of the CSD was proposed. In the aim of optimizing this implementation in terms of hardware resources and execution time, different algorithm transformations have been tested for the considered application. The CSD was applied for different quantization levels in the HMMD color space and for different grey levels, with and without a frame skip. We have demonstrated that the use of a low number of quantization levels and a frame skip can significantly reduce the complexity and assure a better computing performance while preserving a satisfactory level of accuracy in terms of shot boundary detection rate. This is useful for implementation on resource-constrained hardware platform and multiprocessing applications. The performances of the proposed architecture were evaluated for different quantization levels to show the effect on occupied hardware resources and execution time. The comparative study demonstrates the effectiveness of the proposed architecture, which can operate in real-time video and without restriction on image size. In this work we have also designed a system on chip (SOC) for real-time video summarization based on the CSD. The proposed SOC integrating the CSD module was developed using a platform based on a Xilinx Virtex5 FPGA. A complete demonstration, including CSD extraction, shot boundary detection and key-frames visualization, was realized.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Bailer, W., Höller, F., Messina, A., Airola, D., Schallauer, P., Hausenblas, M.: State of the art of content analysis tools for video, audio and speech. In: Report, FP6-IST-507336, PrestoSpace, Deliverable D15.3 MDS3 (2005)

  2. Smeaton, A.F., Over, P., Doherty, A.R.: Video shot boundary detection: seven years of TRECVid activity. Comput. Vis. Image Underst. 114(4), 411–418 (2010)

    Article  Google Scholar 

  3. Chang, S.F., Sikora, T., Puri, A.: Overview of the mpeg-7 standard. IEEE Trans. Circ. Syst. Video Technol. 11, 688–695 (2001)

    Article  Google Scholar 

  4. Manjunath, B.S., Ohm, J.R., Vasudevan, V.V., Yamada, A.: Color and texture descriptors. IEEE Trans. Circuits Syst Video Technol. 11(6), 703–715 (2001)

    Article  Google Scholar 

  5. Jeannin, S., Divakaran, A.: MPEG-7 visual motion descriptors. IEEE Trans.Circuits Syst. Video Technol. 11, 720–724 (2001)

    Article  Google Scholar 

  6. Lee, J.-H., Lee, G.-G., Kim, W.-Y.: Automatic Video Summarizing Tool using MPEG-7 Descriptors for personal Video Recorder. IEEE Trans. Consum. Electron. 49(3), (2003)

  7. Hoeynck M, Auweiler T, Ohm J-R: Application of MPEG-7 descriptors for content-based indexing of sports videos. In: Proceedings of the International Conference on Visual Communications and Image Processing (VCIP2003), 5150, pp. 1317–1328. Lugano, Switzerland (2003)

  8. Hahnel, M., Klunder, D., Kraiss, K.-F.: Color and texture features for person recognition. IEEE Int. Jt. Conf. Neural Netw. 1(25-29), 652 (2004)

    Google Scholar 

  9. Coimbra, M. T., Cunha, J. S.: MPEG-7 visual descriptors contributions for automated feature extraction in capsule endoscopy. IEEE Trans. Circuits Syst Video Technol. 16(5) (2006)

  10. Kim, W., Yoo, S.J., Kim, J.S., Nam, T.Y., & Yoon, K.: Detecting adult images using seven MPEG-7 visual descriptors. Web Commun Technol Internet Relat. Soc. Issues HSI 2005.3597/2005, 336–339 (2005)

  11. Anagnostopoulos, C., Alexandropoulos, T., Loumos, V., Kayafas, E.: Intelligent traffic management through MPEG-7 vehicle flow surveillance. Modern Computing JVA’06. IEEE John Vincent Atanasoff. In: International Symposium on, 202–207 (2006)

  12. Stechele, W.: Video processing using reconfigurable hardware acceleration for driver assistance. In: Workshop on Future Trends in Automotive Electronics and Tool Integration at DATE 2006. Munich (2006)

  13. Fonseca, P.M., Pereira, F.: Automatic video summarization based on MPEG-7 descriptions. Sig. Process. Image Commun. 19(8), 685–699 (2004)

    Article  Google Scholar 

  14. Nunez-Yanez, J.L., Nabina, A., Hung, E., Vafiadis, G.: Cogeneration of fast motion estimation processors and algorithms for advanced video coding. Very Larg. Scale Integr. Syst IEEE Trans. 20(3), 437–448 (2012)

    Article  Google Scholar 

  15. Jinglin Zhang, Nezan, J.-F., Cousin, J.-G.: Implementation of Motion Estimation Based on Heterogeneous Parallel Computing System with OpenCL. In: High performance computing and communication & 2012 IEEE 9th International Conference on Embedded Software and Systems (HPCC-ICESS), 2012 IEEE 14th International Conference on. 41–45 (2012)

  16. González, D., Botella, G., García, C., Prieto, M., Tirado, F.: Acceleration of block-matching algorithms using a custom instruction-based paradigm on a Nios II microprocessor. EURASIP J. Adv. Signal Process. 118 (2013)

  17. González, D., Botella, G., Meyer-Baese, U., García, C., Sanz, C., Prieto-Matías, M., Tirado, F.: A low cost matching motion estimation sensor based on the NIOS II microprocessor. Sensors 12, 13126–13149 (2012)

    Article  Google Scholar 

  18. Monteiro, E., Maule, M., Sampaio, F., Diniz, C., Zatt, B., Bampi. S.: Real-time block matching motion estimation onto GPGPU. In: Image Processing (ICIP), 19th IEEE International Conference on. 1693–1696 (2012)

  19. Botella, G., Meyer-Baese, U., García, A., Rodríguez, M.: Quantization analysis and enhancement of a VLSI gradient-based motion estimation architecture. Digit. Signal Proc. 22(6), 1174–1187 (2012)

    Article  MathSciNet  Google Scholar 

  20. Botella, G., Martín, H.J.A., Santos, M., Meyer-Baese, U.: FPGA-based multimodal embedded sensor system integrating low- and mid-level vision. Sensors 11, 8164–8179 (2011)

    Article  Google Scholar 

  21. Botella, G., Garcia, A., Rodriguez-Alvarez, M., Ros, E., Meyer-Baese, U., Molina, M.C.: Robust bioinspired architecture for optical-flow computation. Very Larg. Scale Integr. Syst. IEEE Trans. 18(4), 616–629 (2010)

    Article  Google Scholar 

  22. Hui, J., Liming. W.: Research on embedded vehicle image monitoring algorithms based on DSP. World Automation Congress (WAC), 1–3 (2012)

  23. Chang, J.Y., Lian, C.J., Chen, L.G.: Architecture and analysis of Color Structure and Scalable Color Descriptor for Real-Time Video Indexing and Retrieval. Adv. Multimed. Info. Process. PCM 2004, LNCS 3332. 130–137 (2004)

  24. Chang, J.Y., Fung, H.C., Hitang, Y.W., Chen, L.G.: Architecture of MPEG-7 color structure description generator for real-time video applications. In: Proc. of International Conference on Image Processing. 2813–2816 (2004)

  25. Abdelali, A.B., Mtibaa, A.: Toward hardware implementation of the compact color descriptor for real time video indexing. Adv. Eng. Softw. 36(7), 475–486 (2005)

    Article  Google Scholar 

  26. Touil, L., Ben Abdelali, A., Mtibaa, A.: in: A hardware acceleration of a real time video processing, 16th IEEE Mediterranean Electrotechnical Conference (MELECON 2012). Tunisia (2012)

  27. Sniatała, P., Kapela, R., Rudnicki, R., & Rybarczyk, A.: Efficient hardware architectures of selected mpeg-7 color descriptors. In: 15th European Signal Processing Conference EUSIPCO. Pozna n, Poland, 1672–1675 (2007)

  28. Kapela, R., Rybarczyk, A.: A real-time shape description system based on MPEG-7 descriptors. J. Syst. Archit. 53(9), 602–618 (2007)

    Article  Google Scholar 

  29. Forczmański, P., Dziurzański, P. (2013): System-Level Hardware Implementation of Simplified Low-Level Color Image Descriptor. In: Proceedings of the 8th International Conference on Computer Recognition Systems CORES. 461–468 (2013)

  30. Kapela, R., Śniatała, P., Rybarczyk, A.: Real-time visual content description system based on MPEG-7 descriptors. Multimed. Tools Appl. 53(1), 119–150 (2011)

    Article  Google Scholar 

  31. Eidenberger, H.: A video browsing application based on visual MPEG-7 descriptors and self-organising maps. Int. J. Fuzzy Syst. 6(3), 125–138 (2004)

    Google Scholar 

  32. Lee, H.K., Kim, C.S., Jung, Y.J., Nam, J.H., Kang, K.O., Ro, Y.M: Video contents summary using the combination of multiple MPEG-7 metadata. In: Electronic Imaging, pp. 1–12, International Society for Optics and Photonics (2002)

  33. A. Ben Abdelali, M. NidhalKrifa, L Touil, A Mtibaa, E Bourennane: A study of the color-structure descriptor for shot boundary detection, IJSTA (2009)

  34. Lu, Z., Shi, Y.: Fast Video Shot Boundary Detection Based on SVD and Pattern Matching. Image Process. IEEE Trans. 22(12), 5136–5145 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  35. Messing, D.S., van Beek, P., Errico, J.H.: The MPEG-7 colour structure descriptor: image description using colour and local spatial information. Int. Conf. Image Process. 1, 670–673 (2001). ISBN: 0-7803-6725-1

    Google Scholar 

  36. Kriechbaum, A., Bailer, W, Neuschmied H., Thallinger, G.: Using the MPEG-7 Colour Structure Descriptor for Human Identification in the POLYMNIA System. In: Proceedings of I-KNOW’06. Graz, Austria (2006)

  37. Singh, R. Sharma, N.: A New Pattern Recognition Approach Based on Color Structure and Image Euclidean Distances. In: IJCTA (2011)

  38. Shih, J.L., Chen, C.H.: Video Object Retrieval using Color Features. In: IMECS. 568–573 (2006)

  39. Lietz, H., Ritter, M., Manthey, R., Wanielik, G.: Improving pedestrian detection using MPEG-7 descriptors. Adv. Radio Sci. 11(8), 101–105 (2013)

    Article  Google Scholar 

  40. Kim, C. S., Ro, Y. M.: Semantic event detection using MPEG-7, Electronic Imaging. International Society for Optics and Photonics. 372–379 (2003)

  41. Ebrahimi, T., Abdeljaoued, Y., Figueras i Ventura, R.M., Divorra-Escoda, O.: MPEG-7 camera. Int. Conf. Image Process. 3, 600–603 (2001)

    Google Scholar 

  42. Yoo, S.J, Jung, M.h. Kang, H.B. Won, C.S. Choi, S.M.: Composition of MPEG-7 Visual Descriptors for Detecting Adult Images on the Internet, Lecture Notes in Computer Science, issu 2713. 682–687 (2003)

  43. Duda, K., Zielinski, T., Duplaga, M., Grega, M., Leszczuk, M.: Vq classification based on mpeg-7 visual descriptors for video endoscopic capsule localisation in the gastrointestinal tract, 15th European Signal Processing Conference (EUSIPCO 2007). Poznan, Poland (2007)

    Google Scholar 

  44. Camargo, L.H., Diaz, G., Romero, E.: Pap smear cell image classification using global MPEG-7 descriptors. Diagn. Pathol. 8(Suppl 1), S38 (2013)

    Article  Google Scholar 

  45. Molina, J., Spyrou, E., Sofou, N., Martínez, J. M.: On the selection of MPEG-7 visual descriptors and their level of detail for nature disaster video sequences classification. In: Proceedings of the 2nd international conference on Semantics And digital Media Technologies. Springer, Genova, Italy, 70–73 (2007)

  46. Touil, L., Ben Abdelali, A., Mibaa, A., Bourennane, E.: Towards Hardware implementation of video applications in new telecommunications devices. J. Telecommun. 2(1), 75–85 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of Electronics and Microelectronics, University of Monastir, Monastir, Tunisia

    Abdessalem Ben Abdelali, Marwa Hannachi, Lamjed Touil & Abdellatif Mtibaa

Authors
  1. Abdessalem Ben Abdelali
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Marwa Hannachi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Lamjed Touil
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Abdellatif Mtibaa
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Abdessalem Ben Abdelali.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ben Abdelali, A., Hannachi, M., Touil, L. et al. Adequation and hardware implementation of the color structure descriptor for real-time temporal video segmentation. J Real-Time Image Proc 13, 739–758 (2017). https://doi.org/10.1007/s11554-014-0438-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-014-0438-6

Keywords