Skip to main content

Advertisement

SpringerLink
Log in

Video scene retrieval with interactive genetic algorithm

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a video scene retrieval algorithm based on emotion. First, abrupt/gradual shot boundaries are detected in the video clip of representing a specific story. Then, five video features such as “average color histogram,” “average brightness,” “average edge histogram,” “average shot duration,” and “gradual change rate” are extracted from each of the videos, and mapping through an interactive genetic algorithm is conducted between these features and the emotional space that a user has in mind. After the proposed algorithm selects the videos that contain the corresponding emotion from the initial population of videos, the feature vectors from them are regarded as chromosomes, and a genetic crossover is applied to those feature vectors. Next, new chromosomes after crossover and feature vectors in the database videos are compared based on a similarity function to obtain the most similar videos as solutions of the next generation. By iterating this process, a new population of videos that a user has in mind are retrieved. In order to show the validity of the proposed method, six example categories of “action,” “excitement,” “suspense,” “quietness,” “relaxation,” and “happiness” are used as emotions for experiments. This method of retrieval shows 70% of effectiveness on the average over 300 commercial videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bach JR, Fuller C, Gupta A, Hampapur A, Horowitz B, Humphrey R, Jain RC, Shu C (1996) The virage image search engine: an open framework for image management. In: Proc. SPIE, vol. 2670: Storage and Retrieval for Images and Video Databases IV, pp 76–86

  2. Banzhaf W (1997) Interactive evolution. Handbook of evolutionary computation. IOP, Oxford

  3. Biles JA (1994) GenJam: a genetic algorithm for generating jazz solos. In: Proc. Int. Computer Music Conf, pp 131–137

  4. Caldwell C, Johnston VS (1991) Tracking a criminal suspect through face–space with a genetic algorithm. In: Proc. Int. Conf. Genetic Algorithm, pp 416–421

  5. Carson C, Belongie S, Greenspan H, Malick J (2002) Blobworld: image segmentation using expectation–maximization and its application to image querying. IEEE Trans Pattern Anal Mach Intell 24(8):1026–1038

    Article  Google Scholar 

  6. Snoek Cees GM, Worring M (2005) Multimodal video indexing: a review of the state-of-the art. Multimedia Tools and Applications 25(1):5–35

    Article  Google Scholar 

  7. Cho S-B (2002) Towards creative evolutionary systems with interactive genetic algorithm. Appl Intell 16(2):129–138

    Article  MATH  Google Scholar 

  8. Colombo C, Del Bimbo A, Pala P (1999) Semantics in visual information retrieval. IEEE Multimed 6(3):38–53

    Article  Google Scholar 

  9. Colombo C, Del Bimbo A, Pala P (2001) Retrieval of commercials by semantic content: the semiotic perspective. Multimedia Tools and Applications 13(1):93–118

    Article  MATH  Google Scholar 

  10. Cox IJ, Miller ML, Minka TP, Papathomas TV, Yianilos PN (2000) The Bayesian image retrieval system, PicHunter: theory, implementation and psycophysical experiments. IEEE Trans Image Process 9(1):20–37

    Article  Google Scholar 

  11. Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image content: the QBIC system. IEEE Computer 28(9):23–31

    Google Scholar 

  12. Gargi U, Kasturi R, Strayer SH (2000) Performance Characterization of Video-shot-change detection methods. IEEE Trans Circuits Syst Video Technol 10(1):1–13

    Article  Google Scholar 

  13. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading, MA

    MATH  Google Scholar 

  14. Itten J (1961) Art of color (Kunst der Farbe). Otto Maier Verlag, Ravensburg, Germany (in German)

  15. Jain AK, Vailaya A, Xiong W (1999) Query by video clip. Multimedia Syst: Special Issue on Video Libraries 7(5):369–384

    Google Scholar 

  16. Joseph T, Cardenas A (1988) PicQuery: a high-level query language for pictorial database management. IEEE Trans Softw Eng 14(5):630–638

    Article  Google Scholar 

  17. Lee J.-Y, Cho S.-B (1998) Interactive genetic algorithm for content-based image retrieval. In: Proc. Asia Fuzzy Systems Symposium, pp 479–484

  18. Ma WY, Manjunath BS (1999) Netra: a toolbox for navigating large image databases. Multimedia Syst 7(3):184–198

    Article  Google Scholar 

  19. Minka TP, Picard RW (1997) Interactive learning using a society of models. Pattern Recogn 30(3):565–581

    Article  Google Scholar 

  20. Pentland A, Picard RW, Sclaroff S (1996) Photobook: content-based manipulation of image databases. Int J Comput Vis 18(3):233–254

    Article  Google Scholar 

  21. Pickens J, Bello JP, Monti G, Crawford T, Dovey M, Sandler M, Byrd D (2002) Polyphonic score retrieval using polyphonic audio queries: a harmonic modeling approach. In: Proc. ISMIR, pp 13–17

  22. Roussopolous N, Faloutsos C, Sellis T (1988) An efficient pictorial database system for pictorial structured query language (PSQL). IEEE Trans Softw Eng 14(5) 639–650

    Article  Google Scholar 

  23. Rui Y, Huang TS, Ortega M, Mehrota S (1998) Relevance feedback: a power tool in interactive content-based image retrieval. IEEE Trans Circuits Syst Video Technol 8(5):644–655

    Article  Google Scholar 

  24. Smith JR, Chang S-E (1996) VisualSEEK: a fully automated content-based image query system. In: Proc. ACM Multimedia, pp 87–98

  25. Soen T, Shimada T, Akita M (1987) Objective evaluation of color design. Color Res Appl 12(4):184–194

    Article  Google Scholar 

  26. Takagi H (2001) Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation. Proc IEEE 89(9):1275–1296

    Article  Google Scholar 

  27. Takagi H, Noda T, Cho S-B (1999) Psychological space to hold impression among media in common for media database retrieval system. In: Proc. IEEE Int. Conf. on System, Man, and Cybernetics, 263–268

  28. Toivanen J, Seppänen T (2002) Prosody-based search features in information retrieval. TMH-QPSR 44 Fonetik

  29. Truong BT, Dorai C, Venkatesh S (2000) New enhancements to cut, fade, and dissolve detection processes in video segmentation. In Proc. ACM Int. Conf. on Multimedia, pp 219–227

  30. Um J-S, Eum K-B, Lee J-W (2002) A study of the emotional evaluation models of color patterns based on the adaptive fuzzy system and the neural network. Color Res Appl 27(3):208–216

    Article  Google Scholar 

  31. Vailaya A, Figueiredo MAT, Jain AK, Zhang HJ (2001) Image classification for content-based indexing. IEEE Trans Image Process 10(1):117–130

    Article  MATH  Google Scholar 

  32. Vailaya A, Jain AK, Zhang HJ (1998) On image classification: city images vs. landscapes. Pattern Recogn 31(12):1921–1936

    Article  Google Scholar 

  33. Yeo BL, Liu B (1995) Rapid scene analysis on compressed video. IEEE Trans Circuits Syst Video Technol 5(6):533–544

    Article  Google Scholar 

  34. Yoo H.-W, Jang D-S (2004) Automated video segmentation using computer vision technique. International Journal of Information Technology and Decision Making 3(1):129–143

    Article  Google Scholar 

  35. Yoo H-W, Jang D-S, Jung S.-H, Park J-H, Song K-S (2002) Visual information retrieval system via content-based approach. Pattern Recogn 35(3):749–769

    Article  MATH  Google Scholar 

  36. Yoo H-W, Jung S-H, Jang D-S, Na Y-K (2002) Extraction of major object features using VQ clustering for content-based image retrieval. Pattern Recogn 35(5):1115–1126

    Article  MATH  Google Scholar 

  37. Zabih R, Miller J, Mai K (1999) A feature-based algorithm for detecting and classifying production effects. Multimedia Syst 7(2):119–128

    Article  Google Scholar 

  38. Zhang HJ, Kankanhalli A, Smoliar SW, Tan SY (1993) Automatic partitioning of full motion video. Multimedia Syst 1(1):10–28

    Article  Google Scholar 

  39. Zhang HJ, Wu J, Zhang D, Smoliar SW (1997) An integrated system for content-based video retrieval and browsing. Pattern Recogn 30(4):643–658

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Cognitive Science, Yonsei University, 134 Shinchon-Dong, Seodaemun-Ku, Seoul, 120-749, Korea

    Hun-Woo Yoo

  2. Department of Computer Science, Yonsei University, 134 Shinchon-Dong, Seodaemun-Ku, Seoul, 120-749, Korea

    Sung-Bae Cho

Authors
  1. Hun-Woo Yoo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sung-Bae Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hun-Woo Yoo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yoo, HW., Cho, SB. Video scene retrieval with interactive genetic algorithm. Multimed Tools Appl 34, 317–336 (2007). https://doi.org/10.1007/s11042-007-0109-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-007-0109-8

Keywords

Search

Navigation