Skip to main content
Log in

@ICT: attention-based virtual content insertion

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

In this paper, we propose an attention-based virtual content insertion solution, called @ICT. Virtual content insertion (VCI) is an emerging application of video analysis and has been used in video augmentation and advertisement insertion. An ideal VCI solution should make the inserted virtual content being noticed by audiences and at the same time should not interfere with audiences’ viewing experience on the original content. To balance these two conflicting issues, meaning high attention and low intrusiveness, we choose higher attentive shots as insertion time while determine insertion place and content interdependently by considering lower attention together with visual consistency. We also propose a measurement of intrusiveness from the viewpoint of visual attention. Furthermore, @ICT includes an in-scene insertion module, which embeds the virtual content into the videos with higher vividness and lower intrusiveness. @ICT is able to obtain an optimal balance between the noticing of the virtual content by audiences and disruption of viewing experience to the original content. It needs little prior knowledge and is applied to general videos. Extensive quantitative and qualitative evaluations on the VCI result have verified the effectiveness of the solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Chang, C.-H., Hsieh, K.-Y., Chung, M.-C., Wu, J.-L.: ViSA: virtual spotlighted advertising. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 837–840 (2008)

  2. Chen, X., Yang, J., Zhang, J. Waibel, A. Automatic detection and recognition of signs from natural scenes. IEEE Trans. Image Process. 13(1), 87–99 (2004)

    Google Scholar 

  3. Deshpande, S., Naphade, P., Rao, C.V.K., Bhadada, K., Rangan, P.V.: Method and apparatus for including virtual ads in video presentations. US Patent, 7158666 (2007)

  4. Duan, L.-Y., Xu, M., Tian, Q., Xu, C.-S., Jin, J.S.: A Unified framework for semantic shot classification in sports video. IEEE Trans. Multimedia 7(6), 1066–1083 (2005)

    Article  Google Scholar 

  5. Dufaux, F., Konrad, J.: Efficient, robust, and fast global motion estimation for video coding. IEEE Trans. Image Process. 9(3), 497–501 (2000)

    Article  Google Scholar 

  6. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  7. Hu, Y., Rajan, D., Chia, L.-T.: Robust subspace analysis for detecting visual attention regions in images. In: Proceedings of the 13th ACM International Conference on Multimedia, pp. 716–724 (2005)

  8. Itti, L., Baldi, P.: A principled approach to detecting surprising events in video. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 631–637 (2005)

  9. Itti, L., Koch, C., Niebur, E.: “A model of saliency-based visual attention for rapid scene analysis”. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  10. Kreitman, H., Bar-El, D., Amir, Y., Tirosh, E.: Method and system for perspectively distorting an image and implanting same into a video stream. US Patent 5,731,846 (1998)

  11. Li, Y., Wah Wan, K., Yan, X., Xu, C.: Real time advertisement insertion in baseball video based on advertisement effect. In: Proceedings of the 13th ACM International Conference on Multimedia, pp. 343–346 (2005)

  12. Li, L., Mei, T., Hua, X.-S.: GameSense: game-like in-image advertising. Multimedia Tools Appl. 49(1), 145–166 (2010)

    Article  Google Scholar 

  13. Liu, C., Liu, H., Jiang, S., Huang, Q., Zheng Y., Zhang W.: JDL at Trecvid 2006 shot boundary detection. In: TRECVID 2006 Workshop

  14. Liu, H., Qiu, X., Huang, Q., Jiang, S., Xu, C.: Advertise gently—in-image advertising with low intrusiveness. In: Proceedings of the IEEE International Conference on Image Processing (2009)

  15. Liu, H., Jiang, S., Huang Q., Xu, C.: A generic virtual content insertion system based on visual attention analysis. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 379–388 (2008)

  16. Liu, H., Jiang, S., Huang Q., Xu, C.: Lower attentive region detection for virtual content insertion. In: Proceedings of the IEEE International Conference on Multimedia & Expo (2008)

  17. Li, H., Edwards, S.M., Lee, J.-H.: Measuring the intrusiveness of advertisements: scale development and validation. J. Advert. 31(2), 37–47 (2002)

    Google Scholar 

  18. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  19. Ma, Y.-F., Hua, X.-S., Lu, L., Zhang, H.-J.: A generic framework of user attention model and its application in video summarization. IEEE Trans. Multimedia 7(5), 907–919 (2005)

    Article  Google Scholar 

  20. Mei, T., Hua, X.-S., Li, S.: Contextual in-image advertising. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 439–448 (2008)

  21. Mei, T., Hua, X.-S., Yang, L., Li, S.: VideoSense-towards effective online video advertising. In: Proceedings of the 15th ACM International Conference on Multimedia, pp. 1075–1084 (2007)

  22. Mei, T., Guo, J., Hua, X.-S., Liu, F.: “AdOn: toward contextual overlay in-video advertising”. Multimedia Syst 16, 335–344 (2010)

    Article  Google Scholar 

  23. Meur, O.L., Callet, P.L., Barba, D., Thoreau, D.: A coherent computational approach to model bottom-up visual attention. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 802–816 (2006)

    Article  Google Scholar 

  24. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  MathSciNet  Google Scholar 

  25. Rolls, E.T., Aggelopoulos, N.C., Zheng, F.: Effective size of receptive fields of inferior temporal visual cortex neurons in natural scenes. J. Neurosci. 23(1), 339–348 (2003)

    Google Scholar 

  26. Rosser, R.J., Das, S., Tan, Y.: Method of tracking scene motion for live video insertion systems. US Patent 5,808,695 (1998)

  27. Sharir, A., Tamir, M., Wilf, I.: Method and apparatus for automatic electronic replacement of billboards in a video image. US Patent 6,384,871 (2002)

  28. Soodak, R.E.: “Two-dimensional modeling of visual receptive fields using Gaussian subunits”. Proc. Natl. Acad. Sci. USA 83, 9259–9263 (1986)

    Article  Google Scholar 

  29. Sun, Y., Fisher, R.B., Wang, F., Gomes, H.M.: A computer vision model for visual-object-based attention and eye movements. Comput. Vis. Image Underst. 112(2), 126–142 (2008)

    Article  Google Scholar 

  30. Wan, K., Xu, C.: Automatic content placement in sports highlights. In: Proceedings of the IEEE International Conference on Multimedia & Expo, pp. 1893–1896 (2006)

  31. Whitley, D.: A genetic algorithm tutorial. Stat. Comput. 4, 65–85 (1994)

    Article  Google Scholar 

  32. Xu, C., Wan, K.W., Bui, S.H., Tian, Q.: Implanting virtual advertisement into broadcast soccer video. In: Pacific-Rim Conference on Multimedia, pp. 264–271 (2004)

  33. Yu, X., Yan, X., Chi, T.T.P., Cheong, L.F.: Inserting 3D projected virtual content into broadcast tennis video. In: Proceedings of the 14th ACM International Conference on Multimedia, pp. 619–622 (2006)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Huiying Liu.

Additional information

Communicated by Thomas Haenselmann.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, H., Huang, Q., Xu, C. et al. @ICT: attention-based virtual content insertion. Multimedia Systems 18, 201–214 (2012). https://doi.org/10.1007/s00530-011-0234-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-011-0234-0

Keywords

Navigation