research-article

Sprite generation using sprite fusion

Authors:

Abhidnya A. Deshpande,

Ramazan S. AygüunAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 8, Issue 2

Article No.: 22, Pages 1 - 24

https://doi.org/10.1145/2168996.2169002

Published: 22 May 2012 Publication History

Abstract

There has been related research for sprite or mosaic generation for over 15 years. In this article, we try to understand the methodologies for sprite generation and identify what has not actually been covered for sprite generation. We first identify issues and focus on the domain of videos for sprite generation. We introduce a novel sprite fusion method that blends two sprites. Sprite fusion method produces good results for tracking videos and does not require object segmentation. We present sample results of our experiments.

References

[1]

Alzoubi, H. and Pan, W. D. 2008. Fast and accurate global motion estimation algorithm using pixel subsampling. Inf. Sci. 178, 17, 3415--3425.

Digital Library

[2]

Asif, M. and Soraghan, J. J. 2008. MPEG-7 Motion Descriptor Extraction for Panning Camera Using Sprite Generated. In Proceedings of the 5th International Conference on Advanced Video and Signal Based Surveillance. IEEE Computer Society, Los Alamitos, CA, 60--66.

Digital Library

[3]

Atrey, P. K., Hossain, M. A., Saddik A. E., and Kankahalli, M. S. 2010. Multimodal fusion for multimedia analysis: Survey. Multimedia Syst. 16, 345--379.

Digital Library

[4]

Aygün, R. S. and Zhang, A. 2002. Reducing blurring-effect in high resolution mosaic generation. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '02). IEEE Computer Society, Los Alamitos, CA, 537--540.

[5]

Aygün, R. S. and Zhang, A. 2004. Integrating virtual camera controls into digital video. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '04). IEEE Computer Society, Los Alamitos, CA,1503--1506.

[6]

Azzari, P., Di Stefano, L., and Bevilacqua, A. 2005. An effective real-time mosaicing algorithm apt to detect motion through background subtraction using a PTZ camera. In Proceedings of the IEEE Conference on Advanced Video and Signal-Based Surveillance. 511--516.

[7]

Chen, L., Lai, Y., and Liao, H. 2006. Video Scene Extraction Using Mosaic Technique. In Proceedings of the 18th International Conference on Pattern Recognition. IEEE Computer Society, Los Alamitos, CA, 723--726.

Digital Library

[8]

Chen, S. Y., Chen, C. Y., Huang, Y. W., and Chen, L. G. 2002. Multiple sprites and frame skipping techniques for sprite generation with high subjective quality and fast speed. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '02). IEEE Computer Society, Los Alamitos, CA, 785--788.

[9]

Chen, Y. and Aygün, R. S. 2010. Synthetic video generation for evaluation of the sprite generation. Int. J. Multimedia Data Engin. Manage. 2, 34--61.

Digital Library

[10]

Cherng, D.-C. and Chien S.-Y. 2007. Video Segmentation with model-based sprite generation for panning surveillance cameras. In Proceedings of the IEEE International Symposium on Circuits and Systems. IEEE Computer Society, Los Alamitos, CA, 27--30.

[11]

Cheung, H.-K. and Siu, W.-C. 2002. Fast global motion estimation for sprite generation. In Proceedings of the IEEE International Symposium on Circuits and Systems. IEEE Computer Society, Los Alamitos, CA.

[12]

Cheung, H.-K. and Siu, W.-C. 2007. Robust global motion estimation and novel updating strategy for sprite generation. IET Image Process. 1, 1, 13--20.

[13]

Cheung, H.-K., Siu, W.-C., and Feng, D. 2008. New block-based motion estimation for sequences with brightness variation and its application to static sprite generation for video compression. IEEE Trans Circ. Syst. Video Technol. 18, 522--527.

Digital Library

[14]

Coorg, S. and Teller, S. 2000. Spherical mosaics with quaternions and dense correlation. Int. J. Comput. Vision 37, 3, 259--273.

Digital Library

[15]

Dasu, A. R. and Panchanathan, S. 2004. A wavelet-based sprite codec. IEEE Trans Circ. Syst. Video Technol. 14, 2, 244--255.

Digital Library

[16]

Deshpande, A. and Aygün, R. S. 2009. Motion-based video classification for sprite generation. In Proceedings of the International Workshop on Database and Expert Systems Applications. 231--235.

Digital Library

[17]

Dufaux, F. and Konrad, J. 2000. Efficient, robust, and fast global motion estimation for video coding. IEEE Trans. Image Process. 9, 3, 497--501.

Digital Library

[18]

Farin, D. and de With, P. H. N. 2006. Enabling arbitrary rotational camera motion using multisprites with minimum coding cost. IEEE Trans. Circ. Syst. Video Technol. 16, 4, 492--506.

Digital Library

[19]

Fraunhaufer. 2009. http://www.iis.fraunhofer.de/amm/download/mpeg4.

[20]

Geys, H. and Van Gool, L. 2006. On-line, interactive view synthesis and augmentation. Signal Process.Image Comm. 21, 9, 709--723.

[21]

Grammalidis, N., Beletsiotis, D., and Strintzis, M. G. 1999. Multi View sprite generation and coding. InProceedings of the International Conference on Image Processing (ICIP '99). IEEE Computer Society, Los Alamitos, CA, 477--481.

Digital Library

[22]

H264. Iso/iec 14496-10:2003. information technology: Coding of audio-visual objects - part 2, also itu-t recommendation h.264 advanced video coding for generic audiovisual services.

[23]

H265. 2009. http://www.h265.net.

[24]

Hsu, C.-T. and Tsan Y.-C. 2004. Mosaics of video sequences with moving objects. Signal Process.Image Comm. 19, 1, 81--98.

[25]

Irani, M. and Anandan, P. 1998. Video indexing based on mosaic representations. Proc. IEEE, 905--921.

[26]

Krutz, A., Frater, M., Kunter, M., and Sikora, T. 2006. Windowed image registration for robust mosaicing of scenes with large background occlusions. In Proceedings of the International Conference on Image Processing (ICIP '06). 353--356.

[27]

Krutz, A., Glantz, A., Sikora, T., Nunes, P., and Pereira, F. 2008. Automatic object segmentation algorithms for sprite coding using MPEG-4. In Proceedings of the 50th International ELMAR Symposium. 459--462.

[28]

Krutz, A., Glantz, A., Haller, M., Droese, M., and Sikora, T. 2008. Multiple background sprite generation using camera motion characterization for object-based video coding. In Proceedings of the 5th International Conference on Advanced Video and Signal Based Surveillance. IEEE Computer Society, Los Alamitos, CA, 313--316.

[29]

Kunter, M., Krey, P., Krutz, A., and Sikora, T. 2008. Extending H.264/AVC with a background sprite prediction mode. In Proceedings of the International Conference on Image Processing (ICIP'08).

[30]

Lai, J., Kao, C., and Chien, S. 2009. Super-resolution sprite with foreground removal. In Proceedings of the IEEE International Conference on Multimedia and Expo.

Digital Library

[31]

Lee, M.-C., Chen, W.-G., Lin, C.-B., Chuang, G., Markoc, T., Zabinsky, S. I., and Szeliski, R. 1997. A layered video object coding system using sprite and affine motion model. IEEE Trans. Circ. Syst. Video Technol. 7, 1, 130--145.

Digital Library

[32]

Lu, Y., Gao, W., and Wu, F. 2001a. Fast and robust sprite generation for MPEG-4 video coding. In Proceedings of the 2nd IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia information Processing. H. Shum, M. Liao, and S. Chang, Eds., Lecture Notes In Computer Science, vol. 2195, Springer, 118--125.

Digital Library

[33]

Lu, Y., Gao, W., and Wu, F. 2001b. Sprite generation for frame-based video coding. In Proceedings of the International Conference on Image Processing. 473--476.

[34]

Lu, Y., Gao, W., and Wu, F. 2003. Efficient background video coding with static sprite generation and arbitrary-shape spatial prediction techniques. IEEE Trans. Circ. Syst. Video Technol. 13, 5, 394--405.

Digital Library

[35]

Marzotto, R., Fusiello, A., and Murino, V. 2004. High resolution video mosaicing with global alignment. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 692--698.

[36]

MPEG4 Software. Iso/iec 14496-7:2001. Information technology: Coding of audio-visual objects - part 7: Optimized software for mpeg-4 visual tools.

[37]

MPEG4-2, Iso/iec 14496-2:2004. Information technology: Coding of audio-visual objects: part 2.

[38]

Nagaraj, R. C., Dasu, A. R., and Panchanathan, S. 2001. Complexity analysis of sprites in MPEG. In Proceedings of SPIE, vol. 4313, 69--73.

[39]

Ostermann, J., Bormans, J., List, P., Marpe, D., Narroschke, M., Perreira, F., Stockhammer, T., and Wedi, T. 2004. Video coding with h.264/avc: tools, performance, and complexity. IEEE Circ. Syst. Mag. 4, 1, 7--28.

[40]

Parikh, P. and Jawahar, C. V. 2007. Enhanced video mosaicing using camera motion properties. In Proceedings of the IEEE Workshop on Motion and Video Computing. IEEE Computer Society, Los Alamitos, CA, 26.

Digital Library

[41]

Peleg, S., Rousso, B., Ravacha, A., and Zomet, A. 2000. Mosaicing on adaptive methods. IEEE Trans. Pattern Anal. Mach. Intell. 22, 10, 1144--1154.

Digital Library

[42]

Prodys. 2009. http://www.prodys.com/.

[43]

Richter, H., Smolic, A., Stabernack, B., and Müller, E., Real time global motion estimation for an MPEG-4 video encoder. In Proceedings of the Picture Coding Symposium.

[44]

Salembier, P., Pujol, O., and Garrido, L. 1998. Connected operators for sprite creation and layered representation of image sequences. In Proceedings of the European Signal Processing Conference. 2105--2108.

[45]

Shen, Y. and Zhang, L. 2004. A Novel Method of Sprite Generation Based on Pixel Similarity. In Proceedings of the 3rd International Conference on Image and Graphics. IEEE Computer Society, Los Alamitos, CA, 560-- 563.

Digital Library

[46]

Sikora, T. 1997. The MPEG-4 video standard verification model. IEEE Trans. Circ. Syst. Video Technol. 7, 19--31.

Digital Library

[47]

Smolic, A. and Ohm, J.-R. 2000. Robust global motion estimation using a simplified m-estimator approach. In Proceedings of the IEEE International Conference on Image Processing.

[48]

Smolic, A., Sikora, T., and Ohm, J.-R. 1999. Long-term global motion estimation and its application for sprite coding, content description and segmentation. IEEE Trans. Circ. Syst. Video Technol. 9, 8, 1227--1242.

Digital Library

[49]

Snoek, C. G., Worring, M., and Smeulders, A. W. 2005. Early versus late fusion in semantic video analysis. In Proceedings of the 13th Annual ACM International Conference on Multimedia (MULTIMEDIA '05). ACM, New York, NY, 399--402.

Digital Library

[50]

Steedly, D., Pal, C., and Szeliski, R. 2005. Efficiently registering video into panoramic mosaics. In Proceedings of the 10th International Conference on Computer Vision. 1300--1307.

Digital Library

[51]

Szeliski, R. 2006. Image alignment and stitching: a tutorial. Found. Trends. Comput. Graph. Vision 2, 1, 1--104.

Digital Library

[52]

Szeliski, R. and Shum, H.-Y. 1997. Creating full view panoramic image mosaics and environment maps. In Proceedings of ACM SIGGRAPH. 251--258.

Digital Library

[53]

Taubman, D. and Marcellin, M. 2002. JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer Academic Publishers, chapter 10.

Digital Library

[54]

Teodosio, L. and Bender, W. 1993. Salient video stills: content and context preserved. In Proceedings of the 1st ACM International Conference on Multimedia (MULTIMEDIA '93). ACM, New York, NY, 39--46.

Digital Library

[55]

Teodosio, L. and Bender, W. 2005. Salient stills. ACM Trans. Multimedia Comput. Comm. Appl. 1, 1, 16--36.

Digital Library

[56]

To, L. T. 2005. Video object segmentation using phase-based detection of moving object buondaries, Ph.D. thesis, University of New South Wales (2005).

[57]

Ye, G., Pickering, M., Frater, M., and Arnold, J. 2005. A robust approach to super-resolution sprite generation. In Proceedings of the IEEE International Conference on Image Processing. IEEE Computer Society, Los Alamitos, CA, 11--14.

[58]

Ye, G., Wang, Y., Xu, J., Herman, G., and Zhang, B. 2008. A practical approach to multiple super-resolution sprite generation. In Proceedings of the IEEE 10th Workshop on Multimedia Signal Processing. IEEE Computer Society, Los Alamitos, CA, 70--75, 8-10.

[59]

Zhu, Z., Xu, G., Riseman, E. M., and Hanson, A. R. 1999. Fast generation of dynamic and multi-resolution 360-degree panorama from video sequences. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems. IEEE Computer Society, Los Alamitos, CA.

Digital Library

[60]

Zoghlami, I., Faugeras, O., and Deriche, R. 1997. Using geometric corners to build a 2d mosaic from a set of images. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 420--425.

Digital Library

Cited By

Fagbemi AAygun RKarakaya MSahin N(2022)Video Alignment of Lecture Notes Missing Corners2022 IEEE 23rd International Conference on Information Reuse and Integration for Data Science (IRI)10.1109/IRI54793.2022.00064(262-269)Online publication date: Aug-2022
https://doi.org/10.1109/IRI54793.2022.00064
Chen YAygün R(2015)SpriteCamMultimedia Tools and Applications10.1007/s11042-013-1711-674:3(1067-1089)Online publication date: 1-Feb-2015
https://dl.acm.org/doi/10.1007/s11042-013-1711-6
Caywood REdwards JAygun R(2014)Object mosaicking: Reconstruction of moving objects captured through a limited viewIEEE SOUTHEASTCON 201410.1109/SECON.2014.6950741(1-6)Online publication date: Mar-2014
https://doi.org/10.1109/SECON.2014.6950741

Index Terms

Sprite generation using sprite fusion
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization

Recommendations

Sprite-from-Sprite: Cartoon Animation Decomposition with Self-supervised Sprite Estimation

We present an approach to decompose cartoon animation videos into a set of "sprites" --- the basic units of digital cartoons that depict the contents and transforms of each animated object. The sprites in real-world cartoons are unique: artists may draw ...
Synthetic Video Generation for Evaluation of Sprite Generation

Sprite generation is the process of aligning, warping, and blending of pixels that belong to an object in a video. The evaluation of the correctness of a sprite is usually accomplished by a combination of objective and subjective evaluations. ...
Fast and Robust Sprite Generation for MPEG-4 Video Coding
PCM '01: Proceedings of the Second IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing

This paper presents a fast and robust sprite generation algorithm for MPEG-4 video coding. Our contributions consist of two aspects. Firstly, a fast and robust Global Motion Estimation (GME) algorithm is proposed here. Spatial and temporal feature point ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 8, Issue 2

May 2012

144 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2168996

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 May 2012

Accepted: 01 February 2011

Revised: 01 September 2010

Received: 01 March 2010

Published in TOMM Volume 8, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
213
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fagbemi AAygun RKarakaya MSahin N(2022)Video Alignment of Lecture Notes Missing Corners2022 IEEE 23rd International Conference on Information Reuse and Integration for Data Science (IRI)10.1109/IRI54793.2022.00064(262-269)Online publication date: Aug-2022
https://doi.org/10.1109/IRI54793.2022.00064
Chen YAygün R(2015)SpriteCamMultimedia Tools and Applications10.1007/s11042-013-1711-674:3(1067-1089)Online publication date: 1-Feb-2015
https://dl.acm.org/doi/10.1007/s11042-013-1711-6
Caywood REdwards JAygun R(2014)Object mosaicking: Reconstruction of moving objects captured through a limited viewIEEE SOUTHEASTCON 201410.1109/SECON.2014.6950741(1-6)Online publication date: Mar-2014
https://doi.org/10.1109/SECON.2014.6950741

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents