Abstract
This paper presents Computational Cartoonist, a comic-style anime summarization system that detects key frame and generates comic layout automatically. In contract to previous studies, we define evaluation criteria based on the correspondence between anime films and original comics to determine whether the result of comic-style summarization is relevant. To detect key frame detection for anime films, the proposed system segments the input video into a series of basic temporal units, and computes frame importance using image characteristics such as motion. Subsequently, comic-style layouts are decided on the basis of pre-defined templates stored in a database. Several results demonstrate the efficiency of our key frame detection over previous methods by evaluating the matching accuracy between key frames and original comic panels.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Boreczky, J., Girgensohn, A., Golovchinsky, G., Uchihashi, S.: An interactive comic book presentation for exploring video. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 185–192. ACM (2000)
Calic, J., Gibson, D.P., Campbell, N.W.: Efficient layout of comic-like video summaries. IEEE Trans. Circ. Syst. Video Technol. 17(7), 931–936 (2007)
Cao, Y., Chan, A.B., Lau, R.W.: Automatic stylistic manga layout. ACM Trans. Graph. (TOG) 31(6), 141 (2012)
Cho, S.-B., Kim, K.-J., Hwang, K.-S.: Generating cartoon-style summary of daily life with multimedia mobile devices. In: Okuno, H.G., Ali, M. (eds.) IEA/AIE 2007. LNCS (LNAI), vol. 4570, pp. 135–144. Springer, Heidelberg (2007)
Farneback, G.: Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In: Eigth IEEE International Conference on Computer Vision, ICCV 2001, Proceedings, vol. 1, pp. 171–177. IEEE (2001)
Fukusato, T., Morishima, S.: Automatic depiction of onomatopoeia in animation considering physical phenomena. In: Proceedings of the Seventh International Conference on Motion in Games, pp. 161–169. ACM (2014)
Kasamatsu, S., Itoh, T.: A browser for summarized multiple videos. In: Proceedings of the 8th NICOGRAPH International (2009)
Levi, K., Weiss, Y.: Learning object detection from a small number of examples: the importance of good features. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II-53. IEEE (2004)
Lian, S., Dong, Y., Wang, H.: Efficient temporal segmentation for sports programs with special cases. In: Qiu, G., Lam, K.M., Kiya, H., Xue, X.-Y., Kuo, C.-C.J., Lew, M.S. (eds.) PCM 2010, Part I. LNCS, vol. 6297, pp. 381–391. Springer, Heidelberg (2010)
McCloud, S.: Making Comics: Storytelling Secrets of Comics, Manga and Graphic Novels Author. William Morrow Paperbacks, New York (2006)
Myodo, E., Ueno, S., Takagi, K., Sakazawa, S.: Automatic comic-like image layout system preserving image order and important regions. In: Proceedings of the 19th ACM International Conference on Multimedia. pp. 795–796. ACM (2011)
Ryu, D.S., Park, S.H., Lee, J.w., Lee, D.H., Cho, H.G.: Cinetoon: A semi-automated system for rendering black/white comic books from video streams. In: IEEE 8th International Conference on Computer and Information Technology Workshops, CIT Workshops 2008, pp. 336–341. IEEE (2008)
Shamir, A., Rubinstein, M., Levinboim, T.: Generating comics from 3d interactive computer graphics. IEEE Comput. Graph. Appl. 26(3), 53–61 (2006)
Taniguchi, Y., Akutsu, A., Tonomura, Y.: Panoramaexcerpts: extracting and packing panoramas for video browsing. In: Proceedings of the fifth ACM International Conference on Multimedia, pp. 427–436. ACM (1997)
Thawonmas, R., Shuda, T.: Comic layout for automatic comic generation from game log. In: Ciancarini, P., Nakatsu, R., Rauterberg, M., Roccetti, M. (eds.) New Frontiers for Entertainment Computing. IFIP International Federation for Information Processing, vol. 279, pp. 105–115. Springer, Heidelberg (2008)
Tobita, H.: Comic engine: interactive system for creating and browsing comic books with attention cuing. In: Proceedings of the International Conference on Advanced Visual Interfaces, pp. 281–288. ACM (2010)
Uchihashi, S., Foote, J.: Summarizing video using a shot importance measure and a frame-packing algorithm. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 6, pp. 3041–3044. IEEE (1999)
Uchihashi, S., Foote, J., Girgensohn, A., Boreczky, J.: Video manga: generating semantically meaningful video summaries. In: Proceedings of the Seventh ACM International Conference on Multimedia (Part 1), pp. 383–392. ACM (1999)
Wang, M., Hong, R., Yuan, X.T., Yan, S., Chua, T.S.: Movie2comics: towards a lively video content presentation. IEEE Trans. Multimedia 14(3), 858–870 (2012)
Zhuang, Y., Rui, Y., Huang, T.S., Mehrotra, S.: Adaptive key frame extraction using unsupervised clustering. In: 1998 International Conference on Image Processing, ICIP 1998, Proceedings, vol. 1, pp. 866–870. IEEE (1998)
Acknowledgements
This research is supported in part by OngaCREST, CREST, JST.and by Research Fellowship for Young Scientists of Japan Society for the Promotion of Science (JSPS).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Fukusato, T., Hirai, T., Kawamura, S., Morishima, S. (2016). Computational Cartoonist: A Comic-Style Video Summarization System for Anime Films. In: Tian, Q., Sebe, N., Qi, GJ., Huet, B., Hong, R., Liu, X. (eds) MultiMedia Modeling. MMM 2016. Lecture Notes in Computer Science(), vol 9516. Springer, Cham. https://doi.org/10.1007/978-3-319-27671-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-27671-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27670-0
Online ISBN: 978-3-319-27671-7
eBook Packages: Computer ScienceComputer Science (R0)