Yilin Hu
ABSTRACT
The film-television industry continues to generate a large amount of information at all times, and the massive amount of movie data has promoted an increase in the demand for trailers. It has become a major research challenge to choose a movie of interest from the massive amount of movie data. This has driven the growth in demand for trailer production. Using computer technology to generate trailers automatically has two benefits: on the one hand, it can help people browse the content of a movie quickly and decide whether to pay for the movie; on the other hand, it can reduce the work of video creators and help them attract viewers with less cost. In this article, we construct a GCN-based convolution joint framework, which selects the trailer shots in the full-length movie according to the visual characteristics of the shots and the relationship between the shots. Firstly, the movie data is preprocessed for its division into sparse shots, followed by shot boundary detection and stratified sampling. Secondly, the visual features of shots are learned through multi-layer CNN. The topological relationship between shots is established by GCN to extract the features that include the shot relationship. These extracted features are then intelligently fused based on the assignment of different weights; shots with the fusion score higher than a certain threshold are selected for the trailer generation. The proposed framework is shown to other video summarization methods in the field of trailer generation.
- Tsai, C. M., Kang, L. W., Lin, C. W., & Lin, W. (2013). Scene-Based Movie Summarization Via Role-Community Networks. IEEE Transactions on Circuits and Systems for Video Technology, 23(11), 1927-1940Google Scholar
Digital Library
- Hesham, M., Hani, B., Fouad, N., & Amer, E. (2018Smart trailer: Automatic generation of movie trailer using only subtitles. Paper presented at the 2018 First International Workshop on Deep and Representation Learning (IWDRL).Google Scholar
- Papalampidi, P., Keller, F., & Lapata, M. (2020). Movie Summarization via Sparse Graph ConstructionGoogle Scholar
- Koutras, P., Zlatintsi, A., Iosif, E., Katsamanis, A., & Potamianos, A. (2015Predicting audio-visual salient events based on visual, audio and text modalities for movie summarization. Paper presented at the ICIP-2015, Quebec, Canada, 2015.Google Scholar
- Chung, Y. N., Lu, T. C., Yeh, M. T., Huang, Y. X., & Wu, C. Y. (2015). Applying the video summarization algorithm to surveillance systems. Journal of Image and Graphics, 3(1).Google ScholarCross Ref
- Yadav, A., & Vishwakarma, D. K. (2020). A unified framework of deep networks for genre classification using movie trailer - ScienceDirect. Applied Soft Computing, 96Google Scholar
- Liu, X., & Jiang, J. (2015Semi-supervised Learning Towards Computerized Generation of Movie Trailers. Paper presented at the 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC).Google Scholar
- Smith, J. R., Joshi, D., Huet, B., Hsu, W., & Cota, J. (2017Harnessing A.I. for Augmenting Creativity: Application to Movie Trailer Creation. Paper presented at the the 2017 ACM.Google Scholar
- Sheng, J., Chen, Y., Li, Y., & Liang, L. (2018). Embedded learning for computerized production of movie trailers. Multimedia Tools and Applications, 77(22), 29347-29365Google ScholarDigital Library
- Iuh, A., Km, B., Th, A., Jdsc, D., Ms, E.,... Swb, A. (2021). QuickLook: Movie Summarization using Scene-based Leading Characters with Psychological Cues Fusion. Information FusionGoogle Scholar
- Kip F, T. N., & Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional NetworksGoogle Scholar
- Wang, X., & Gupta, A. (2018). Videos as Space-Time Region Graphs: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part V: Computer Vision – ECCV 2018.Google Scholar
- Zeng, R., Huang, W., Gan, C., Tan, M., & Huang, J. (2019Graph Convolutional Networks for Temporal Action Localization. Paper presented at the 2019 IEEE/CVF International Conference on Computer Vision (ICCV).Google Scholar
- Liu, H., Xiao, Z., Fan, B., Zeng, H., & Jiang, G. (2021). PrGCN: Probability prediction with graph convolutional network for person re-identification. Neurocomputing, 423(12), 57-70Google ScholarCross Ref
- Neyman, J. (1934). On the Two Different Aspects of the Representative Method The Method of Stratified Sampling and the Method of Purposive Selection. Journal of the Royal Statistical Society, 97(4), 558-625Google ScholarCross Ref
- Liu, T., Fan, W., & Agrawal, G. (2010). Stratified Sampling for Data Mining on the Deep Web. IEEEGoogle ScholarDigital Library
- Mello, R. D., Silva, P., & Travassos, G. H. (2015). Investigating probabilistic sampling approaches for large-scale surveys in software engineering. Journal of Software Engineering Research and Development, 3(1), 8Google ScholarCross Ref
- Snyder, B., Jones, R., Bygrave, S., Llp, P. W., Bromwich, D.,... Mungello, D. (2013). Save the cat! : the last book on screenwriting you'll ever need: Save the cat! : the last book on screenwriting you'll ever need.Google Scholar
- Mccallum, A., Nigam, K., & Ungar, L. H. (2000)Efficient clustering of high-dimensional data sets with application to reference matching. Paper presented at the Proceedings of the Sixth International Conference on Knowledge Discovery and Data Mining.Google Scholar
- Hamerly, G., & Drake, J. (2015). Accelerating Lloyd's Algorithm for k-Means Clustering. Springer International PublishingGoogle Scholar
- Fajtl, J., Sokeh, H. S., Argyriou, V., Monekosso, D., & Remagnino, P. (2018Summarizing videos with attention. Paper presented at the Asian Conference on Computer Vision.Google Scholar
- Mahasseni, B., Lam, M., & Todorovic, S. (2017)Unsupervised video summarization with adversarial lstm networks. Paper presented at the Proceedings of the IEEE conference on Computer Vision and Pattern Recognition.Google Scholar
- Zhou, K., Qiao, Y., & Xiang, T. (2017). Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness RewardGoogle Scholar
- Zhang, K., Chao, W. L., Sha, F., & Grauman, K. (2016Video Summarization with Long Short-term Memory. Paper presented at the Springer International Publishing.Google Scholar
- Szegedy, C., Liu, W., Jia, Y., Sermanet, P., & Rabinovich, A. (2014). Going Deeper with Convolutions. IEEE Computer SocietyGoogle Scholar
- B. Castellano. (2018). Pyscenedetect: Intelligent scene cut detection and video splitting tool 2018.2, 2018, from https://pyscenedetect.readthedocs.io/en/latest/Google Scholar
- He, K., Zhang, X., Ren, S., & Sun, J. (2016Deep residual learning for image recognition. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.Google Scholar
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N.,... Polosukhin, I. (2017Attention is all you need. Paper presented at the Advances in neural information processing systems.Google Scholar
- Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal Loss for Dense Object Detection. IEEE Transactions on Pattern Analysis & Machine Intelligence, PP(99), 2999-3007Google Scholar
Recommendations
A Framework to Automate the generation of movies' trailers using only subtitles
ICSIE '18: Proceedings of the 7th International Conference on Software and Information EngineeringWith the rapidly increasing rate of user-generated videos over the World Wide Web, it becoming a high necessity for users to navigate through them efficiently. Video summarization is considered to be one of the promising and effective approach for ...
Automatic trailer generation
MM '10: Proceedings of the 18th ACM international conference on MultimediaThis paper presents a content-based movie trailer generation method, named Vid2Trailer (V2T). Since trailers are intended to advertise movies, they must show specific symbols such as the title logo and the main theme music. Moreover, it is expected to ...
Automatically selecting shots for action movie trailers
MIR '06: Proceedings of the 8th ACM international workshop on Multimedia information retrievalMovie trailers, or previews, are an important method of advertising movies. They are extensively shown before movies in cinemas, as well as on television and increasingly, over the Internet. Making a trailer is a creative process, in which a number of ...
Comments