skip to main content
10.1145/3573428.3573777acmotherconferencesArticle/Chapter ViewAbstractPublication PageseitceConference Proceedingsconference-collections
research-article

Automatic Scene Segmentation Algorithm for Image Color Restoration

Published: 15 March 2023 Publication History

Abstract

In the process of the automatic coloring of black and white movies, due to the issue of inaccurate scene segmentation, the selected reference image coloring is not suitable for two scenes, which further leads to poor coloring effects. Towards the issue, this paper designs an automatic scene segmentation algorithm based on deep learning, which can combine depth features and semantic similarities and not use the color distribution in the video. More specifically, the method first employs the pre-trained model VGG19 to learn the multi-layer feature representation of adjacent two frames. Secondly, the residual network is adopted to combine the multi-layer feature representations produced by the pre-trained model VGG19 to form feature vectors. Finally, the paper calculates the semantic similarity between the two feature vectors and designs an adaptive threshold scheme for determining the boundary frames, which can perform well in the scene segmentation task for various categories of videos. Experimental results show that this paper can effectively address the scene segmentation issue in various movies and thus improve the coloring effect of the ones.

References

[1]
P. Sidiropoulos, V. Mezaris, I. Kompatsiaris, "Temporal Video Segmentation to Scenes Using High-Level Audiovisual Features," IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 8, pp. 1163-1177, 2011.
[2]
S. Jianbo and J. Malik, "Normalized cuts and image segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, 2000.
[3]
I. U. Haq, K. Muhammad, T. Hussain, "Movie scene segmentation using object detection and set theory," International Journal of Distributed Sensor Networks, vol. 15, no. 6, p. 1550147719845277, 2019, 06, 01, 2019.
[4]
N. Kumar and N. Sukavanam, "Keyframes and Shot Boundaries: The Attributes of Scene Segmentation and Classification," in Harmony Search and Nature Inspired Optimization Algorithms, Singapore, N. Yadav, A. Yadav, J. C. Bansal, K. Deep, and J. H. Kim, Eds., 2019// 2019: Springer Singapore, pp. 771-782.
[5]
A. Rao, L. Xu, Y. Xiong, "A Local-to-Global Approach to Multi-Modal Movie Scene Segmentation," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June, 2020, 2020, pp. 10143-10152.
[6]
T. H. Trojahn and R. Goularte, "Temporal video scene segmentation using deep-learning," Multimedia Tools and Applications, vol. 80, no. 12, pp. 17487-17513, 2021, 05, 01, 2021.
[7]
K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," in International Conference on Learning Representations (ICLR), 2015.
[8]
K. He, X. Zhang, S. Ren, "Deep Residual Learning for Image Recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778.
[9]
H. Ji, D. Hooshyar, K. Kim, "A semantic-based video scene segmentation using a deep neural network," Journal of Information Science, vol. 45, no. 6, pp. 833-844, 2019, 12, 01, 2018.
[10]
J. Deng, W. Dong, R. Socher, "ImageNet: A large-scale hierarchical image database," in 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 20-25 June, 2009, 2009, pp. 248-255.
[11]
D. Ulyanov, A. Vedaldi, and V. S. Lempitsky, "Instance Normalization: The Missing Ingredient for Fast Stylization," ArXiv, vol. abs/1607.08022, 2016.
[12]
K. He, X. Zhang, S. Ren, "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification," in 2015 IEEE International Conference on Computer Vision (ICCV), 7-13 Dec. 2015, 2015, pp. 1026-1034.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering
October 2022
1999 pages
ISBN:9781450397148
DOI:10.1145/3573428
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 March 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Deep Learning
  2. Image Color Restoration
  3. Scene Segmentation
  4. Semantic Similarity

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

EITCE 2022

Acceptance Rates

Overall Acceptance Rate 508 of 972 submissions, 52%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 25
    Total Downloads
  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media