skip to main content
10.1145/3571600.3571649acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvgipConference Proceedingsconference-collections
research-article

Insta Net: Recurrent Residual Network for Instagram Filter Removal✱

Published: 12 May 2023 Publication History

Abstract

Social media filters cause various manipulations in the original image. While these manipulations make the images visibly pleasant, they can affect their authenticity and the performance of further image-processing tasks. Therefore, removing these social media filters is a crucial pre-processing step for the data pipeline of any image-processing task. In this paper, a recurrent residual UNet-based model is proposed to remove the filters. The recurrent residual block allows the units of each layer to evolve over discrete time steps so that the activity of each unit is modulated by the activities of its neighboring units. Because of this characteristic, the networks learn contextual information more efficiently and effectively. Assuming that a filter can be noise, the proposed model learns the feature representation of the filter, and then the original, unfiltered image is retrieved by taking the residuals of the input image and the filter representation. The experimental results show that the proposed model outperforms both supervised and unsupervised methods by a considerable margin in the IFFI dataset.

References

[1]
Abdelrahman Abdelhamed, Mahmoud Afifi, Radu Timofte, and Michael S Brown. 2020. NTIRE 2020 challenge on real image denoising: Dataset, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 496–497.
[2]
Abdelrahman Abdelhamed, Stephen Lin, and Michael S Brown. 2018. A high-quality denoising dataset for smartphone cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1692–1700.
[3]
Abdelrahman Abdelhamed, Radu Timofte, and Michael S Brown. 2019. NTIRE 2019 challenge on real image denoising: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 0–0.
[4]
Md Zahangir Alom, Mahmudul Hasan, Chris Yakopcic, Tarek M Taha, and Vijayan K Asari. 2018. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv preprint arXiv:1802.06955(2018).
[5]
Md Zahangir Alom, Mahmudul Hasan, Chris Yakopcic, Tarek M Taha, and Vijayan K Asari. 2020. Improved inception-residual convolutional neural network for object recognition. Neural Computing and Applications 32, 1 (2020), 279–293.
[6]
Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. 2016. Fully-convolutional siamese networks for object tracking. In European conference on computer vision. Springer, 850–865.
[7]
Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft. 2016. Simple online and realtime tracking. In 2016 IEEE international conference on image processing (ICIP). IEEE, 3464–3468.
[8]
Simone Biancoa, Claudio Cusanob, Flavio Piccolia, and Raimondo Schettinia. [n.d.]. Artistic Photo Filter Removal Using CNNs. ([n. d.]).
[9]
Antoni Buades, Bartomeu Coll, and J-M Morel. 2005. A non-local algorithm for image denoising. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), Vol. 2. Ieee, 60–65.
[10]
Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun. 2022. Simple baselines for image restoration. arXiv preprint arXiv:2204.04676(2022).
[11]
Yu-Hsiu Chen, Ting-Hsuan Chao, Sheng-Yi Bai, Yen-Liang Lin, Wen-Chin Chen, and Winston H Hsu. 2015. Filter-invariant image classification on social media photos. In Proceedings of the 23rd ACM international conference on Multimedia. 855–858.
[12]
Wei-Ta Chu and Yu-Tzu Fan. 2019. Photo filter classification and filter recommendation without much manual labeling. In 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP). IEEE, 1–6.
[13]
Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. 2007. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on image processing 16, 8 (2007), 2080–2095.
[14]
Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, Kun Zhang, and Dacheng Tao. 2019. Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2427–2436.
[15]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[16]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.
[17]
Furkan Kinli, Baris Ozcan, and Furkan Kirac. 2021. Instagram filter removal on fashionable images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 736–745.
[18]
Furkan Kınlı, Barış Özcan, and Furkan Kıraç. 2022. Patch-wise Contrastive Style Learning for Instagram Filter Removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 578–588.
[19]
Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2018. Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE transactions on pattern analysis and machine intelligence 41, 11(2018), 2599–2613.
[20]
Hsin-Ying Lee, Hung-Yu Tseng, Qi Mao, Jia-Bin Huang, Yu-Ding Lu, Maneesh Singh, and Ming-Hsuan Yang. 2020. Drit++: Diverse image-to-image translation via disentangled representations. International Journal of Computer Vision 128, 10 (2020), 2402–2417.
[21]
Ming Liang and Xiaolin Hu. 2015. Recurrent convolutional neural network for object recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3367–3375.
[22]
Ming Liang and Xiaolin Hu. 2015. Recurrent convolutional neural network for object recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3367–3375.
[23]
Taesung Park, Alexei A Efros, Richard Zhang, and Jun-Yan Zhu. 2020. Contrastive learning for unpaired image-to-image translation. In European conference on computer vision. Springer, 319–345.
[24]
Tobias Plotz and Stefan Roth. 2017. Benchmarking denoising algorithms with real photographs. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1586–1595.
[25]
Yijun Quan, Xufeng Lin, and Chang-Tsun Li. 2018. Provenance analysis for instagram photos. In Australasian Conference on Data Mining. Springer, 372–383.
[26]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234–241.
[27]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234–241.
[28]
Oleksii Sidorov. 2019. Conditional gans for multi-illuminant color constancy: Revolution or yet another approach?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 0–0.
[29]
Jingwen Su, Boyan Xu, and Hujun Yin. 2022. A survey of deep learning approaches to image restoration. Neurocomputing 487(2022), 46–65.
[30]
Chunwei Tian, Lunke Fei, Wenxian Zheng, Yong Xu, Wangmeng Zuo, and Chia-Wen Lin. 2020. Deep learning on image denoising: An overview. Neural Networks 131(2020), 251–275.
[31]
Zhe Wu, Zuxuan Wu, Bharat Singh, and Larry Davis. 2020. Recognizing instagram filtered images with feature de-stylization. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 12418–12425.
[32]
Syed Waqas Zamir and Aditya etc Arora. 2020. Learning Enriched Features for Real Image Restoration and Enhancement. In Computer Vision – ECCV 2020. Springer International Publishing, Cham, 492–511.
[33]
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26, 7 (2017), 3142–3155.
[34]
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26, 7 (2017), 3142–3155.
[35]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.
[36]
Qiang Zuo, Songyu Chen, and Zhifang Wang. 2021. R2AU-Net: attention recurrent residual convolutional neural network for multimodal medical image segmentation. Security and Communication Networks 2021 (2021).

Cited By

View all
  • (2023)End-to-End Depth-Guided Relighting Using Lightweight Deep Learning-Based MethodJournal of Imaging10.3390/jimaging90901759:9(175)Online publication date: 28-Aug-2023

Index Terms

  1. Insta Net: Recurrent Residual Network for Instagram Filter Removal✱
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        ICVGIP '22: Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing
        December 2022
        506 pages
        ISBN:9781450398220
        DOI:10.1145/3571600
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 12 May 2023

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Convolutional Neural Networks
        2. Instagram filter removal
        3. Recurrent-Residual learning

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        ICVGIP'22

        Acceptance Rates

        Overall Acceptance Rate 95 of 286 submissions, 33%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)7
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 17 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2023)End-to-End Depth-Guided Relighting Using Lightweight Deep Learning-Based MethodJournal of Imaging10.3390/jimaging90901759:9(175)Online publication date: 28-Aug-2023

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media