skip to main content
research-article

Robust Facial Image Super-Resolution by Kernel Locality-Constrained Coupled-Layer Regression

Published: 09 June 2021 Publication History

Abstract

Super-resolution methods for facial image via representation learning scheme have become very effective methods due to their efficiency. The key problem for the super-resolution of facial image is to reveal the latent relationship between the low-resolution (LR) and the corresponding high-resolution (HR) training patch pairs. To simultaneously utilize the contextual information of the target position and the manifold structure of the primitive HR space, in this work, we design a robust context-patch facial image super-resolution scheme via a kernel locality-constrained coupled-layer regression (KLC2LR) scheme to obtain the desired HR version from the acquired LR image. Here, KLC2LR proposes to acquire contextual surrounding patches to represent the target patch and adds an HR layer constraint to compensate the detail information. Additionally, KLC2LR desires to acquire more high-frequency information by searching for nearest neighbors in the HR sample space. We also utilize kernel function to map features in original low-dimensional space into a high-dimensional one to obtain potential nonlinear characteristics. Our compared experiments in the noisy and noiseless cases have verified that our suggested methodology performs better than many existing predominant facial image super-resolution methods.

References

[1]
Simon Baker and Takeo Kanade. 2000. Hallucinating faces. In Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 83–88.
[2]
Simon Baker and Takeo Kanade. 2002. Limits on super-resolution and how to break them.IEEE Trans. Pattern Anal. Mach. Intell. 24, 9 (September 2002), 1167–1183.
[3]
Debarpan Bhattacharya, Sudip Misra, Nidhi Pathak, and Anandarup Mukherjee. 2020. IDeA: IoT-Based autonomous aerial demarcation and path planning for precision agriculture with UAVs.ACM Trans. Internet Things 1, 3 (June 2020), 1–21.
[4]
Ayan Chakrabarti, AN Rajagopalan, and Rama Chellappa. 2007. Super-resolution of face images using kernel PCA-based prior.IEEE Trans. Multimedia 9, 4 (June 2007), 888–892.
[5]
Berk Dogan, Shuhang Gu, and Radu Timofte. 2019. Exemplar guided face image super-resolution without facial landmarks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1–10.
[6]
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image super-resolution using deep convolutional networks.IEEE Trans. Pattern Anal. Mach. Intell. 38, 2 (February 2016), 295–307.
[7]
Guangwei Gao and Jian Yang. 2014. A novel sparse representation based framework for face image super-resolution.Neurocomputing 134 (June 2014), 92–99.
[8]
Guangwei Gao, Jian Yang, Xiao-Yuan Jing, Fumin Shen, Wankou Yang, and Dong Yue. 2017. Learning robust and discriminative low-rank representations for face recognition with occlusion.Pattern Recogn. 66 (June 2017), 129–143.
[9]
Guangwei Gao, Yi Yu, Jin Xie, Jian Yang, Meng Yang, and Jian Zhang. 2020. Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution.Pattern Recogn. (July 2020), 107539.
[10]
Guangwei Gao, Yi Yu, Meng Yang, Heyou Chang, Pu Huang, and Dong Yue. 2020. Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression.Inf. Sci. 506 (January 2020), 19–36.
[11]
Wen Gao, Bo Cao, Shiguang Shan, Xilin Chen, Delong Zhou, Xiaohua Zhang, and Debin Zhao. 2008. The CAS-PEAL large-scale Chinese face database and baseline evaluations.IEEE Trans. Syst., Man, Cybern. A, Syst., Humans 38, 1 (January 2008), 149–161.
[12]
Shiming Ge, Shengwei Zhao, Xindi Gao, and Jia Li. 2019. Fewer-shots and lower-resolutions: Towards ultrafast face recognition in the wild. In Proceedings of the 27th ACM International Conference on Multimedia. ACM, 229–237.
[13]
Bilal Ghanem, Paolo Rosso, and Francisco Rangel. 2020. An emotional analysis of false information in social media and news articles.ACM Trans. Internet Technol. 20, 2 (April 2020), 1–18.
[14]
Klemen Grm, Walter J. Scheirer, and Vitomir Štruc. 2019. Face hallucination using cascaded super-resolution and identity priors.IEEE Trans. Image Proc. 29, 1 (October 2019), 2150–2165.
[15]
Hu He and Lisimachos P. Kondi. 2006. An image super-resolution algorithm for different error levels per frame.IEEE Trans. Image Proc. 15, 3 (March 2006), 592–603.
[16]
Alain Hore and Djemel Ziou. 2010. Image quality metrics: PSNR vs. SSIM. In Proceedings of the IEEE International Conference on Pattern Recognition. IEEE, 2366–2369.
[17]
Xiao Hu, Peirong Ma, Zhuohao Mai, Shaohu Peng, Zhao Yang, and Li Wang. 2019. Face hallucination from low quality images using definition-scalable inference.Pattern Recogn. 94 (October 2019), 110–121.
[18]
Kui Jia and Shaogang Gong. 2008. Generalized face super-resolution.IEEE Trans. Image Proc. 17, 6 (June 2008), 873–886.
[19]
Junjun Jiang, Ruimin Hu, Zhongyuan Wang, and Zhen Han. 2014. Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning.IEEE Trans. Image Process. 23, 10 (August 2014), 4220–4231.
[20]
Junjun Jiang, Ruimin Hu, Zhongyuan Wang, and Zhen Han. 2014. Noise robust face hallucination via locality-constrained representation.IEEE Trans. Multimedia 16, 5 (August 2014), 1268–1281.
[21]
Junjun Jiang, Jiayi Ma, Chen Chen, Xinwei Jiang, and Zheng Wang. 2017. Noise robust face image super-resolution through smooth sparse representation.IEEE Trans. Cybern. 47, 11 (November 2017), 3991–4002.
[22]
Junjun Jiang, Yi Yu, Suhua Tang, Jiayi Ma, Akiko Aizawa, and Kiyoharu Aizawa. 2019. Context-patch based face hallucination via thresholding locality-constrained representation and reproducing learning.IEEE Trans. Cybern. 50, 1 (January 2019), 324–337.
[23]
Juncheng Li, Faming Fang, Kangfu Mei, and Guixu Zhang. 2018. Multi-scale residual network for image super-resolution. In Proceedings of the European Conference on Computer Vision. 517–532.
[24]
Mengyan Li, Yuechuan Sun, Zhaoyu Zhang, and Jun Yu. 2018. A coarse-to-fine face hallucination method by exploiting facial prior knowledge. In Proceedings of the 25th IEEE International Conference on Image Processing (ICIP ’18). IEEE, 61–65.
[25]
Xiaoming Li, Ming Liu, Yuting Ye, Wangmeng Zuo, Liang Lin, and Ruigang Yang. 2018. Learning warped guidance for blind face restoration. In Proceedings of the European Conference on Computer Vision (ECCV ’18). 272–289.
[26]
Zhouchen Lin and Heung-Yeung Shum. 2004. Fundamental limits of reconstruction-based superresolution algorithms under local translation.IEEE Trans. Pattern Anal. Mach. Intell. 26, 1 (January 2004), 83–97.
[27]
Tao Lu, Jiaming Wang, Junjun Jiang, and Yanduo Zhang. 2020. Global-local fusion network for face super-resolution.Neurocomputing 387 (April 2020), 309–320.
[28]
Xiang Ma, Junping Zhang, and Chun Qi. 2010. Hallucinating face by position-patch.Pattern Recogn. 43, 6 (June 2010), 2224–2236.
[29]
Mohamed Naleer Haju Mohamed, Yao Lu, and Feng Lv. 2013. Effective two-step method for face hallucination based on sparse compensation on over-complete patches.IET Image Process. 7, 6 (August 2013), 624–632.
[30]
Sivaram Prasad Mudunuri, Shashanka Venkataramanan, and Soma Biswas. 2019. Dictionary alignment with re-ranking for low-resolution NIR-VIS face recognition.IEEE Trans. Inf. Forensics Secur. 14, 4 (April 2019), 886–896.
[31]
Sung Cheol Park, Min Kyu Park, and Moon Gi Kang. 2003. Super-resolution image reconstruction: A technical overview.IEEE Signal Process. Mag. 20, 3 (May 2003), 21–36.
[32]
Shyam Singh Rajput, K. V. Arya, and Vinay Singh. 2018. Robust face super-resolution via iterative sparsity and locality-constrained representation.Inf. Sci. 463 (October 2018), 227–244.
[33]
Ching-Ting Tu and Jang-Ren Luo. 2015. Robust face hallucination using ensemble of feature-based regression functions and classifiers.Image Vis. Comput. 44 (December 2015), 59–72.
[34]
Chih-Yuan Yang, Sifei Liu, and Ming-Hsuan Yang. 2013. Structured face hallucination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1099–1106.
[35]
Yi Yu, Suhua Tang, Francisco Raposo, and Lei Chen. 2019. Deep cross-modal correlation learning for audio and lyrics in music retrieval.ACM Trans. Multim. Comput. 15, 1 (February 2019), 1–16.

Cited By

View all
  • (2024)LSDN Empowers the Metaverse Communication to Achieve High-Resolution Visuals with Lower BandwidthACM Transactions on Internet Technology10.1145/3672075Online publication date: 24-Jun-2024
  • (2024)Space-Time-Aware Proactive QoS Monitoring for Mobile Edge ComputingIEEE Transactions on Network and Service Management10.1109/TNSM.2024.342484721:5(5662-5676)Online publication date: 1-Oct-2024
  • (2024)TinyDeepUAV: A Tiny Deep Reinforcement Learning Framework for UAV Task Offloading in Edge-Based Consumer ElectronicsIEEE Transactions on Consumer Electronics10.1109/TCE.2024.344529070:4(7357-7364)Online publication date: 1-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology
ACM Transactions on Internet Technology  Volume 21, Issue 3
August 2021
522 pages
ISSN:1533-5399
EISSN:1557-6051
DOI:10.1145/3468071
  • Editor:
  • Ling Liu
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2021
Accepted: 01 March 2021
Revised: 01 November 2020
Received: 01 December 2019
Published in TOIT Volume 21, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Face super-resolution
  2. contextual patches
  3. locality-constrained representation
  4. coupled-layer representation

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Six Talent Peaks Project in Jiangsu Province
  • Natural Science Foundation of Jiangsu Province
  • Computer Information Processing Technology (Soochow University)

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)LSDN Empowers the Metaverse Communication to Achieve High-Resolution Visuals with Lower BandwidthACM Transactions on Internet Technology10.1145/3672075Online publication date: 24-Jun-2024
  • (2024)Space-Time-Aware Proactive QoS Monitoring for Mobile Edge ComputingIEEE Transactions on Network and Service Management10.1109/TNSM.2024.342484721:5(5662-5676)Online publication date: 1-Oct-2024
  • (2024)TinyDeepUAV: A Tiny Deep Reinforcement Learning Framework for UAV Task Offloading in Edge-Based Consumer ElectronicsIEEE Transactions on Consumer Electronics10.1109/TCE.2024.344529070:4(7357-7364)Online publication date: 1-Nov-2024
  • (2023)A semantic and service-based approach for adaptive mutli-structured data curation in data lakehousesWorld Wide Web10.1007/s11280-023-01218-326:6(4001-4023)Online publication date: 6-Nov-2023
  • (2023)Online-learning task scheduling with GNN-RL scheduler in collaborative edge computingCluster Computing10.1007/s10586-022-03957-w27:1(589-605)Online publication date: 30-Jan-2023
  • (2023)Remote Sensing Image Super-Resolution Using Deep Convolutional Neural Networks and AutoencoderArtificial Intelligence and Industrial Applications10.1007/978-3-031-43520-1_14(161-171)Online publication date: 15-Sep-2023
  • (2022)Probabilistic cooperative coded forwarding for broadcast transmissions in industrial mobile edge communicationsEURASIP Journal on Wireless Communications and Networking10.1186/s13638-022-02132-42022:1Online publication date: 13-Jun-2022
  • (2022)A Blockchain-Based Privacy Preservation Scheme in Mobile MedicalSecurity and Communication Networks10.1155/2022/98892632022Online publication date: 1-Jan-2022
  • (2022)A Composite Service Provisioning Mechanism in Edge ComputingMobile Information Systems10.1155/2022/90312012022Online publication date: 18-Aug-2022
  • (2022)SoDaWireless Communications & Mobile Computing10.1155/2022/78629112022Online publication date: 1-Jan-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media