Abstract
Previous methods of highlight removal in image processing have exclusively addressed images taken in specific illumination environments. However, most of these methods have limitations in natural scenes and thus, introduce artifacts to nature images. In this work, we propose a specular highlight removal method that is applicable to natural scene image. Firstly, we decompose the input image into a reflectance image and an illumination image based on Retinex theory, and show that the illumination image of natural scene is obviously different from that of commonly used experimental scene. Then, the smooth features of the input image are extracted to help estimate the specular reflection coefficient in chromaticity space. Finally, the space transformation is used to calculate the specular components and the highlight removal is achieved by subtracting the specular reflection component from the original image. The experimental results show that our method outperforms most of existing methods on natural scene images, especially in some challenging scenarios with saturated pixels and complex textures.
This research was supported in part by the National Natural Science Foundation of China 61727809 and in part by the Special Fund for Key Program of Science and Technology of Anhui Province 201903a05020022 and 201903c08020002. The first two authors contribute equally to this work.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Son, M., Lee, Y., Chang, H.S.: Toward specular removal from natural images based on statistical reflection models. IEEE Trans. Image Process. 29, 4204–4218 (2020)
Chen, H., Jin, Y., Duan, M., Zhu, C., Chen, E.: DOF: a demand-oriented framework for image denoising. IEEE Trans. Industr. Inform. 17(8), 5369–5379 (2021)
Chen, H., Jin, Y., Xu, K., Chen, Y., Zhu, C.: Multiframe-to-multiframe network for video denoising. IEEE Trans. Multimedia, 1–15 (2021)
Zhu, T., Xia, S., Bian, Z., Lu, C.: Highlight removal in facial images. In: Proceedings of the Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 422–433 (2020)
Chen, H., Jin, Y., Jin, G., Zhu, C., Chen, E.: Semisupervised semantic segmentation by improving prediction confidence. IEEE Trans. Neural Netw. Learn. Syst., 1–13 (2021)
Umeyama, S., Godin, G.: Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 639–647 (2004)
Wang, F., Ainouz, S., Petitjean, C., Bensrhair, A.: Specularity removal: a global energy minimization approach based on polarization imaging. Comput. Vis. Image Underst. 158, 31–39 (2017)
Yang, Q., Wang, S., Ahuja, N.: Real-time specular highlight removal using bilateral filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 87–100. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_7
Yang, Q., Tang, J., Ahuja, N.: Efficient and robust specular highlight removal. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1304–1311 (2015)
Shen, H.-L., Zheng, Z.-H.: Real-time highlight removal using intensity ratio. Appl. Opt. 52(19), 4483–4493 (2013)
Suo, J., An, D., Ji, X., Wang, H., Dai, Q.: Fast and high quality highlight removal from a single image. IEEE Trans. Image Process. 25(11), 5441–5454 (2016)
Ren, W., Tian, J., Tang, Y.: Specular reflection separation with color-lines constraint. IEEE Trans. Image Process. 26(5), 2327–2337 (2017)
Tan, R., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27(2), 178–193 (2005)
Tan, R., Ikeuchi, K.: Reflection components decomposition of textured surfaces using linear basis functions. In: Proceedings of the IEEE Conference on Computer Visual Pattern Recognition (CVPR), vol. 1, pp. 125–131 (2005)
Shen, H.-L., Zhang, H.-G., Shao, S.-J., Xin, J.H.: Chromaticity-based separation of reflection components in a single image. Pattern Recognit. 41(8), 2461–2469 (2008)
Klinker, G.J., Shafer, S.A., Kanade, T.: The measurement of highlights in color images. Int. J. Comput. Vis. 2(1), 309–334 (1992)
Guo, J., Zhou, Z., Wang, L.: Single image highlight removal with a sparse and low-rank reflection model. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 282–298. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_17
Fu, G., Zhang, Q., Song, C., Lin, Q., Xiao, C.: Specular highlight removal for real-world images. Comput. Graph. Forum. 38(7), 253–263 (2019)
Wu, Z., Zhuang, C., Shi, J., Xiao, J., Guo, J.: Deep specular highlight removal for single real-world image. In: SIGGRAPH Asia, pp. 1–2. ACM (2020)
Fu, G., Zhang, Q., Zhu, L., Li, P., Xiao, C.: A multi-task network for joint specular highlight detection and removal. In: Proceedings of the IEEE/CVF Conference on Computer Visual Pattern Recognition (CVPR), pp. 7752–7761 (2021)
Wang, S., Zheng, J., Hu, H.-M., Li, B.: Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 22(9), 3538–3548 (2013)
Yefeng, H., Bo, L., Jin, Z., Bai, Y.: An adaptive image enhancement based on the vector closed operations. In: Proceedings of the International Conference on Image and Graphics (ICIG), pp. 75–80 (2007)
Tan, T., Nishino, K., Ikeuchi, K.: Illumination chromaticity estimation using inverse-intensity chromaticity space. In: Proceedings of the IEEE Conference on Computer Visual Pattern Recognition (CVPR), vol. 1, pp. 673–680 (2003)
Cai, J., Gu, S., Zhang, L.: Learning a deep single image contrast enhancer from multi-exposure images. IEEE Trans. Image Process. 27(4), 2049–2062 (2018)
Wright, J., Yang, A., Ganesh, A., Sastry, S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)
Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), pp. 612–620 (2011)
Zhang, Y., Jiang, Z., Davis, L.S.: Learning structured low-rank representations for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 676–683 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, H. et al. (2021). Single Image Specular Highlight Removal on Natural Scenes. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13021. Springer, Cham. https://doi.org/10.1007/978-3-030-88010-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-88010-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88009-5
Online ISBN: 978-3-030-88010-1
eBook Packages: Computer ScienceComputer Science (R0)