Abstract
Detecting and removing specular highlights is a highly challenging task that can effectively enhance the performance of other computer vision tasks in real-world scenarios. Traditional highlight removal algorithms struggle to accurately distinguish the differences between pure white or nearly white materials and highlights, while recent deep learning-based highlight removal algorithms suffer from complex network architectures, lack of flexibility, and limited object adaptability. To address these issues, we propose an end-to-end framework for single-image highlight detection and removal that utilizes mask guidance. Our framework adopts an encoder-decoder structure, with EfficientNet serving as the backbone network for feature extraction in the encoder. The decoder gradually restores the feature map to its original size through upsampling. In the highlight detection module, the network layers are deepened, and residual modules are introduced to extract more feature information and improve detection accuracy. In the highlight removal module, we introduce the Convolutional Block Attention Module, which dynamically learns the importance of each channel and spatial position in the input feature map. This helps the model better distinguish foreground and background and improve adaptability and accuracy in complex scenes.Experimental results demonstrate that our proposed method outperforms existing methods, as evaluated on the public SHIQ dataset through comparative experiments. Our method achieves superior performance in highlight detection and removal.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akashi, Y., Okatani, T.: Separation of reflection components by sparse non-negative matrix factorization. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9007, pp. 611–625. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16814-2_40
Feng, W., Cheng, X., Sun, J., Xiong, Z., Zhai, Z.: Specular highlight removal and depth estimation based on polarization characteristics of light field. Optics Commun. 537, 129467 (2023)
Feng, W., Li, X., Cheng, X., Wang, H., Xiong, Z., Zhai, Z.: Specular highlight removal of light field based on dichromatic reflection and total variation optimizations. Opt. Lasers Eng. 151, 106939 (2022)
Fu, G., Zhang, Q., Lin, Q., Zhu, L., Xiao, C.: Learning to detect specular highlights from real-world images. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 1873–1881 (2020)
Fu, G., Zhang, Q., Zhu, L., Li, P., Xiao, C.: A multi-task network for joint specular highlight detection and removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7752–7761 (2021)
Guo, J., Zhou, Z., Wang, L.: Single image highlight removal with a sparse and low-rank reflection model. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 268–283 (2018)
Hou, S., Wang, C., Quan, W., Jiang, J., Yan, D.-M.: Text-aware single image specular highlight removal. In: Ma, H., Wang, L., Zhang, C., Wu, F., Tan, T., Wang, Y., Lai, J., Zhao, Y. (eds.) PRCV 2021. LNCS, vol. 13022, pp. 115–127. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88013-2_10
Ikeuchi, K., Miyazaki, D., Tan, R.T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. In: Digitally Archiving Cultural Objects, pp. 353–384 (2008)
Jie, L., Zhang, H.: MGRLN-NET: mask-guided residual learning network for joint single-image shadow detection and removal. In: Proceedings of the Asian Conference on Computer Vision, pp. 4411–4427 (2022)
Ramos, V.S., Júnior, L.G.D.Q.S., Silveira, L.F.D.Q.: Single image highlight removal for real-time image processing pipelines. IEEE Access 8, 3240–3254 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Shen, H.L., Zheng, Z.H.: Real-time highlight removal using intensity ratio. Appl. Opt. 52(19), 4483–4493 (2013)
Shi, J., Dong, Y., Su, H., Yu, S.X.: Learning non-lambertian object intrinsics across shapenet categories. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1685–1694 (2017)
Shi, X., et al.: Deep learning for precipitation nowcasting: a benchmark and a new model. Adv. Neural Inf. Process. Syst. 30 (2017)
Souza, A.C., Macedo, M.C., Nascimento, V.P., Oliveira, B.S.: Real-time high-quality specular highlight removal using efficient pixel clustering. In: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 56–63. IEEE (2018)
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Wang, C., Wu, Z., Guo, J., Zhang, X.: Contour-constrained specular highlight detection from real-world images. In: The 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry, pp. 1–4 (2022)
Wu, S., et al.: Specular-to-diffuse translation for multi-view reconstruction. In: Proceedings of the European conference on computer vision (ECCV), pp. 183–200 (2018)
Wu, Z., Guo, J., Zhuang, C., Xiao, J., Yan, D.M., Zhang, X.: Joint specular highlight detection and removal in single images via unet-transformer. Comput. Visual Media 9(1), 141–154 (2023)
Yamamoto, T., Nakazawa, A.: General improvement method of specular component separation using high-emphasis filter and similarity function. ITE Trans. Media Technol. Appl. 7(2), 92–102 (2019)
Yang, J., Liu, L., Li, S.: Separating specular and diffuse reflection components in the HSI color space. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 891–898 (2013)
Yang, Q., Tang, J., Ahuja, N.: Efficient and robust specular highlight removal. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1304–1311 (2014)
Yang, Q., Wang, S., Ahuja, N.: Real-time specular highlight removal using bilateral filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 87–100. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_7
Yi, R., Tan, P., Lin, S.: Leveraging multi-view image sets for unsupervised intrinsic image decomposition and highlight separation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12685–12692 (2020)
Zhang, W., Zhao, X., Morvan, J.M., Chen, L.: Improving shadow suppression for illumination robust face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41(3), 611–624 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, H., Li, L., Yu, N. (2024). Mask-Guided Joint Single Image Specular Highlight Detection and Removal. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14433. Springer, Singapore. https://doi.org/10.1007/978-981-99-8546-3_37
Download citation
DOI: https://doi.org/10.1007/978-981-99-8546-3_37
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8545-6
Online ISBN: 978-981-99-8546-3
eBook Packages: Computer ScienceComputer Science (R0)