Herbert Potechius
ABSTRACT
Over the past decades, an overwhelming number of scientific contributions have been published related to the topic of color transfer, where the color statistic of an image is transferred to another image. Recently, this idea was further extended to 3D point clouds. Due to the fact that the results are normally evaluated subjectively, an objective comparison of multiple algorithms turns out to be difficult. Therefore, this paper introduces the ColorTransferLab, a web based test bed that offers a large set of state-of-the-art color transfer implementations. Furthermore, it allows users to integrate their implementations with the ultimate goal of providing a library of state-of-the-art algorithms for the scientific community. This test bed can manipulate both 2D images, 3D point clouds and textured triangle meshes, and it allows us to objectively evaluate and compare color transfer algorithms by providing a large set of objective metrics. As part of ColorTransferLab, we are introducing a comprehensive dataset of freely available images. This dataset comprises a diverse range of content with a wide array of color distributions, sizes, and color depths which helps in appropriately evaluating color transfer. Its comprehensive nature makes it invaluable for accurately evaluating color transfer methods.
- E. Adelson. 1993. Perceptual organization and the judgment of brightness. Science 262 5142 (1993), 2042–4.Google Scholar
- M. Afifi, A. Abuolaim, M. Hussien, M. A. Brubaker, and M. S. Brown. 2021a. CAMS: Color-Aware Multi-Style Transfer. arXiv (2021). https://doi.org/10.48550/ARXIV.2106.13920Google ScholarCross Ref
- M. Afifi, M. Brubaker, and S. Brown. 2021b. HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.48550/ARXIV.2011.11731Google ScholarCross Ref
- B. Berlin and P. Kay. 1969. Basic Color Terms: Their Universal-ity and Evolution. University of California Press, Berkely.Google Scholar
- X. Cao, W. Wang, K. Nagao, and R. Nakamura. 2020. PSNet: A Style Transfer Network for Point Cloud Stylization on Geometry and Color. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 3326–3334. https://doi.org/10.1109/WACV45572.2020.9093513Google ScholarCross Ref
- H. Chang, O. Fried, Y. Liu, S. DiVerdi, and A. Finkelstein. 2015. Palette-based Photo Recoloring. ACM Transactions on Graphics (TOG) 34 (07 2015), 139:1–139:11. https://doi.org/10.1145/2766978Google ScholarDigital Library
- Y. Chang, S. Saito, and M. Nakajima. 2003. A framework for transfer colors based on the basic color categories. Proceedings of the Computer Graphics International Conference (CGI), 176– 181. https://doi.org/10.1109/CGI.2003.1214463Google ScholarCross Ref
- R. Charles, H. Su, M. Kaichun, and L. Guibas. 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 77–85. https://doi.org/10.1109/CVPR.2017.16Google ScholarCross Ref
- G. Chen, C. Yang, and S. Xie. 2006. Gradient-Based Structural Similarity for Image Quality Assessment. In IEEE International Conference on Image Processing (ICIP). 2929–2932. https://doi.org/10.1109/ICIP.2006.313132Google ScholarCross Ref
- W. Chen, M. Huang, and C. Wang. 2016. Optimizing color transfer using color similarity measurement. In IEEE/ACIS International Conference on Computer and Information Science (ICIS). 1–6. https://doi.org/10.1109/ICIS.2016.7550799Google ScholarCross Ref
- C. Gao, K. Panetta, and S. Agaian. 2015. Color image retrieval and analysis using image color measures. In Mobile Multimedia/Image Processing, Security, and Applications 2015, Sos S. Agaian, Sabah A. Jassim, and Eliza Yingzi Du (Eds.). Vol. 9497. International Society for Optics and Photonics, SPIE, 949702. https://doi.org/10.1117/12.2180605Google ScholarCross Ref
- L. Gatys, A. Ecker, and M. Bethge. 2015. A Neural Algorithm of Artistic Style. arXiv (08 2015). https://doi.org/10.1167/16.12.326Google ScholarCross Ref
- I. Goudé, R. Cozot, O. Le Meur, and K. Bouatouch. 2021. Example‐Based Colour Transfer for 3D Point Clouds. Computer Graphics Forum (CGF) 40 (08 2021). https://doi.org/10.1111/cgf.14388Google ScholarCross Ref
- M. Grogan and R. Dahyot. 2019. L 2 Divergence for Robust Colour Transfer. Computer Vision and Image Understanding (CVIU) 181, C (apr 2019), 39–49. https://doi.org/10.1016/j.cviu.2019.02.002Google ScholarDigital Library
- C. Gu, X. Lu, and C. Zhang. 2022. Example-based color transfer with Gaussian mixture modeling. Pattern Recognition 129 (2022), 108716. https://doi.org/10.1016/j.patcog.2022.108716Google ScholarDigital Library
- D. Hasler and S. Suesstrunk. 2003. Measuring Colourfulness in Natural Images. Proceedings of SPIE - The International Society for Optical Engineering 5007 (06 2003), 87–95. https://doi.org/10.1117/12.477378Google ScholarCross Ref
- Y. Huang, P. Wang, C. Chou, and K. Lin. 2011. An Automatic Selective Color Transfer Algorithm for Images. In Proceedings of the 2011 ACM Symposium on Applied Computing (SAC) (TaiChung, Taiwan) (SAC ’11). Association for Computing Machinery, New York, NY, USA, 66–71. https://doi.org/10.1145/1982185.1982201Google ScholarDigital Library
- J. Lee and S. Lee. 2016. Hallucination from Noon to Night Images Using CNN. In SIGGRAPH ASIA 2016 Posters (Macau) (SA ’16). Article 15, 1 pages. https://doi.org/10.1145/3005274.3005320Google ScholarDigital Library
- J. Lee, H. Son, G. Lee, J. Lee, S. Cho, and S. Lee. 2020. Deep Color Transfer Using Histogram Analogy. The Visual Computer (Vis. Comput.) 36, 10–12 (oct 2020), 2129–2143. https://doi.org/10.1007/s00371-020-01921-6Google ScholarDigital Library
- C. Li and A. Bovik. 2010. Content-partitioned structural similarity index for image quality assessment. Signal Processing: Image Communication 25, 7 (2010), 517–526. https://doi.org/10.1016/j.image.2010.03.004 Special Issue on Image and Video Quality Assessment.Google ScholarDigital Library
- Z. Li, Z. Tan, L. Cao, H. Chen, L. Jiao, and Y. Zhong. 2019. Directive local color transfer based on dynamic look-up table. Signal Processing: Image Communication 79 (2019), 1–12. https://doi.org/10.1016/j.image.2019.06.010Google ScholarDigital Library
- F. Luan, S. Paris, E. Shechtman, and K. Bala. 2017. Deep Photo Style Transfer. Conference on Computer Vision and Pattern Recognition (CVPR) (2017). https://doi.org/10.48550/ARXIV.1703.07511Google ScholarCross Ref
- A. Mittal, A. Moorthy, and A. Bovik. 2012. No-Reference Image Quality Assessment in the Spatial Domain. IEEE Transactions on Image Processing (TIP) 21, 12 (2012), 4695–4708. https://doi.org/10.1109/TIP.2012.2214050Google ScholarDigital Library
- A. Mittal, R. Soundararajan, and A. Bovik. 2013. Making a “Completely Blind” Image Quality Analyzer. IEEE Signal Processing Letters 20, 3 (2013), 209–212. https://doi.org/10.1109/LSP.2012.2227726Google ScholarCross Ref
- S. Nercessian and K. Panetta. 2011. An Image Similarity Measure Using Enhanced Human Visual System Characteristics. Proceedings of SPIE - The International Society for Optical Engineering 8063 (05 2011). https://doi.org/10.1117/12.883301Google ScholarCross Ref
- M. Oskarsson. 2021. Robust Image-to-Image Color Transfer Using Optimal Inlier Maximization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 786–795. https://doi.org/10.1109/CVPRW53098.2021.00088Google ScholarCross Ref
- K. Panetta, L. Bao, and S. Agaian. 2016. Novel multi-color transfer algorithms and quality measure. IEEE Transactions on Consumer Electronics (TCE) 62, 3 (2016), 292–300. https://doi.org/10.1109/TCE.2016.7613196Google ScholarDigital Library
- K. Panetta, L. Bao, and S. Agaian. 2020. Fast Hue-Division-Based Selective Color Transfer. IEEE Transactions on Circuits and Systems for Video Technology (TCSVT) 30, 9 (2020), 2853–2866. https://doi.org/10.1109/TCSVT.2019.2921524Google ScholarDigital Library
- F. Pitie and A. Kokaram. 2007. The linear Monge-Kantorovitch linear colour mapping for example-based colour transfer. In European Conference on Visual Media Production (CVMP). 1–9. https://doi.org/10.1049/cp:20070055Google ScholarCross Ref
- F. Pitie, A. Kokaram, and R. Dahyot. 2005. N-dimensional probability density function transfer and its application to color transfer. In IEEE International Conference on Computer Vision (ICCV), Vol. 2. 1434–1439 Vol. 2. https://doi.org/10.1109/ICCV.2005.166Google ScholarDigital Library
- F. Pitié and R. Dahyot. 2005. Towards automated colour grading. Conference on Visual Media Production (CVMP) (01 2005).Google Scholar
- F. Pitié, A. Kokaram, and R. Dahyot. 2007. Automated colour grading using colour distribution transfer. Computer Vision and Image Understanding (CVIU) 107, 1 (2007), 123–137. https://doi.org/10.1016/j.cviu.2006.11.011 Special issue on color image processing.Google ScholarDigital Library
- H. Potechius, T. Sikora, and S. Knorr. 2021. Color Transfer of 3D Point Clouds For XR Applications. In IEEE International Conference on 3D Immersion (IC3D). 1–8. https://doi.org/10.1109/IC3D53758.2021.9687162Google ScholarCross Ref
- X. Qian, B. Wang, and L. Han. 2010. An efficient fuzzy clustering-based color transfer method. In International Conference on Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Vol. 2. 520–523. https://doi.org/10.1109/FSKD.2010.5569560Google ScholarCross Ref
- Y. Qian, D. Liao, and J. Zhou. 2013. Manifold alignment based color transfer for multiview image stitching. In IEEE International Conference on Image Processing (ICIP). 1341–1345. https://doi.org/10.1109/ICIP.2013.6738276Google ScholarCross Ref
- E. Reinhard, M. Ashikhmin, B. Gooch, and P. Shirley. 2001. Color Transfer between Images. IEEE Comput. Graph. Appl. 21, 5 (sep 2001), 34–41. https://doi.org/10.1109/38.946629Google ScholarDigital Library
- O. Ronneberger, P. Fischer, and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI)(LNCS, Vol. 9351). Springer, 234–241. http://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15aGoogle Scholar
- D. Ruderman, T. Cronin, and C. Chiao. 1998. Statistics of cone responses to natural images: implications for visual coding. Journal of the Optical Society of America (JOSA) 15, 8 (Aug 1998), 2036–2045. https://doi.org/10.1364/JOSAA.15.002036Google ScholarCross Ref
- K. Simonyan and A. Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations ICLR. http://arxiv.org/abs/1409.1556Google Scholar
- H. Talebi and P. Milanfar. 2018. NIMA: Neural Image Assessment. IEEE Transactions on Image Processing (TIP) 27, 8 (aug 2018), 3998–4011. https://doi.org/10.1109/tip.2018.2831899Google ScholarCross Ref
- Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing (TIP) 13, 4 (2004), 600–612. https://doi.org/10.1109/TIP.2003.819861Google ScholarDigital Library
- Z. Wang, E. Simoncelli, and A. Bovik. 2003. Multiscale structural similarity for image quality assessment. In Asilomar Conference on Signals, Systems and Computers (ACSSC), Vol. 2. 1398–1402 Vol.2. https://doi.org/10.1109/ACSSC.2003.1292216Google ScholarCross Ref
- T. Welsh, M. Ashikhmin, and K. Mueller. 2002. Transferring Color to Greyscale Images. ACM Transactions on Graphics (ACM Trans. Graph.) 21, 3 (jul 2002), 277–280. https://doi.org/10.1145/566654.566576Google ScholarDigital Library
- Y. Xiang, B. Zou, and H. Li. 2009. Selective color transfer with multi-source images. Pattern Recognition Letters 30, 7 (2009), 682–689. https://doi.org/10.1016/j.patrec.2009.01.004Google ScholarDigital Library
- Y. Xiao, L. Wan, C. Leung, Y. Lai, and T. Wong. 2013. Example-Based Color Transfer for Gradient Meshes. IEEE Transactions on Multimedia (TMM) 15, 3 (2013), 549–560. https://doi.org/10.1109/TMM.2012.2233725Google ScholarDigital Library
- B. Xie, Y. Li, S. Yang, Y. Xu, and G. Wang. 2023. Combined intrinsic decomposition and histogram matching for color migration. In International Conference on Signal Processing and Communication Technology (SPCT 2022), Vol. 12615. International Society for Optics and Photonics, SPIE, 126150F. https://doi.org/10.1117/12.2673997Google ScholarCross Ref
- C. Yao, C. Chang, and S. Chien. 2016. Example-based video color transfer. In IEEE International Conference on Multimedia and Expo (ICME). 1–6. https://doi.org/10.1109/ICME.2016.7552926Google ScholarCross Ref
- L. Zhang, Y. Shen, and H. Li. 2014. VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment. IEEE Transactions on Image Processing (TIP) 23, 10 (2014), 4270–4281. https://doi.org/10.1109/TIP.2014.2346028Google ScholarCross Ref
- L. Zhang, L. Zhang, X. Mou, and D. Zhang. 2011. FSIM: A Feature Similarity Index for Image Quality Assessment. IEEE Transactions on Image Processing (TIP) 20, 8 (2011), 2378–2386. https://doi.org/10.1109/TIP.2011.2109730Google ScholarDigital Library
- R. Zhang, P. Isola, A. Efros, E., and O. Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. arxiv:1801.03924Google Scholar
Index Terms
- A software test bed for sharing and evaluating color transfer algorithms for images and 3D objects
Recommendations
Color Transfer for Images: A Survey
High-quality image generation is an important topic in digital visualization. As a sub-topic of the research, color transfer is to produce high-quality image with ideal color scheme learned from the reference one. In this paper, we investigate the ...
Using assorted color spaces and pixel window sizes for colorization of grayscale images
ICWET '10: Proceedings of the International Conference and Workshop on Emerging Trends in TechnologyThere is no exact solution for colorization of grayscale images. In the initial work done [23], color traits transfer techniques to color grayscale images are proposed. The main focus of the techniques [23] is to minimise the human efforts needed in ...
Color Transfer Using Probabilistic Moving Least Squares
CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern RecognitionThis paper introduces a new color transfer method which is a process of transferring color of an image to match the color of another image of the same scene. The color of a scene may vary from image to image because the photographs are taken at ...
Comments