Elsevier

Neurocomputing

Volume 119, 7 November 2013, Pages 222-231
Neurocomputing

Color-to-gray based on chance of happening preservation

https://doi.org/10.1016/j.neucom.2013.03.037Get rights and content

Abstract

It is important to convert color images into grayscale ones for both commercial and scientific applications, such as reducing the publication cost and making the color blind people capture the visual content and semantics from color images. Recently, a dozen of algorithms have been developed for color-to-gray conversion. However, none of them considers the visual attention consistency between the color image and the converted grayscale one. Therefore, these methods may fail to convey important visual information from the original color image to the converted grayscale image. Inspired by the Helmholtz principle (Desolneux et al. 2008 [16]) that “we immediately perceive whatever could not happen by chance”, we propose a new algorithm for color-to-gray to solve this problem. In particular, we first define the Chance of Happening (CoH) to measure the attentional level of each pixel in a color image. Afterward, natural image statistics are introduced to estimate the CoH of each pixel. In order to preserve the CoH of the color image in the converted grayscale image, we finally cast the color-to-gray to a supervised dimension reduction problem and present locally sliced inverse regression that can be efficiently solved by singular value decomposition. Experiments on both natural images and artificial pictures suggest (1) that the proposed approach makes the CoH of the color image and that of the converted grayscale image consistent and (2) the effectiveness and the efficiency of the proposed approach by comparing with representative baseline algorithms. In addition, it requires no human–computer interactions.

Introduction

Color-to-gray algorithms convert color images to grayscale ones for different purposes, such as, publishing less expensive alternatives to full color printings, e-ink based book reader, and producing reading materials for color blind people. By converting a color image to a grayscale one, we will inevitably lose the visual information, and thus a dozen of color-to-gray algorithms [1] have been presented to remain the visual information as much as possible.

Color-to-gray through a linear combination of R, G and B channels is the most popular and efficient strategy to convert a color image into a grayscale one. Wyszecki and Stiles [2] have combined the R, G, B channels by using a group of weighted linear functions. Wu and Rao [3] have linearly combined the R, G, and B channels by defining a series of policies to separate luminance value from the chrominance value so as to construct the gray-scale image based on the luminance value.

Color-to-gray can also be treated as a dimension reduction problem. Therefore, unsupervised dimension reduction algorithms can be employed to transform the three dimensional color-space to one dimensional grayscale. In particular, principal component analysis (PCA) [5] constructs the covariance matrix based on the statistics of the R, G, and B values of pixels and projects color pixels to the leading principle component. The method assumes that observations are drawn from a single Gaussian and ignores the spatial distribution of color pixels. Although its kernelization [6] can deal with the non-Gaussian distribution property of color pixels, the computational complexity is too expensive to be applicable in practice, let alone the sensitivity of the kernel parameter to different images.

Bala and Braun [7] have sorted all of the colors in an image according to their lightness and arranged the colors to grayscales by using a series of weights which are proportional to their color distance. It performs well for images with simple structures, e.g., color bins and blocks, but fails for images with high frequency, e.g., a rasterized graphic. Afterward, Bala and Eschbach [8] have applied the high frequency chrominance information to the luminance channel to preserve the difference between adjacent colors locally in the converted grayscale image.

Rasche et al. [10] have defined an objective function that enforces the proportional color differences across the mapping between color and grayscale based on a subset of pixels in a color image. By minimizing an objective function, a linear mapping is obtained for color-to-gray. However, it ignores the spatial distribution of different colors and the different perceiving effects of such spatial distribution. Therefore, it fails to deal well with images containing small splashes of colors. To reduce this problem, they [11] have introduced multidimensional scaling (MDS) [12] to the CIELAB color space [2] for color-to-gray. Unfortunately, its computational cost is not acceptable for images with abundant colors.

By considering the spatial distribution of colors, Socolinsky and Wolff [9] have utilized image gradient information to describe the color difference between neighboring pixels and applied such a difference to implement color-to-gray. This method cannot deal with long-scale different regions. Especially it is challenging for this method to convert the pseudo-isochromatic plates to grayscale images while preserving the semantic content.

Gooch et al. [13] have presented an interactive Color2Gray method that tuned three parameters: the difference in color chrominance, the contrast between colors, and the size of a pixel's neighborhood. Both the color difference and the spatial distribution of colors are taken into account. By using this algorithm, users can preserve their preferred visual cues via tuning the above three parameters manually. Smith et al. [14] have presented a color-to-gray method by combining the global Helmholtz–Kohlrausch color appearance mapping and the multiscale local contrast enhancement. Human–computer interactions (HCIs) are also required to obtain the multiscale local contrast enhancement, which makes the scheme suited for natural images.

Most recently, Song et al. [15] have defined three visual cues by considering the color spatial distribution, gradient information and the perception priority of hue, chroma and lightness. By casting color-to-gray as a visual cue preservation procedure (VCP), a probabilistic graphical model and an efficient optimization have been presented and achieved top level performance compared with representative conventional algorithms without HCIs.

Although many representative methods have been proposed as aforementioned, it is obvious that these existing methods fail to consider the importance of distribution of visual information from the view of human visual system. It has been widely acknowledged that human visual system is usually sensitive to the important visual cues and neglects unimportant ones in an image [4]. For example, as shown in the converted grayscale image in Fig. 1, the grayscale renderings of the color image by linearly combining R, G, and B according to [3], by principal component analysis (PCA) [5], by multi-dimensional scaling (MDS) [11] and by visual cue preservation (VCP) [15], all fail to transfer the visual attention from the color image to the converted grayscale one. In the original color image, the yellow bush attracts observers' attention but the bush in the converted grayscale images cannot. Therefore, it is essential to develop a new color-to-gray algorithm to analyze the attentional importance of pixel and make the important pixels being distinct in the color-to-gray operation.

According to the Helmholtz principle in Gestalt psychological theory [16] “Whenever some large deviation from randomness occurs, a structure is perceived”. As a commonsense statement, the Helmholtz principle means that “we immediately perceive whatever could not happen by chance”. This description inspires us with two cues: on one hand, the importance of a region depends heavily on the Chance of Happening (CoH), which is estimated based on the previous experience of HVS; on the other hand, only the region which “could not happen by chance” tends to be “immediately perceived” by HVS, where “could not happen by chance” means the CoH of the region is small.

Inspired by the Helmholtz principle above, we employ the CoH to evaluate the attentional importance of pixels in the color image for the subsequent color-to-gray. In particular, we estimate CoH based on the natural image statics and develop locally sliced inverse regression (LSIR) to transfer CoHs of pixels in the color image (and visual attention of the color image accordingly) to the converted grayscale image. Thorough empirical evaluations show the effectiveness of this CoH preserving based color-to-gray by comparing with representative color-to-gray baseline algorithms.

The rest of the paper is organized as follows. Section 2 introduces the LSIR based CoH preserving for color-to-gray. Section 3 defines CoH and presents its computation based on natural image statistics. Afterward, LSIR for CoH preserving based color-to-gray is described in Section 4. Experimental results in Section 5 thoroughly demonstrate the effectiveness of the new algorithm. Section 6 concludes the paper.

Section snippets

LSIR based CoH preserving for color-to-gray

Given a color image Ic, the color-to-gray conversion can be treated as a dimension reduction process by seeking a proper projection vector β to map Ic to a grayscale image Ig, Ig=βTIc.

Different from the existing approaches, the new color-to-gray algorithm preserves the CoH of each pixel in Ic. The CoHs of pixels in a color image indicate particular properties of the image that attract receivers' attention. Fig. 2 shows that the proposed scheme for color-to-gray by using LSIR for preserving CoH

On calculating the chance of happening

The CoH of a pixel represents how likely this pixel will appear at a specific location in an image. It is mainly determined by two groups of priors, which are color statistics based priors (CSP) and spatial correlations based priors (SCP). After obtaining CSP and SCP from a set of natural images, we present a Bayesian framework to obtain the CoH of a pixel.

LSIR for CoH preserving

We cast the process of color-to-gray to a supervised dimension reduction problem. Given a color image of size W×H, we can obtain a set of labelled samples D={I(i),L(i)|1iW×H}, wherein I(i) is the RGB vector of the i-th pixel and L(i) is the corresponding label obtained by separating CoHs of pixels into two groups according to a predefined threshold εL(i)=L(x,y)={1CoH(x,y)<ε0Otherwise

We introduce locally sliced inverse regression (LSIR) that extends sliced inverse regression (SIR) [20] by

Experimental results

In this section, we first evaluate the proposed CoH preserving based color-to-gray algorithm by comparing with well-known automatic color-to-gray techniques, which are the color-to-gray function provided by Adobe Photoshop [1], the principle component analysis based color-to-gray (PCA) [5], the multidimensional scaling based color-to-gray (MDS) [11], and the visual cue preserving based color-to-gray (VCP) [15] in terms of effectiveness and efficiency. We do not use the interactive approaches as

Conclusion and future work

Conventional color-to-gray algorithms have been successfully applied in practice but fail to preserve the visual attention consistency between the color image and the converted grayscale image. To solve this problem, this paper proposed an algorithm that transfers the Chance of Happening (CoH) of the pixels in the original color image to the converted grayscale image. CoH is defined according to the Helmholtz principle in Gestalt psychological theory that “we immediately perceive whatever could

Acknowledgments

This work was supported in part by National Natural Science Foundation of China (61170142), National Key Technology R&D Program (2011BAG05B04),the Program of International S&T Cooperation under Grant 2013DFG12840, and the Fundamental Research Funds for the Central Universities.

Mingli Song received the Ph.D. degree incomputer science from Zhejiang University, China, in 2006. He is currently an associate professor in the College of Computer Science and Microsoft Visual Perception Laboratory, Zhejiang university. His research interests include visual perception analysis, image enhancement, and face modeling. He is a Senior Member of the IEEE.

References (28)

  • C. Archambeau et al.

    Mixtures of robust probabilistic principal component analyzers

    Neurocomputing

    (2008)
  • R. Brown, Photoshop tips: converting color to black-and-white. 〈 http://www.russellbrown.com/tips_tech.html〉,...
  • G. Wyszecki et al.

    Color ScienceConcepts and Methods, Quantitative Data and Formulae

    (2000)
  • H.R. Wu, K.R. Rao, Digital Video Image Quality and Perceptual Coding, CRC Press,...
  • W. James

    The Principle of Psychology

    (1890)
  • S. Mika, B. Schölkopf, A. Smola, K.-R. Müller, M. Scholz, G. Räsche, Kernel PCA and de-noising in feature spaces, in:...
  • R. Bala, K. Braun, Color-to-grayscale conversion to maintain discriminability, in: Proceedings of the SPIE, 2004, pp....
  • R. Bala, R. Eschbach, Spatial color-to-grayscale transformation preserving chrominance edge information, in:...
  • D. Socolinsky et al.

    Multispectral image visualization through first-order fusion

    IEEE Trans. Image Process.

    (2002)
  • K. Rasche et al.

    Detail preserving reproduction of colour images for monochromats and dischromats

    IEEE Comput. Graph. Appl.

    (2005)
  • K. Rashce et al.

    Re-colouring images for gamuts of lower dimension

    Comput. Graph. Forum

    (2005)
  • S. Flori

    Visualization of Riemannian-manifold-valued elements by multidimensional scaling

    Neurocomputing

    (2011)
  • A. Gooch et al.

    Color2Graysalience-preserving color removal

    ACM Trans. Graph.

    (2005)
  • K. Smith et al.

    Apparent grayscalea simple and fast conversion to perceptually accurate images and video

    Comput. Graph. Forum

    (2008)
  • Cited by (26)

    • Real-time framework for image dehazing based on linear transmission and constant-time airlight estimation

      2018, Information Sciences
      Citation Excerpt :

      Pixels with high values in the minimal channel map are expected to have a significant haze effect. Alternatively, colour-to-grey conversion techniques, such as that proposed in [31], could be employed. The ALII technique exhibits the following distinctive characteristics:

    • Quality assessment of retargeted images by salient region deformity analysis

      2017, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      Using these criteria, not only facilitates online quality monitoring of retargeted images, but also retargeting methods can be optimized through it. Although distorted image quality assessment methods for natural images have made good progress [10–13], the field of quality assessment for retargeted images is still in its infancy. For natural images that have gone through noisy channels, or been compressed, or have been filtered, there are powerful objective image quality assessment (IQA) methods which produce quality scores that are very close to subjective scores.

    • Boosted random contextual semantic space based representation for visual recognition

      2016, Information Sciences
      Citation Excerpt :

      Since Internet images were often contaminated with noise, the resulting semantic spaces may be biased and cannot represent images well. The use of attributes had also been widely studied [15–17,36,38]. Attributes were chosen by humans as the concepts that could be easily distinguished by computers.

    • Biologically inspired image quality assessment

      2016, Signal Processing
      Citation Excerpt :

      For example, images of poor quality may lead to obstacles in learning or applying such systems for practical applications, e.g. scene recognition [1], image retrieval [2], and so on. In addition, image quality can be adopted as a criterion for evaluating the performance of image processing systems [3–5], optimizing image processing algorithms, and monitoring the working condition of devices [6]. Thus it is meaningful to develop image quality assessment (IQA) methods that can precisely and automatically estimate human perceived image quality.

    View all citing articles on Scopus

    Mingli Song received the Ph.D. degree incomputer science from Zhejiang University, China, in 2006. He is currently an associate professor in the College of Computer Science and Microsoft Visual Perception Laboratory, Zhejiang university. His research interests include visual perception analysis, image enhancement, and face modeling. He is a Senior Member of the IEEE.

    Dapeng Tao received the B.S. degree in Electronics and Information Engineering from Northwestern Polytechnical University, Xi'an, China. He is currently a Ph.D. candidate in Information and Communication Engineering at South China University of Technology, Guangzhou, China. His research interests include machine learning, computer vision and cloud computing.

    Chun Chen is a professor in the College of Computer Science at Zhejiang University, China. His research interests include computer vision, computer graphics, and embedded technology.

    Jiajun Bu received the B.S and Ph.D. degrees in Computer Science from Zhejiang University, China, in 1995 and 2000, respectively. He is a professor in College of Computer Science and the Director of Embedded System and Software Center at Zhejiang University. His research interests include video coding in embedded system, data mining, and mobile database.

    Yezhou Yangis a Ph.D. student at Department of Computer Science at University of Maryland, College Park. He received the B.S degree in Computer Science from Zhejiang University, China, in 2010. His research interests include computer vision and robot vision.

    View full text