Abstract
Bayer pattern has been widely used in commercial digital cameras. In NASA’s mast camera (Mastcams) onboard the Mars rover Curiosity, Bayer pattern has also been used in capturing the RGB bands. It is well known that debayering, also known as demosaicing in the literature, introduces artifacts such as false colors and zipper edges. In this paper, we first present four fusion approaches, including weighted and the well-known alpha-trimmed mean filtering approaches. Each fusion approach combines demosaicing results from seven debayering algorithms in the literature, which are selected based on their performance mentioned in other survey papers and the availability of open source codes. Second, we present debayering results using two benchmark image data sets: IMAX and Kodak. It was observed that none of the seven algorithms in the literature can yield the best performance in terms of peak signal-to-noise ratio (PSNR), CIELAB score, and subjective evaluation. Although the fusion algorithms are simple, it turns out that the debayering performance can be improved quite dramatically after fusion based on our extensive evaluations. In particular, the average PSNR improvements of the weighted fusion algorithm over the best individual method are 1.1 dB for the IMAX database and 1.8 dB for the Kodak database, respectively. Third, we applied the various algorithms to 36 actual Mastcam images. Subjective evaluation indicates that the fusion algorithms still work well, but not as good as the existing debayering algorithm used by NASA.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bell III, J.F., et al.: The Mars Science Laboratory Curiosity Rover Mast Camera (Mastcam) Instruments: pre-flight and in-flight calibration, validation, and data archiving. AGU J. Earth Space Sci. 4(7), 396–452 (2017)
Ayhan, B., Kwan, C., Vance, S.: On the use of a linear spectral unmixing technique for concentration estimation of APXS spectrum. J. Multidiscipl. Eng. Sci. Technol. 2(9), 2469–2474 (2015)
Wang, W., Li, S., Qi, H., Ayhan, B., Kwan, C., Vance, S.: Revisiting the preprocessing procedures for elemental concentration estimation based on CHEMCAM LIBS on MARS Rover. In: 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing. (WHISPERS), Lausanne, Switzerland (2014)
Wang, W., Ayhan, B., Kwan, C., Qi, H., Vance, S.: A novel and effective multivariate method for compositional analysis using laser induced breakdown spectroscopy. In: 35th International Symposium on Remote Sensing of Environment (2014)
Dao, M., Kwan, C., Ayhan, B., Bell III, J.F.: Enhancing Mastcam images for Mars Rover mission. In: 14th International Symposium on Neural Networks, Hokkaido (2017)
Ayhan, B., Dao, M., Kwan, C., Chen, H., Bell, J., Kidd, R.: A novel utilization of image registration techniques to process Mastcam images in Mars Rover with applications to image fusion, pixel clustering, and anomaly detection. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 10(10), 4553–4564 (2017)
Kwan, C., Budavari, B., Dao, M., Ayhan, B., Bell, J.: Pansharpening of Mastcam images. In: IEEE International Geoscience and Remote Sensing Symposium, Fort Worth (2017)
Kwan, C., Dao, M., Chou, B., Kwan, L.M., Ayhan, B.: Mastcam image enhancement using estimated point spread functions. In: IEEE Ubiquitous Computing, Electronics and Mobile Communication Conference, New York City (2017)
Qu, Y., Guo, R., Wang, W., Qi, H., Ayhan, B., Kwan, C., Vance, S.: Anomaly detection in hyperspectral images through spectral unmixing and low rank decomposition. In: IEEE International Geoscience and Remote Sensing Symposium. IGARSS), Beijing (2016)
Bayer, B.: Color imaging array, US Patent 3,971,065 (1976)
Li, X., Gunturk, B., Zhang, L.: Image demosaicing: a systematic survey. In: Proceedings of SPIE, vol. 6822, Visual Communications and Image Processing (2008)
Lesson, O., Macaire, L., Yang, Y.: Comparison of color demosaicing methods. In: Hawkes, Peter W. (ed.) Advances in Imaging and Electron Physics, vol. 162, pp. 173–265. Elsevier, Amsterdam (2010)
Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. Imaging 20(2), 023016 (2011)
Malvar, H., He, L.-W., Culter, R.: High-quality linear interpolation for demosaicking of color images. In: Proceedings of IEEE International Conference of Acoustics, Speech and Signal Processing, pp. 485–488 (2004)
Zhang, L., Wu, X.: Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans. Image Process. 14(12), 2167–2178 (2005)
Lu, W., Tan, Y.: Color filter array demosaicking: new method and performance measures. IEEE Trans. Image Process. 12(10), 1194–1210 (2003)
Dubois, E.: Frequency-domain methods for demosaicking of Bayer-sampled color images. IEEE Signal Proc. Lett. 12(12), 847–850 (2005)
Gunturk, B., Altunbasak, Y., Mersereau, R.: Color plane interpolation using alternating projections. IEEE Trans. Image Process. 11(9), 997–1013 (2002)
Wu, X., Zhang, N.: Primary-consistent soft-decision color demosaicking for digital cameras. IEEE Trans. IP 13(9), 1263–1274 (2004)
Bednar, J., Watt, T.: Alpha-trimmed means and their relationship to median filters. IEEE Trans. Acoust. Speech Signal Process. 32(1), 145–153 (1984)
Zhang, X., Wandell, B.: A spatial extension of CIELAB for digital color image reproduction. SID J. 5(1), 61–63 (1997)
Kwan, C., Chou, B., Kwan, L. M., Budavari, B.: Debayering RGBW color filter arrays: a pansharpening approach. In: IEEE Ubiquitous Computing, Electronics and Mobile Communication Conference, New York City (2017)
Acknowledgements
This research was supported by NASA under Contract No. NNX16CP38P. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Kwan, C., Chou, B., Kwan, LY.M. et al. Demosaicing enhancement using pixel-level fusion. SIViP 12, 749–756 (2018). https://doi.org/10.1007/s11760-017-1216-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-017-1216-2