Skip to main content
SpringerLink
Log in

A Patch Aware Multiple Dictionary Framework for Demosaicing

  • Conference paper
  • First Online:
Computer Vision -- ACCV 2014 (ACCV 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9005))

Included in the following conference series:

Abstract

Most digital cameras rely on demosaicing algorithms to restore true color images. Data captured by these cameras is reduced by two thirds because a Color Filter Array (CFA) allows only one particular channel go through at each pixel. This paper proposes a Patch Aware Multiple Dictionary (PAMD) framework for demosaicing. Instead of using a common dictionary, multiple dictionaries are trained for different classes of signals. These class-specific dictionaries comprise a super overcomplete dictionary. The most suitable dictionary would be adaptively selected based on the patch class, determined by the Energy Exclusiveness Feature (EEF) which measures the degree of energy domination in the representation code. In this way, candidate atoms are constrained in a set of atoms with low correlations; and meanwhile, the signal would have sparser representation over this adapted dictionary than over the common one, making the fixed sample rate relatively high and thus, accomplishing satisfying restoration according to the compressive sensing theory. Extensive experiments demonstrated that PAMD outperforms traditional Single Dictionary (SD) based approach as well as leading algorithms in diffusion-based family significantly, with respect to both PSNR and visual quality. Especially, the general artifact by Bayer CFA, Moiré Pattern, is dramatically reduced. Furthermore, on several images, it also significantly outperforms the state-of-the-art algorithms which are also sparse coding based but very complicated. PAMD is a general framework which can cooperate with existing demosaicing algorithms based on sparse coding.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In the setting of Bayer “RGGB” CFA, if the sliding-window moves forward by odd number pixel(s) at each step, then there would be 4 patterns for \(\varvec{M}_t\) as “RGGB”, “GRBG”, “GBRG”, and “BGGR”. However, if by even number pixels per step, there would be only one pattern as “RGGB”, then in this case, we do not have to first restore the whole image using \(\varvec{D}_{all}\) in Algorithm 1.

References

  1. Moghadam, A., Aghagolzadeh, M., Kumar, M., Radha, H.: Compressive framework for demosaicing of natural images. IEEE Trans. Image Process. 22, 2356–2371 (2013)

    Article  MathSciNet  Google Scholar 

  2. Shao, L., Rehman, A.U.: Image demosaicing using content and colour-correlation analysis. Signal Process. 103, 84–91 (2013)

    Article  Google Scholar 

  3. Rehman, U.: A., Shao, L.: Classification-based de-mosaicing for digital cameras. Neurocomputing 83, 222–228 (2012)

    Article  Google Scholar 

  4. Lu, Y., Karzand, M., Vetterli, M.: Demosaicking by alternating projections: theory and fast one-step implementation. IEEE Trans. Image Process. 19, 2085–2098 (2010)

    Article  MathSciNet  Google Scholar 

  5. Gunturk, B., Glotzbach, J., Altunbasak, Y., Schafer, R., Mersereau, R.: Demosaicking: color filter array interpolation. IEEE Signal Process. Mag. 22, 44–54 (2005)

    Article  Google Scholar 

  6. Gunturk, B., Altunbasak, Y., Mersereau, R.: Color plane interpolation using alternating projections. IEEE Trans. Image Process. 11, 997–1013 (2002)

    Article  Google Scholar 

  7. Menon, D., Calvagno, G.: Regularization approaches to demosaicking. IEEE Trans. Image Process. 18, 2209–2220 (2009)

    Article  MathSciNet  Google Scholar 

  8. Ferradans, S., Bertalmío, M., Caselles, V.: Geometry-based demosaicking. IEEE Trans. Image Process. 18, 665–670 (2009)

    Article  MathSciNet  Google Scholar 

  9. Malvar, H.S., He, L.w., Cutler, R.: High-quality linear interpolation for demosaicing of bayer-patterned color images. In: 2004. Proceedings IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004), vol. 3. IEEE (2004)

    Google Scholar 

  10. Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: IEEE 12th International Conference on Computer Vision, pp. 2272–2279 (2009)

    Google Scholar 

  11. Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17, 53–69 (2008)

    Article  MathSciNet  Google Scholar 

  12. Zhang, D., Wu, X.: Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans. Image Process. 14, 2167–2178 (2005)

    Article  Google Scholar 

  13. Chung, K.H., Chan, Y.H.: Color demosaicing using variance of color differences. IEEE Trans. Image Process. 15, 2944–2955 (2006)

    Article  Google Scholar 

  14. Tanaka, M., Okutomi, M.: Color Kernel regression for robust direct upsampling from raw data of general color filter array. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 290–301. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  15. Candes, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  16. Candes, E., Wakin, M.: An introduction to compressive sampling. IEEE Signal Process. Mag. 25, 21–30 (2008)

    Article  Google Scholar 

  17. Zhang, M., Tao, L.: Volume reconstruction for MRI. In: 2014 22nd International Conference on Pattern Recognition (ICPR), ICPR 2014, pp. 3351–3356. IEEE (2014)

    Google Scholar 

  18. Mohimani, H., Babaie-Zadeh, M., Jutten, C.: A fast approach for overcomplete sparse decomposition based on smoothed \(\ell ^{0}\) norm. IEEE Trans. Signal Process. 57, 289–301 (2009)

    Article  MathSciNet  Google Scholar 

  19. Tropp, J., Gilbert, A.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53, 4655–4666 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  20. Candes, E., Romberg, J.: l1-magic: Recovery of sparse signals via convex programming (2005). www.acm.caltech.edu/l1magic/downloads/l1magic.pdf

  21. Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 801–808. MIT Press, Cambridge (2007)

    Google Scholar 

  22. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B (Methodolog.) 58, 267–288 (1996)

    MATH  MathSciNet  Google Scholar 

  23. Aharon, M., Elad, M., Bruckstein, A.: k-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 4311–4322 (2006)

    Article  Google Scholar 

  24. Dhillon, I.S., Sra, S.: Generalized nonnegative matrix approximations with bregman divergences. In: Neural Information Processing Systems, pp. 283–290 (2005)

    Google Scholar 

  25. Bach, F.R., Jordan, M.I.: Kernel independent component analysis. J. Mach. Learn. Res. 3, 1–48 (2003)

    MATH  MathSciNet  Google Scholar 

  26. Filipovic, M., Kopriva, I., Cichocki, A.: Inpainting color images in learned dictionary. In: 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO), pp. 66–70. IEEE (2012)

    Google Scholar 

  27. Paliy, D., Katkovnik, V., Bilcu, R., Alenius, S., Egiazarian, K.: Spatially adaptive color filter array interpolation for noiseless and noisy data. Int. J. Imaging Syst. Technol. 17, 105–122 (2007)

    Article  Google Scholar 

  28. Menon, D., Calvagno, G.: Regularization approaches to demosaicking. IEEE Trans. Image Process. 18, 2209–2220 (2009)

    Article  MathSciNet  Google Scholar 

  29. Li, X., Gunturk, B., Zhang, L.: Image demosaicing: a systematic survey. In: Electronic Imaging 2008, International Society for Optics and Photonics, pp. 68221J–68221J (2008)

    Google Scholar 

  30. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  31. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: ICML, p. 87 (2009)

    Google Scholar 

  32. Olmos, A., et al.: A biologically inspired algorithm for the recovery of shading and reflectance images. Perception 33, 1463–1473 (2003)

    Article  Google Scholar 

  33. Hyvärinen, A., Oja, E.: A fast fixed-point algorithm for independent component analysis. Neural Comput. 9, 1483–1492 (1997)

    Article  Google Scholar 

  34. Kodak: Kodak lossless true color image suite (2004). http://r0k.us/graphics/kodak

  35. Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. Imaging 20, 023016 (2011)

    Article  Google Scholar 

  36. Kiku, D., Monno, Y., Tanaka, M., Okutomi, M.: Residual interpolation for color image demosaicking. In: IEEE International Conference on Image Processing, ICIP 2013, pp. 2304–2308, Melbourne, Australia, 15–18 September 2013

    Google Scholar 

Download references

Acknowledgement

This work was supported in part by the National Natural Science Foundation of China under Grants 61272232 and MOST under Grants 2012AA011602.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, China

    Meiqing Zhang & Linmi Tao

Authors
  1. Meiqing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linmi Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meiqing Zhang .

Editor information

Editors and Affiliations

  1. Technische Universität München, Garching, Bayern, Germany

    Daniel Cremers

  2. University of Adelaide, Adelaide, South Australia, Australia

    Ian Reid

  3. Keio University, Yokohama, Kanagawa, Japan

    Hideo Saito

  4. University of California at Merced, Merced, California, USA

    Ming-Hsuan Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Zhang, M., Tao, L. (2015). A Patch Aware Multiple Dictionary Framework for Demosaicing. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9005. Springer, Cham. https://doi.org/10.1007/978-3-319-16811-1_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16811-1_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16810-4

  • Online ISBN: 978-3-319-16811-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation