Skip to main content
SpringerLink
Log in

Efficient spectral reconstruction using a trichromatic camera via sample optimization

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Training-based multispectral reconstruction can effectively recover spectral reflectance of captured objects using a trichromatic camera. However, existing methods are based on synthesized data, and the sizes of training sample set (e.g., multispectral images, reflectance targets) are usually large. In this paper, we present a spectral reconstruction approach using real measured data. To improve the efficiency and accuracy of spectral reconstruction, we propose a volume maximization method for sample optimization without any prior knowledge of light and cameras. We use heuristic global search algorithms to optimize samples and give an efficient spectral reconstruction method which is suitable for sparse sampling. Experimental results show that the proposed sample selection method outperforms other existing methods in terms of both spectral and colorimetric reconstruction errors. Moreover, the proposed reconstruction method achieves higher efficiency and accuracy due to lower sample redundancy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Agahian, F., Amirshahi, S.A., Amirshahi, S.H.: Reconstruction of reflectance spectra using weighted principal component analysis. Color Res. Appl. 33(5), 360–371 (2008)

    Article  Google Scholar 

  2. Aharon, M., Elad, M., Bruckstein, A.: K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  3. Alvarez-Cortes, S., Kunkel, T., Masia, B.: Practical low-cost recovery of spectral power distributions. In: Computer Graphics Forum, vol. 35, pp. 166–178. Wiley (2016)

  4. Arad, B., Ben-Shahar, O.: Sparse recovery of hyperspectral signal from natural rgb images. In: European Conference on Computer Vision, pp. 19–34. Springer (2016)

  5. Brill, M.H.: Acquisition and reproduction of color images: colorimetric and multispectral approaches. Color Res. Appl. 27(4), 304–305 (2002)

    Article  Google Scholar 

  6. Brown, M., Süsstrunk, S.: Multi-spectral sift for scene category recognition. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 177–184. IEEE (2011)

  7. Carroll, R., Ramamoorthi, R., Agrawala, M.: Illumination decomposition for material recoloring with consistent interreflections. In: ACM Transactions on Graphics (TOG), vol. 30, p. 43. ACM (2011)

    Article  Google Scholar 

  8. Cheung, V., Westland, S.: Methods for optimal color selection. J. Imaging Sci. Technol. 50(5), 481–488 (2006)

    Article  Google Scholar 

  9. Çivril, A., Magdon-Ismail, M.: On selecting a maximum volume sub-matrix of a matrix and related problems. Theor. Comput. Sci. 410(47), 4801–4811 (2009)

    Article  MathSciNet  Google Scholar 

  10. Fu, Y., Zheng, Y., Sato, I., Sato, Y.: Exploiting spectral-spatial correlation for coded hyperspectral image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3727–3736 (2016)

  11. Han, S., Sato, I., Okabe, T., Sato, Y.: Fast spectral reflectance recovery using dlp projector. Int. J. Comput. Vis. 110(2), 172–184 (2014)

    Article  Google Scholar 

  12. Heikkinen, V., Cámara, C., Hirvonen, T., Penttinen, N.: Spectral imaging using consumer-level devices and kernel-based regression. JOSA A 33(6), 1095–1110 (2016)

    Article  Google Scholar 

  13. Heikkinen, V., Jetsu, T., Parkkinen, J., Hauta-Kasari, M., Jaaskelainen, T., Lee, S.D.: Regularized learning framework in the estimation of reflectance spectra from camera responses. JOSA A 24(9), 2673–2683 (2007)

    Article  Google Scholar 

  14. Iizuka, S., Simo-Serra, E., Ishikawa, H.: Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016), vol. 35, No. 4, pp. 110:1–110:11 (2016)

    Article  Google Scholar 

  15. Jiang, J., Liu, D., Gu, J., Süsstrunk, S.: Camera spectral sensitivity. http://www.cis.rit.edu/dxl5849/projects/camspec/

  16. Kalantari, N.K., Ramamoorthi, R.: Deep high dynamic range imaging of dynamic scenes. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH 2017), vol. 36, No. 4 (2017)

    Article  Google Scholar 

  17. Kim, S., Min, D., Ham, B., Do, M., Sohn, K.: Dasc: Robust dense descriptor for multi-modal and multi-spectral correspondence estimation. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1712–1729 (2016)

    Article  Google Scholar 

  18. Lan, Y., Wang, J., Lin, S., Gong, M., Tong, X., Guo, B.: Interactive chromaticity mapping for multispectral images. Visual Comput. 29(6–8), 773–783 (2013)

    Article  Google Scholar 

  19. Lee, M.H., Park, H., Ryu, I., Park, J.I.: Fast model-based multispectral imaging using nonnegative principal component analysis. Opt. Lett. 37(11), 1937–1939 (2012)

    Article  Google Scholar 

  20. Li, Y., Majumder, A., Lu, D., Gopi, M.: Content-independent multi-spectral display using superimposed projections. Comput. Graph. Forum 34(2), 337–348 (2015)

    Article  Google Scholar 

  21. Liu, C., Yuen, J., Torralba, A.: Sift flow: dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)

    Article  Google Scholar 

  22. Mansouri, A., Sliwa, T., Hardeberg, J.Y., Voisin, Y.: Representation and estimation of spectral reflectances using projection on pca and wavelet bases. Color Res. Appl. 33(6), 485–493 (2008)

    Article  Google Scholar 

  23. Melanie, M.: An introduction to genetic algorithms. Cambridge, Massachusetts London, England, Fifth printing vol. 3, pp. 62–75 (1999)

  24. Mohammadi, M., Nezamabadi, M., Berns, R.S., Taplin, L.A.: Spectral imaging target development based on hierarchical cluster analysis. In: Color and Imaging Conference, vol. 2004, pp. 59–64. Society for Imaging Science and Technology (2004)

  25. Nalbach, O., Seidel, H.P., Ritschel, T.: Practical capture and reproduction of phosphorescent appearance. In: Computer Graphics Forum, vol. 36, pp. 409–420. Wiley (2017)

  26. Nguyen, R.M., Prasad, D.K., Brown, M.S.: Training-based spectral reconstruction from a single rgb image. In: European Conference on Computer Vision, pp. 186–201. Springer (2014)

  27. Park, J.I., Lee, M.H., Grossberg, M.D., Nayar, S.K.: Multispectral imaging using multiplexed illumination. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, pp. 1–8. IEEE (2007)

  28. Parkkinen, J.P., Hallikainen, J., Jaaskelainen, T.: Characteristic spectra of munsell colors. JOSA A 6(2), 318–322 (1989)

    Article  Google Scholar 

  29. Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: 1993 Conference Record of The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, 1993, pp. 40–44. IEEE (1993)

  30. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  31. Shen, H.L., Yao, J.F., Li, C., Du, X., Shao, S.J., Xin, J.H.: Channel selection for multispectral color imaging using binary differential evolution. Appl. Opt. 53(4), 634–642 (2014)

    Article  Google Scholar 

  32. Shen, H.L., Zhang, H.G., Xin, J.H., Shao, S.J.: Optimal selection of representative colors for spectral reflectance reconstruction in a multispectral imaging system. Appl. Opt. 47(13), 2494–2502 (2008)

    Article  Google Scholar 

  33. University of Joensuu Color Group: Spectral Database. http://cs.joensuu.fi/spectral/databases/download/

  34. Wagadarikar, A., John, R., Willett, R., Brady, D.: Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 47(10), B44–B51 (2008)

    Article  Google Scholar 

  35. Wug O.S., Brown, M.S., Pollefeys, M., Joo K.S.: Do it yourself hyperspectral imaging with everyday digital cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2461–2469 (2016)

  36. Yasuma, F., Mitsunaga, T., Iso, D., Nayar, S.K.: Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Trans. Image Process. 19(9), 2241–2253 (2010)

    Article  MathSciNet  Google Scholar 

  37. Zhang, L., Li, B., Pan, Z., Liang, D., Kang, Y., Zhang, D., Ma, X.: A method for selecting training samples based on camera response. Laser Phys. Lett. 13(9), 095201 (2016)

    Article  Google Scholar 

  38. Zhang, Q., Zheng, G., Zhou, D.: Comparison study of gauss, mq and tps for interpolation application. Int. J. Ind. Syst. Eng. 18(2), 185–198 (2014)

    Google Scholar 

  39. Zhang, W.F., Tang, G., Dai, D.Q., Nehorai, A.: Estimation of reflectance from camera responses by the regularized local linear model. Opt. Lett. 36(19), 3933–3935 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The work in this paper has been supported by the National Natural Science Foundation of China (Grant Nos. 61602268, 61603202, 61571247), the National Natural Science Foundation of Zhejiang Province (Nos. LY13F020050, LZ16F030001), and the K.C.Wong Magna Fund in Ningbo University.

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Ningbo University, 818 Fenghua Road, Ningbo, 315211, Zhejiang, China

    Yuqi Li, Chong Wang & Jieyu Zhao

  2. College of Computer Science and Technology, Zhejiang University, 38 Zheda Road, Ningbo, 310000, Zhejiang, China

    Yuqi Li

  3. Hangzhou Institute of Service Engineering, Hangzhou Normal University, Hangzhou, 310000, Zhejiang, China

    Qingshu Yuan

Authors
  1. Yuqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jieyu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingshu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuqi Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Wang, C., Zhao, J. et al. Efficient spectral reconstruction using a trichromatic camera via sample optimization. Vis Comput 34, 1773–1783 (2018). https://doi.org/10.1007/s00371-017-1469-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-017-1469-3

Keywords

Search

Navigation