skip to main content
10.1145/3458305.3459594acmconferencesArticle/Chapter ViewAbstractPublication PagesmmsysConference Proceedingsconference-collections
research-article

Enabling hyperspectral imaging in diverse illumination conditions for indoor applications

Published: 15 July 2021 Publication History

Abstract

Hyperspectral imaging provides rich information across many wavelengths of the captured scene, which is useful for many potential applications such as food quality inspection, medical diagnosis, material identification, artwork authentication, and crime scene analysis. However, hyperspectral imaging has not been widely deployed for such indoor applications. In this paper, we address one of the main challenges stifling this wide adoption, which is the strict illumination requirements for hyperspectral cameras. Hyperspectral cameras require a light source that radiates power across a wide range of the electromagnetic spectrum. Such light sources are expensive to setup and operate, and in some cases, they are not possible to use because they could damage important objects in the scene. We propose a data-driven method that enables indoor hyper-spectral imaging using cost-effective and widely available lighting sources such as LED and fluorescent. These common sources, however, introduce significant noise in the hyperspectral bands in the invisible range, which are the most important for the applications. Our proposed method restores the damaged bands using a carefully-designed supervised deep-learning model. We conduct an extensive experimental study to analyze the performance of the proposed method and compare it against the state-of-the-art using real hyperspectral datasets that we have collected. Our results show that the proposed method outperforms the state-of-the-art across all considered objective and subjective metrics, and it produces hyperspectral bands that are close to the ground truth bands captured under ideal illumination conditions.

References

[1]
Dataset and source code. https://github.com/pazadimo/HS_In_Diverse_Illuminations, 2021.
[2]
Mohammad Amin Arab, Puria Azadi Moghadam, Mohamed Hussein, Wael Abd-Almageed, and Mohamed Hefeeda. Revealing true identity: Detecting makeup attacks in face-based biometric systems. In Proc. of ACM Conference on Multimedia (MM'20), page 3568--3576, Seattle, WA, October 2020.
[3]
Seung-Hwan Baek, Incheol Kim, Diego Gutierrez, and Min H. Kim. Compact single-shot hyperspectral imaging using a prism. ACM Transaction on Graphics, 36(6), November 2017.
[4]
Marcus Borengasser, William S Hungate, and Russell Watkins. Hyperspectral Remote Sensing: Principles and Applications. CRC press, 2007.
[5]
Chein-I Chang. An information-theoretic approach to spectral variability, similarity, and discrimination for hyperspectral image analysis. IEEE Transactions on Information Theory, 46(5):1927--1932, 2000.
[6]
Costanza Cucci and Andrea Casini. Chapter 3.8 - hyperspectral imaging for artworks investigation. In José Manuel Amigo, editor, Hyperspectral Imaging, volume 32 of Data Handling in Science and Technology, pages 583--604. Elsevier, 2020.
[7]
Gerda Edelman, Ton G. van Leeuwen, and Maurice C.G. Aalders. Hyperspectral imaging for the age estimation of blood stains at the crime scene. Forensic Science International, 223:72 -- 77, 2012.
[8]
Gamal Elmasry, Mohammed Kamruzzaman, Da-Wen Sun, and Paul Allen. Principles and applications of hyperspectral imaging in quality evaluation of agro-food products: A review. Critical Reviews in Food Science and Nutrition, 52(11):999--1023, 2012.
[9]
C. Elvidge, D. V. Keith, B. Tuttle, and K. Baugh. Spectral identification of lighting type and character. Sensors (Basel, Switzerland), 10:3961 -- 3988, 2010.
[10]
Y. Fu, Y. Zheng, L. Zhang, and H. Huang. Spectral reflectance recovery from a single rgb image. IEEE Transactions on Computational Imaging, 4(3):382--394, 2018.
[11]
B. J. Fubara, M. Sedky, and D. Dyke. Rgb to spectral reconstruction via learned basis functions and weights. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'20), pages 1984--1993, Seattle, WA, June 2020.
[12]
Elham Kordi Ghasrodashti, Azam Karami, Rob Heylen, and Paul Scheunders. Spatial resolution enhancement of hyperspectral images using spectral unmixing and bayesian sparse representation. Remote Sensing, 9(6):541, 2017.
[13]
D. Girod, J. A. Landry, G. Doyon, J. A. Osuna-Garcia, S. Salazar-Garcia, and R. Geonaga. Evaluating hass avocado maturity using hyperspectral imaging. In Proc. of the Caribbean Food Crops Society, Miami, FL, August 2008.
[14]
X. Han, B. Shi, and Y. Zheng. Residual hsrcnn: Residual hyper-spectral reconstruction cnn from an rgb image. In Proc. of International Conference on Pattern Recognition (ICPR'18), pages 2664--2669, Beijing, China, August 2018.
[15]
Mohammed Kamruzzaman, Gamal ElMasry, Da-Wen Sun, and Paul Allen. Non-destructive prediction and visualization of chemical composition in lamb meat using nir hyperspectral imaging and multivariate regression. Innovative Food Science & Emerging Technologies, 16:218--226, 2012.
[16]
A. C. Karaca, A. Ertürk, M. K. Güllü, M. Elmas, and S. Ertürk. Analysis of evidence in forensic documents using hyperspectral imaging system. In Proc. of Signal Processing and Communications Applications Conference (SIU'12), pages 1--4, Istanbul, Turkey, October 2012.
[17]
Shawn C Kefauver, Josep Peñuelas, and Susan L Ustin. Applications of hyperspectral remote sensing and gis for assessing forest health and air pollution. In Proc. of IEEE International Geoscience and Remote Sensing Symposium, pages 3379--3382, 2012.
[18]
F.A. Kruse, A.B. Lefkoff, J.W. Boardman, K.B. Heidebrecht, A.T. Shapiro, P.J. Barloon, and A.F.H. Goetz. The spectral image processing system (sips)---interactive visualization and analysis of imaging spectrometer data. Remote Sensing of Environment, 44(2):145 -- 163, 1993.
[19]
C. Lanaras, E. Baltsavias, and K. Schindler. Hyperspectral super-resolution by coupled spectral unmixing. In Proc. of IEEE International Conference on Computer Vision (ICCV'15), pages 3586--3594, Santiago, Chile, December 2015.
[20]
V. Lempitsky, A. Vedaldi, and D. Ulyanov. Deep image prior. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition(CVPR'18), pages 9446--9454, Salt Lake City, UT, June 2018.
[21]
Marena Manley. Near-infrared spectroscopy and hyperspectral imaging: non-destructive analysis of biological materials. Chemical Society Reviews, 43(24):8200--8214, 2014.
[22]
Patrick M Mehl, Yud-Ren Chen, Moon S Kim, and Diane E Chan. Development of hyperspectral imaging technique for the detection of apple surface defects and contaminations. Journal of Food Engineering, 61(1):67--81, 2004.
[23]
Adam Polak, Timothy Kelman, Paul Murray, Stephen Marshall, David J.M. Stothard, Nicholas Eastaugh, and Francis Eastaugh. Hyperspectral imaging combined with data classification techniques as an aid for artwork authentication. Journal of Cultural Heritage, 26:1 -- 11, 2017.
[24]
Marco AC Potenza, Daniele Nazzari, Llorenç Cremonesi, Ilaria Denti, and Paolo Milani. Hyperspectral imaging with deformable gratings fabricated with metal-elastomer nanocomposites. Review of Scientific Instruments, 88(11), 2017.
[25]
Behnood Rasti, Paul Scheunders, Pedram Ghamisi, Giorgio Licciardi, and Jocelyn Chanussot. Noise reduction in hyperspectral imagery: Overview and application. Remote Sensing, 10(3):482, 2018.
[26]
Stefano Selci. The future of hyperspectral imaging. Journal of Imaging, 5(11), 2019.
[27]
Neha Sharma and Mohamed Hefeeda. Hyperspectral reconstruction from rgb images for vein visualization. In Proc. of ACM Multimedia Systems Conference(MMSys'20), page 77--87, Istanbul, Turkey, June 2020.
[28]
Z. Shi, C. Chen, Z. Xiong, D. Liu, and F. Wu. Hscnn+: Advanced cnn-based hyperspectral recovery from rgb images. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW'18), pages 1052--10528, Salt Lake City, UT, June 2018.
[29]
O. Sidorov and J. Y. Hardeberg. Deep hyperspectral prior: Single-image denoising, inpainting, super-resolution. In Proc. of IEEE International Conference on Computer Vision Workshop (ICCVW'19), pages 3844--3851, Seoul, Korea (South), October 2019.
[30]
J. Snell, K. Ridgeway, R. Liao, B. D. Roads, M. C. Mozer, and R. S. Zemel. Learning to generate images with perceptual similarity metrics. In Proc. of IEEE International Conference on Image Processing (ICIP'17), pages 4277--4281, Beijing, China, September 2017.
[31]
Petra Tatzer, Markus Wolf, and Thomas Panner. Industrial application for inline material sorting using hyperspectral imaging in the nir range. Real-Time Imaging, 11(2):99--107, 2005.
[32]
B. Uzkent, M. J. Hoffman, and A. Vodacek. Real-time vehicle tracking in aerial video using hyperspectral features. Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'16), pages 1443--1451, July 2016.
[33]
Nan-Nan Wang, Da-Wen Sun, Yi-Chao Yang, Hongbin Pu, and Zhiwei Zhu. Recent advances in the application of hyperspectral imaging for evaluating fruit quality. Food analytical methods, 9(1):178--191, 2016.
[34]
K. Wei, Y. Fu, and H. Huang. 3-d quasi-recurrent neural network for hyperspectral image denoising. IEEE Transactions on Neural Networks and Learning Systems, 32(1):363--375, 2021.
[35]
Di Wu and Da-Wen Sun. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review---part i: Fundamentals. Innovative Food Science & Emerging Technologies, 19:1--14, 2013.
[36]
Hongyan Zhu, Bingquan Chu, Yangyang Fan, Xiaoya Tao, Wenxin Yin, and Yong He. Hyperspectral imaging for predicting the internal quality of kiwifruits based on variable selection algorithms and chemometric models. Scientific Reports, 7(1):1--13, 2017.

Cited By

View all
  • (2023)MobiSpectral: Hyperspectral Imaging on Mobile DevicesProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613296(1-15)Online publication date: 2-Oct-2023

Index Terms

  1. Enabling hyperspectral imaging in diverse illumination conditions for indoor applications

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MMSys '21: Proceedings of the 12th ACM Multimedia Systems Conference
      June 2021
      254 pages
      ISBN:9781450384346
      DOI:10.1145/3458305
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      In-Cooperation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 July 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Badges

      Author Tags

      1. deep learning
      2. hyperspectral imaging
      3. illumination

      Qualifiers

      • Research-article

      Funding Sources

      • NSERC

      Conference

      MMSys '21
      Sponsor:
      MMSys '21: 12th ACM Multimedia Systems Conference
      September 28 - October 1, 2021
      Istanbul, Turkey

      Acceptance Rates

      MMSys '21 Paper Acceptance Rate 18 of 55 submissions, 33%;
      Overall Acceptance Rate 176 of 530 submissions, 33%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)43
      • Downloads (Last 6 weeks)3
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)MobiSpectral: Hyperspectral Imaging on Mobile DevicesProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613296(1-15)Online publication date: 2-Oct-2023

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media