Mayank Goel
Alex Mariakakis
Dan Morris
Gaetano Borriello
Shwetak N. Patel
ABSTRACT
Emerging uses of imaging technology for consumers cover a wide range of application areas from health to interaction techniques; however, typical cameras primarily transduce light from the visible spectrum into only three overlapping components of the spectrum: red, blue, and green. In contrast, hyperspectral imaging breaks down the electromagnetic spectrum into more narrow components and expands coverage beyond the visible spectrum. While hyperspectral imaging has proven useful as an industrial technology, its use as a sensing approach has been fragmented and largely neglected by the UbiComp community. We explore an approach to make hyperspectral imaging easier and bring it closer to the end-users. HyperCam provides a low-cost implementation of a multispectral camera and a software approach that automatically analyzes the scene and provides a user with an optimal set of images that try to capture the salient information of the scene. We present a number of use-cases that demonstrate HyperCam's usefulness and effectiveness.
Supplemental Material
- Azuma, R., Behringer, R., Feiner, S., Julier, S., and Macintyre, B. Recent Advances in Augmented Reality. IEEE Computer Graphics and Applications 21, 6., December (2001). Google Scholar
Digital Library
- Baronti, S., Casini, a, Lotti, F., and Porcinai, S. Multispectral Imaging System for the Mapping of Pigments in Works of Art by use of Principal-Component Analysis. Applied Optics 37, 8 (1998), 1299--309.Google ScholarCross Ref
- Van Beers, R., Aernouts, B., De Baerdemaeker, J., and Saeys, W. Apple ripeness detection using hyperspectral laser scatter imaging. Sensing Technologies for Biomaterial, Food, and Agriculture 8881, (2013).Google Scholar
- Burns, D. A. and Ciurczak, E. W. Handbook of Near-Infrared Analysis. 2007.Google ScholarCross Ref
- Coffey, V. C. Multispectral Imaging Moves into the Mainstream. OPN Optics & Photonics News, 18--24.Google Scholar
- Cuper, N. J., Verdaasdonk, R. M., de Roode, R., et al. Visualizing veins with near-infrared light to facilitate blood withdrawal in children. Clinical pediatrics 50, 6 (2011), 508--12.Google Scholar
- Friedhelm König and Praefcke, W. The Practice of Multispectral Image Acquisition. Proceedings of SPIE vol. 3409, EUROPTO Conference on Electronic Imaging: Processing, Printing, and Publishing in Color, (1998).Google Scholar
- Goltz, D. M., Cloutis, E., Norman, L., and Attas, M. Enhancement of Faint Text Using Visible (420--720 nm) Multispectral Imaging. Restaurator 28, 1 (2007), 11--28.Google Scholar
- De Greef, L., Goel, M., Seo, M. J., et al. BiliCam: Using Mobile Phones to Monitor Newborn Jaundice. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing 2014. Google ScholarDigital Library
- Heinemann, L. and Schmelzeisen-redeker, G. Non-invasive continuous glucose monitoring in Type I diabetic patients with optical glucose sensors. Diabetologia (1998) 41, 1998, 848--854.Google ScholarCross Ref
- Ho, B., Kao, H. C., Chen, N., et al. HeatProbe: A Thermal-based Power Meter for Accounting Disaggregated Electricity Usage. Proceedings of the UbiComp 2011., (2011). Google ScholarDigital Library
- Holz, C. and Baudisch, P. Fiberio: A Touchscreen that Senses Fingerprints. Proceedings of the 2013 ACM conference on UIST 2013. Google ScholarDigital Library
- Imai, F. H. and Berns, R. S. Spectral estimation using trichromatic digital cameras. Proc. of the International Symposium on Multispectral Imaging and Color Reproduction for Digital Archives, (2012).Google Scholar
- Kanzawa, Y. Human Skin Detection by Visible and Near-Infrared Imaging. IAPR Conference on Machine Vision Applications, (2011), 503--507.Google Scholar
- Kapsokalyvas, D., Bruscino, N., Alfieri, D., et al. Imaging of human skin lesions with the Multispectral Dermoscope. SPIE Proceedings 7548, (2010), 754808-754808-8.Google Scholar
- Kargel, J. S., Abrams, M. J., Bishop, M. P., et al. Multispectral imaging contributions to global land ice measurements from space. Remote Sensing of Environment 99, 1--2 (2005), 187--219.Google ScholarCross Ref
- Kim, D., Hilliges, O., Izadi, S., et al. Digits: Freehand 3D Interactions Anywhere Using a Wrist-Worn Gloveless Sensor. Proceedings of the ACM Conference of UIST 2012. Google ScholarDigital Library
- Kim, Y.-J. and Yoon, G. Prediction of glucose in whole blood by near-infrared spectroscopy: influence of wavelength region, preprocessing, and hemoglobin concentration. Journal of biomedical optics 11, 4 (2013), 041128.Google Scholar
- Kong, S. G., Heo, J., Boughorbel, F., et al. Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition. International Journal of Computer Vision 71, 2 (2006), 215--233. Google ScholarDigital Library
- Larson, E., Cohn, G., Gupta, S., et al. HeatWave: Thermal Imaging for Surface User Interaction. Proc. CHI 2011. Google ScholarDigital Library
- Levenson, R. M. and Mansfield, J. R. Multispectral Imaging in Biology and Medicine: Slices of Life. 2006 International Society for Analytical Cytology 758, (2006), 748--758.Google Scholar
- Li, H., Feng, J., Yang, W., et al. Multi-spectral imaging using LED illuminations. 2012 5th International Congress on Image and Signal Processing, Cisp (2012), 538--542.Google ScholarCross Ref
- Lu, R. Predicting Firmness and Sugar Content of Sweet Cherries Using Near-Infrared Diffuse Reflectance Spectroscopy. Transactions of the American Society of Agricultural Engineers 44, 5 (2001), 1265--1271.Google Scholar
- Lu, R. Multispectral imaging for predicting firmness and soluble solids content of apple fruit. Postharvest Biology and Technology 31, 2 (2004), 147--157.Google ScholarCross Ref
- Matsushita, N. and Rekimoto, J. HoloWall: Designing a Finger; Hand, Body, and Object Sensitive Wall. Proc. UIST'97. Google ScholarDigital Library
- Murthy, G. R. S. and Jadon, R. S. A review of vision based hand gestures recognition. International Journal of Information Technology and Knowledge Management 2, 2 (2009), 405--410.Google Scholar
- Murugappan, S., Elmqvist, N., and Ramani, K. Extended Multitouch: Recovering Touch Posture and Differentiating Users using a Depth Camera. Proc. UIST 2012. Google ScholarDigital Library
- Reinhard, E., Ward, G., Pattanaik, S., Debevec, P., Heidrich, W., and Myszkowski, K. High Dynamic Range Imaging. 2010.Google Scholar
- Saltveit, M. E. Effect of ethylene on quality of fresh fruits and vegetables. Postharvest Biology and Technology 15, June 1998, 279--292.Google ScholarCross Ref
- Shirazi, A. S., Abdelrahman, Y., Henze, N., Schneegass, S., Khalilbeigi, M., and Schmidt, A. Exploiting Thermal Reflection for Interactive Systems. Proc. CHI 2014, (2014). Google ScholarDigital Library
- Smith, M. O., Ustin, S. L., Adams, J. B., and Gillespie, A. R. Vegetation in Deserts: I. A Regional Measure of Abundance from Multispectral Images. Int. Symp. Remote Sens. Environ. 26, February (1990), 1--26.Google ScholarCross Ref
- Sobel, I. and Feldman, G. A 3x3 isotropic gradient operator for image processing. 1968.Google Scholar
- Song, K.-Y., Oh, H.-J., Choi, J., Park, I., Lee, C., and Lee, S. Prediction of landslides using ASTER imagery and data mining models. Advances in Space Research 49, 5 (2012), 978--993.Google ScholarCross Ref
- Wadhawan, T., Lancaster, K., and Zouridakis, G. Implementation of the 7-point checklist for melanoma detection on smart handheld devices. 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (2011), 3180--3183.Google ScholarCross Ref
- Wang, L., Leedham, G., and Cho, S. Infrared imaging of hand vein patterns for biometric purposes. The Institution of Engineering and Technology, (2007), 113--122.Google ScholarCross Ref
- Wilson, A. D. PlayAnywhere: A Compact Interactive Tabletop Projection-Vision System. Proc. UIST'05, (2005). Google ScholarDigital Library
- Yoon, H., Yang, K., and Shahabi, C. Feature subset selection and feature ranking for multivariate time series. IEEE Transactions on Knowledge and Data Engineering 17, 9 (2005), 1186--1198. Google ScholarDigital Library
- Realization of natural color reproduction in Digital Still Cameras, closer to the natural sight perception of the human eye. http://www.sony.net/SonyInfo/News/Press_Archive/200 307/03-029E/.Google Scholar
- OV4682 Color CMOS 4-megapixel (2688x1520) Image Sensor with OmniBSI-2™ Technology. http://www.ovt.com/products/sensor.php?id=145&limit=302.Google Scholar
- Hyperspectral Imaging. http://www.hyspex.no/hyperspectral_imaging/.Google Scholar
Index Terms
- HyperCam: hyperspectral imaging for ubiquitous computing applications
Recommendations
Efficient multispectral facial capture with monochrome cameras
SIGGRAPH '18: ACM SIGGRAPH 2018 PostersWe propose a variant to polarized gradient illumination facial scanning which uses monochrome instead of color cameras to achieve more efficient and higher-resolution results. In typical polarized gradient facial scanning, sub-millimeter geometric ...
A Prism-Mask System for Multispectral Video Acquisition
This paper presents a prism-mask system for capturing multispectral videos. The system is composed of a triangular prism, a monochrome camera, and an occlusion mask. Incoming light beams from the scene are sampled by the occlusion mask, dispersed into ...
A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor
Single-sensor imaging using the Bayer color filter array (CFA) and demosaicking is well established for current compact and low-cost color digital cameras. An extension from the CFA to a multispectral filter array (MSFA) enables us to acquire a ...
Comments