Abstract
Hyperspectral images are data cubes that offer very rich spectral and spatial resolutions. These images are so highly dimensioned that we generally reduce them in a pre-processing step in order to process them efficiently. In this context, Local Fisher Discriminant Analysis (LFDA) is a feature extraction technique that proved better than several commonly used dimensionality reduction techniques. However, this method suffers from memory problems and long execution times on commodity hardware. In this paper, to solve these problems, we first added an optimization step to LFDA to make it executable on commodity hardware and to make it suitable for parallel and distributed computing, then, we implemented it in a parallel and distributed way using Apache Spark. We tested our implementation on Amazon Web Services (AWS)’s Elastic MapReduce (EMR) clusters, using different hyperspectral images with different sizes. This proved higher performances with a speedup of up to 70x.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available online at www.ms.k.u-tokyo.ac.jp/software.html#LFDA.
- 2.
Amazon Web Services (AWS) - Cloud Computing Services (aws.amazon.com).
- 3.
Amazon S3 aws.amazon.com/s3.
- 4.
Available online at www.github.com/boto/boto3.
- 5.
Amazon EMR aws.amazon.com/emr.
- 6.
Amazon EC2 Instance Types (aws.amazon.com/ec2/instance-types).
- 7.
Apache Hadoop YARN - Yet Another Resource Negotiator hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/YARN.html.
References
Wu, Z., Yonglong, L., Antonio, P., Li, J., Xiao, F., Wei, Z.: Parallel and distributed dimensionality reduction of hyperspectral data on cloud computing architectures. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 9(6), 2270–2278 (2016)
Haut, J.M., Paoletti, M., Plaza, J., Plaza, A.: Cloud implementation of the k-means algorithm for hyperspectral image analysis. J. Supercomput. 73(1), 514–529 (2017)
Quirita, V.A.A., et al.: A new cloud computing architecture for the classification of remote sensing data. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 10(2), 409–416 (2017)
Yin, X., Wu, Z., Liao, W., Wei, Z., Tan, C.: Cloud implementation of hyperspectral image restoration with PCA and total variation based on spark. In: 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 3405–3408. IEEE (2017)
Sugiyama, M.: Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J. Mach. Learn. Res. 8, 1027–1061 (2007)
Shen, P., Lu, X., Liu, L., Kawai, H.: Local fisher discriminant analysis for spoken language identification. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5825–5829. IEEE (2016)
Chen, H.-L., Liu, D.-Y., Yang, B., Liu, J., Wang, G.: A new hybrid method based on local fisher discriminant analysis and support vector machines for hepatitis disease diagnosis. Expert. Syst. Appl. 38(9), 11796–11803 (2011)
Pedagadi, S., Orwell, J., Velastin, S., Boghossian, B.: Local fisher discriminant analysis for pedestrian re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3318–3325 (2013)
Zhang, S., Lei, B., Chen, A., Chen, C., Chen, Y.: Spoken emotion recognition using local fisher discriminant analysis. In: 2010 IEEE 10th International Conference on Signal Processing (ICSP), pp. 538–540. IEEE (2010)
Rahulamathavan, Y., Phan, R.C.-W., Chambers, J.A., Parish, D.J.: Facial expression recognition in the encrypted domain based on local fisher discriminant analysis. IEEE Trans. Affect. Comput. 4(1), 83–92 (2013)
Guo, J., Chen, H., Li, Y.: Palmprint recognition based on local fisher discriminant analysis. JSW 9(2), 287–292 (2014)
Li, W., Prasad, S., Fowler, J.E., Bruce, L.M.: Locality-preserving dimensionality reduction and classification for hyperspectral image analysis. IEEE Trans. Geosci. Remote. Sens. 50(4), 1185–1198 (2012)
Li, W., Prasad, S., Fowler, J.E.: Hyperspectral image classification using gaussian mixture models and markov random fields. IEEE Geosci. Remote. Sens. Lett. 11(1), 153–157 (2014)
Zaatour, R., Bouzidi, S., Zagrouba, E.: Impact of feature extraction and feature selection techniques on extended attribute profile-based hyperspectral image classification. In: Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, vol.4, VISAPP, (VISIGRAPP 2017), pp. 579–586. INSTICC, ScitePress (2017). ISBN 978-989-758-225-7, https://doi.org/10.5220/0006171305790586
Zaharia, M.: Fast and interactive analytics over hadoop data with spark. Usenix Login 37(4), 45–51 (2012)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)
Niyogi, X.: Locality preserving projections. In: Neural Information Processing Systems, vol. 16, p. 153. MIT (2004)
Oliphant, T.E.: Scipy: open source scientific tools for python. Comput. Sci. Eng. 9, 10–20 (2007)
Oliphant, Travis E.: A Guide to NumPy, vol. 1. Trelgol Publishing USA (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zaatour, R., Bouzidi, S., Zagrouba, E. (2018). Parallel and Distributed Local Fisher Discriminant Analysis to Reduce Hyperspectral Images on Cloud Computing Architectures. In: Blanc-Talon, J., Helbert, D., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2018. Lecture Notes in Computer Science(), vol 11182. Springer, Cham. https://doi.org/10.1007/978-3-030-01449-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-01449-0_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01448-3
Online ISBN: 978-3-030-01449-0
eBook Packages: Computer ScienceComputer Science (R0)