Abstract
A Stacked Denoising Autoencoder (SDAE) is a deep neural network (NN) model trained and designed in one-by-one stacked layers to reconstruct the non-noisy version of the original input data. It is an architecture used with great success in statistical pattern recognition problems.
The objective of this contribution is to determine if an MSDAE can benefit from the greater capabilities of representation of features obtained when two layers are introduced in the stacking process instead of a single one. To do this, the design and performance of these machines in regression problems are presented and analyzed both in terms of error and calculation cost.
The experimental results underline interesting performance capabilities for specific purposes.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103 (2008)
Ashfahani, A., Pratama, M., Lughofer, E., Ong, Y.S.: DEVDAN: deep evolving denoising autoencoder. Neurocomputing 390, 297–314 (2020)
Eiteneuer, B., Hranisavljevic, N., Niggemann, O.: Dimensionality reduction and anomaly detection for cpps data using autoencoder. In: 2019 IEEE International Conference on Industrial Technology (ICIT), pp. 1286–1292 (2019)
Lin, E., Mukherjee, S., Kannan, S.: A deep adversarial variational autoencoder model for dimensionality reduction in single-cell RNA sequencing analysis. BMC Bioinform. 21(1), 1 (2020)
Lim, J., Ryu, S., Kim, J.W., Kim, W.Y.: Molecular generative model based on conditional variational autoencoder for de novo molecular design. J. Cheminform. 10(1), 1–9 (2018)
Mishra, A., Krishna Reddy, S., Mittal, A., Murthy, H.A.: A generative model for zero shot learning using conditional variational autoencoders. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2188–2196 (2018)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)
Alvear-Sandoval, R.F., Figueiras-Vidal, A.R.: On building ensembles of stacked denoising auto-encoding classifiers and their further improvement. Inf. Fusion 39, 41–52 (2018)
Wang, X., Zhao, Y., Pourpanah, F.: Recent advances in deep learning. Int. J. Mach. Learn. Cybern. 11(4), 747–750 (2020). https://doi.org/10.1007/s13042-020-01096-5
Sen, R., Yu, H.F., Dhillon, I.S.: Think globally, act locally: a deep neural network approach to high-dimensional time series forecasting. Adv. Neural Inf. Process. Syst. 2019, 32 (2018)
Dua, D., Graff, C.: UCI Machine Learning Repository. School of Information and Computer Sci., Univ. California at Irvine (2017). http://archive.ics.uci.edu/ml
Clark, D., Schreter, Z., Adams, A.: A quantitative comparison of dystal and backpropagation. In: Proceedings of the 7th Australian Conference on Neural Networks, Australian Nat. Univ., pp. 132–137 (1996)
Yeh, I.C.: Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)
Delve database (1995–1996) Univ. Toronto. https://www.dcc.fc.up.pt/~ltorgo/Regression/DataSets.html. Accessed 10 Dec 2021
Ballester-Ripoll, R., Paredes, E.G., Pajarola, R.: Sobol tensor trains for global sensitivity analysis. Reliab. Eng. Syst. Saf. 183, 311–322 (2019)
De Vito, S., Massera, E., Piga, M., Martinotto, L., Di Francia, G.: On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens. Actuators B Chem. 129, 750–757 (2008)
Szeged Weather database (2016) Kaggle. https://www.kaggle.com/budincsevity/szeged-weather. Accessed 10 Dec 2021
Fernández-García, M.-E., Larrey-Ruiz, J., Ros-Ros, A., Figueiras-Vidal, A.R., Sancho-Gómez, J.-L.: Machine-health application based on machine learning techniques for prediction of valve wear in a manufacturing plant. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Toledo Moreo, J., Adeli, H. (eds.) IWINAC 2019. LNCS, vol. 11487, pp. 389–398. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19651-6_38
Acknowledgements
Thanks to ILBOC, Fundación Séneca (Program 20348/ FPI/17 and Project 20901/PI/18) and Instituto de Salud Carlos III (Project 2018-PI17-00771) for supporting and funding this research work. Part of the experimental results were obtained using the George Mason University Office of Research computing Argo Research Cluster.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Fernández-García, ME., Ros-Ros, A., Hernández, E.H., Figueiras-Vidal, A.R., Sancho-Gómez, JL. (2022). Double-Layer Stacked Denoising Autoencoders for Regression. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence. IWINAC 2022. Lecture Notes in Computer Science, vol 13259. Springer, Cham. https://doi.org/10.1007/978-3-031-06527-9_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-06527-9_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06526-2
Online ISBN: 978-3-031-06527-9
eBook Packages: Computer ScienceComputer Science (R0)