Skip to main content
SpringerLink
Log in

Selection of optimal denoising-based regularization hyper-parameters for performance improvement in a sensor validation model

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Multilayered auto-associative neural architectures have widely been used in empirical sensor modeling. Typically, such empirical sensor models are used in sensor calibration and fault monitoring systems. However, simultaneous optimization of related performance metrics, i.e., auto-sensitivity, cross-sensitivity, and fault-detectability, is not a trivial task. Learning procedures for parametric and other relevant non-parametric empirical models are sensitive to optimization and regularization methods. Therefore, there is a need for active learning strategies that can better exploit the underlying statistical structure among input sensors and are simple to regularize and fine-tune. To this end, we investigated the greedy layer-wise learning strategy and denoising-based regularization procedure for sensor model optimization. We further explored the effects of denoising-based regularization hyper-parameters such as noise-type and noise-level on sensor model performance and suggested optimal settings through rigorous experimentation. A visualization procedure was introduced to obtain insight into the internal semantics of the learned model. These visualizations allowed us to suggest an implicit noise-generating process for efficient regularization in higher-order layers. We found that the greedy-learning procedure improved the overall robustness of the sensor model. To keep experimentation unbiased and immune to noise-related artifacts in real sensors, the sensor data were sampled from simulators of a nuclear steam supply system of a pressurized water reactor and a Tennessee Eastman chemical process. Finally, we compared the performance of an optimally regularized sensor model with auto-associative neural network, auto-associative kernel regression, and fuzzy similarity-based sensor models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33

Similar content being viewed by others

Abbreviations

AANN:

Auto-associative neural networks

AAKR:

Auto-associative kernel regression

APWR:

Advanced pressurized water reactor

DAE:

Denoising auto-encoder

FS:

Fuzzy similarity

MSET:

Multivariate state estimation technique

NLPS:

Non-linear partial least square

NSSS:

Nuclear steam supply system

PWR:

Pressurized water reactor

PCA:

Principal component analysis

RBM:

Restricted boltzman machine

TE:

Tennessee Eastman

\(S_{Auto}\) :

Auto-sensitivity

\(S_{Cross}\) :

Cross-sensitivity

R :

Robustness

SO :

Spillover

S :

Original non-corrupted sensor readings

\(\tilde{S}\) :

Corrupted sensor readings

\(\hat{S}\) :

Model-predicted sensor readings

References

  • Baraldi P, Canesi R, Zio E et al (2011) Genetic algorithm-based wrapper approach for grouping condition monitoring signals of nuclear power plant components. Integr Comput Aided Eng 18:221–234. doi:10.3233/ICA-2011-0375

    Article  Google Scholar 

  • Baraldi P, Di Maio F, Genini D, Zio E (2015) Reconstruction of missing data in multidimensional time series by fuzzy similarity. Appl Soft Comput J 26:1–9. doi:10.1016/j.asoc.2014.09.038

    Article  Google Scholar 

  • Bengio Y (2009) Learning Deep Architectures for AI. Found Trends\(\textregistered \) Mach Learn 2:1–127. doi:10.1561/2200000006

  • Bengio Y, Simard P, Frasconi P (1994) Learning long term dependencies with gradient descent is difficult. IEEE Trans Neural Networks 5:157–166. doi:10.1109/72.279181

    Article  Google Scholar 

  • Bengio Y, Lamblin P (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst (NIPS) 19:153–160

    Google Scholar 

  • Coble JB, Meyer RM, Ramuhalli P, et al (2012) A Review of Sensor Calibration Monitoring for Calibration Interval Extension in Nuclear Power Plants. Technical report PNNL-21687, Pacific Northwest Natl Lab Richland, Wash, USA. doi:10.2172/1061413

  • Davis E, Rubin L, Hussey A (2006) On-line monitoring cost-benefit guide. Final report 1006777,EPRI, Palo Alto,Calif,USA

  • Di Maio F, Baraldi P, Zio E, Seraoui R (2013) Fault detection in nuclear power plants components by a combination of statistical methods. Reliab IEEE Trans 62:833–845

    Article  Google Scholar 

  • Downs JJ, Vogel EF (1993) A plant-wide industrial process control problem. Comput Chem Eng 17:245–255. doi:10.1016/0098-1354(93)80018-I

    Article  Google Scholar 

  • Elnokity O, Mahmoud II, Refai MK, Farahat HM (2012) ANN based sensor faults detection, isolation, and reading estimates—SFDIRE: applied in a nuclear process. Ann Nucl Energy 49:131–142. doi:10.1016/j.anucene.2012.06.003

    Article  Google Scholar 

  • Erhan D, Courville A, Vincent P (2010) Why does unsupervised pre-training help deep learning? J Mach Learn Res 11:625–660. doi:10.1145/1756006.1756025

    MathSciNet  MATH  Google Scholar 

  • Erhan D, Bengio Y, Courville A, Vincent P (2009) Visualizing higher-layer features of a deep network. Departement d’Informatique Rech Operationnelle, Tech Rep 1341:1–13

  • Fantoni PF, Hoffmann MI, Shankar R, Davis EL (2003) On-line monitoring of instrument channel performance in nuclear power plant using PEANO. Prog Nucl Energy 43:83–89. doi:10.1016/S0149-1970(03)00017-9

  • Fantoni PF (2005) Experiences and applications of PEANO for monitoring in power plants. Prog Nucl Energy 46:206–225. doi:10.1016/j.pnucene.2005.03.005

    Article  Google Scholar 

  • Fantoni PF, Mazzola A (1996) A pattern recognition-artificial neural networks based model for signal validation in nuclear power plants. Ann Nucl Energy 23:1069–1076. doi:10.1016/0306-4549(96)84661-5

    Article  Google Scholar 

  • Garvey J, Garvey D, Seibert R, Hines JW (2007) Validation of on-line monitoring techniques to nuclear plant data. Nucl Eng Technol 39:133–142. doi:10.5516/NET.2007.39.2.133

    Article  Google Scholar 

  • Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of 13th international conference artificial intelligence, vol 9. Aistats, pp 249–256

  • Gribok AV, Hines JW, Urmanov A, Uhrig RE (2000) Regularization of ill-posed surveillance and diagnostic measurements. Power plant surveillance and diagnostics. Appl Res Artif Intell, Springer

  • Gribok AV, Hines JW, Urmanov A, Uhrig RE (2002) Heuristic, systematic, and informational regularization for process monitoring. Int J Intell Syst 17:723–749. doi:10.1002/int.10047

    Article  MATH  Google Scholar 

  • Gross KC, Singer RM, Wegerich SW, Herzog JP (1997) Application of a model-based fault detection system to nuclear plant signals. In: Proceedings of the intelligent system applications to power systems, ISAP, Seoul, Korea, pp 66–70

  • Gross KC, Singer RM, Wegerich S, Mott J (1998) Multivariate state estimation technique (MSET) based surveillance system. In: U.S. Patent No. 5764509

  • Hashemian HM (2011) On-line monitoring applications in nuclear power plants. Prog Nucl Energy 53:167–181. doi:10.1016/j.pnucene.2010.08.003

    Article  Google Scholar 

  • Heo G-Y (2008) Condition monitoring using empirical models: technical review and prospects for nuclear applications. Nucl Eng Technol 40:49–68. doi:10.5516/NET.2008.40.1.049

    Article  Google Scholar 

  • Hines JW, Gribok A, Attieh I, Uhrig R (2000) Regularization Methods for Inferential Sensing in Nuclear Power Plants. In: Ruan D (ed) Fuzzy systems and soft computing in nuclear engineering SE - 13. Physica-Verlag HD, pp 285–314

  • Hines JW, Usynin A (2005) Autoassociative model input variable selection for process monitoring. In: International symposium on the future I&C for nuclear power plants

  • Hines JW, Uhrig RE, Wrest DJ (1998) Use of autoassociative neural networks for signal validation. J Intell Robot Syst 21:143–154

    Article  Google Scholar 

  • Hines JW, Garvey DR (2006) Development and application of fault detectability performance metrics for instrument calibration verification and anomaly detection. Pattern Recognit 1:2–15

    Article  Google Scholar 

  • Hochreiter S, Bengio Y, Frasconi P, Schmidhuber J (2001) Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. A F Guid to Dyn Recurr Networks 237–243: doi:10.1109/9780470544037.ch14

  • IAEA (2008) On-line monitoring for improving performance of nuclear power plants part 1: instrument channel monitoring. Technical report nuclear energy series NP-T-1.1, International Atomic Energy Agency, Vienna, Austria

  • Karklin Y, Simoncelli EP (2011) Efficient coding of natural images with a population of noisy linear–nonlinear neurons. Adv Neural Inf Process Syst 24:999–1007

  • Kim KD, Lee SW, Hwang M et al (2007) Development of a Visual System Analyzer based on reactor system analysis codes. Prog Nucl Energy 49:452–462. doi:10.1016/j.pnucene.2007.07.005

    Article  Google Scholar 

  • Kramer MA (1991) Nonlinear principal component analysis using autoassociative neural networks. AIChE J 37:233–243

    Article  Google Scholar 

  • Kramer MA (1992) Autoassociative neural networks. Comput Chem Eng 16:313–328. doi:10.1016/0098-1354(92)80051-A

    Article  Google Scholar 

  • Larochelle H, Larochelle H, Bengio Y et al (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10:1–40

    MATH  Google Scholar 

  • Liu E, Zhang D (2003) Diagnosis of component failures in the space shuttlemain engines using Bayesian belief network: a feasibility study. Int J Artif Intell Tools 12:355–374. doi:10.1142/S0218213003001277

    Article  Google Scholar 

  • Ma J, Jiang J (2011) Applications of fault detection and diagnosis methods in nuclear power plants: a review. Prog Nucl Energy 53:255–266. doi:10.1016/j.pnucene.2010.12.001

    Article  Google Scholar 

  • Marseguerra M, Zoia A (2005) The autoassociative neural network in signal analysis: II. Application to on-line monitoring of a simulated BWR component. Ann Nucl Energy 32:1207–1223. doi:10.1016/j.anucene.2005.03.005

    Article  Google Scholar 

  • Marseguerra M, Zoia A (2005) The autoassociative neural network in signal analysis: I. The data dimensionality reduction and its geometric interpretation. Ann Nucl Energy 32:1191–1206. doi:10.1016/j.anucene.2005.03.006

    Article  Google Scholar 

  • Penha R, Hines J (2001) Using principal component analysis modeling to monitor temperature sensors in a nuclear research reactor. In: Maintenance and reliability conference (MARCON 2001)

  • Petrović I, Baotić M, Perić N (2000) Regularization and validation of neural network models of nonlinear systems. e i. Elektrotech Informationstech 117:24–31. doi:10.1007/BF03161395

    Article  Google Scholar 

  • Rasmussen B, Hines JW, Uhrig RE (2000) Nonlinear partial least squares modeling for instrument surveillance and calibration verification. In: proceedings of the maintenance and reliability conference (MARCON 2000), Knoxville, TN

  • Reyes J, Vellasco M, Tanscheit R (2014) Fault detection and measurements correction for multiple sensors using a modified autoassociative neural network. Neural Comput Appl 24:1929–1941. doi:10.1007/s00521-013-1429-4

    Article  Google Scholar 

  • Rifai S, Muller X (2011) Contractive auto-encoders: explicit invariance during feature extraction. Icml 85:833–840

    Google Scholar 

  • Şeker S, Ayaz E, Türkcan E (2003) Elman’s recurrent neural network applications to condition monitoring in nuclear power plant and rotating machinery. Eng Appl Artif Intell 16:647–656. doi:10.1016/j.engappai.2003.10.004

    Article  Google Scholar 

  • Shaheryar A, Yin X-C, Hao H-W et al (2016) A denoising based autoassociative model for robust sensor monitoring in nuclear power plants. Sci Technol Nucl Install 2016:1–17. doi:10.1155/2016/9746948

    Article  Google Scholar 

  • Urmanov AM, Gribok A V, Hines JW, Uhrig RE (2000) Complexity-penalized model selection for feedwater inferential measurements in nuclear power plants. In: International topical meeting on nuclear plant instrumentation, controls, and human-machine interface technologies (NPIC&HMIT 2000), Washington, DC, November, 2000

  • Usynin A, Hines Wesley J, Ding J (2004) On-line monitoring robustness measures and comparisons. In: International atomic energy agency technical meeting on increasing instrument calibration intervals through on-line calibration technology, OECD Halden Reactor Project, Halden, Norway

  • Usynin A, Hines JW (2005) MSET performance optimization through regularization. Nucl Eng Technol 37:159–166

    Google Scholar 

  • Utgoff PE, Stracuzzi DJ (2002) Many-Layered Learning. Neural Comput 14:2497–2529. doi:10.1162/08997660260293319

    Article  MATH  Google Scholar 

  • Vincent P, Larochelle H, Lajoie I et al (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408. doi:10.1111/1467-8535.00290

    MathSciNet  MATH  Google Scholar 

  • Vincent P, Larochelle H, Bengio Y, Manzagol P-A (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning - ICML ’08. pp 1096–1103

  • Wrest DJ, Hines JW, Uhrig RE (1996) Instrument surveillance and calibration verification through plant wide monitoring using autoassociative neural networks. In: In the proceedings of the 1996 American nuclear society international topical meeting on nuclear plant instrumentation, Control and human machine interface technologies. pp 6–9

  • Yin S, Ding SX, Haghani A et al (2012) A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. J Process Control 22:1567–1581. doi:10.1016/j.jprocont.2012.06.009

    Article  Google Scholar 

  • Zavaljevski N, Gross KC (2000) Sensor fault detection in nuclear power plants using multivariate state estimation technique and support vector machines. In: ANS international topical meeting on “advances in reactor physics and mathematics and computation into the next millennium”

Download references

Acknowledgements

The authors acknowledge the support of the China Scholarship Council for research funding. We are also thankful to the International Atomic Energy Agency’s (IAEA) simulator development program for providing the nuclear power plant simulator tools, which were primarily used in the preparation of the sensor dataset for this research.

Author information

Authors and Affiliations

  1. University of Science and Technology Beijing, Beijing, China

    Ahmad Shaheryar, Xu-Cheng Yin, Hong-Wei Hao, Zahid Mahmood & Adnan O. M. Abuassba

Authors
  1. Ahmad Shaheryar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu-Cheng Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong-Wei Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zahid Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adnan O. M. Abuassba
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmad Shaheryar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shaheryar, A., Yin, XC., Hao, HW. et al. Selection of optimal denoising-based regularization hyper-parameters for performance improvement in a sensor validation model. Artif Intell Rev 50, 341–382 (2018). https://doi.org/10.1007/s10462-017-9546-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-017-9546-6

Keywords

Search

Navigation