Skip to main content
SpringerLink
Log in

Robust Deep Neural Network Using Fuzzy Denoising Autoencoder

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

Deep neural network (DNN) has been applied in many fields and achieved great successes. However, DNN suffers from poor robustness for uncertainties because of its characteristic of the deterministic representation. To overcome this problem, a novel robust DNN (RDNN) is designed in this paper. First, a fuzzy denoising autoencoder (FDA) is developed to replace the general base-building unit in DNN. Then, the proposed RDNN is able to extract the robust representations to weaken the uncertainties. Second, a compact parameter strategy (CPS) is designed to reconstruct the parameters of FDA. Then, the computational burden of FDA can be alleviated to speed up the learning process. Third, an adaptive back-propagation (ABP) algorithm, with an adaptive learning rate strategy, is proposed to update the parameters of RDNN. Then, the performance of RDNN can be improved. Finally, the results on the benchmark problems and real applications demonstrate the effectiveness of RDNN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Athanasios, V., Nikolaos, D.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 49, 2329–2338 (2018)

    Google Scholar 

  2. Hinton, G., Deng, L., Dong, Y.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag. 29(6), 82–97 (2012)

    Google Scholar 

  3. Hwang, M., Wang, D., Jiang, W.C., Pan, X.: An adaptive regularization approach to colonoscopic polyp detection using a cascaded structure of encoder–decoders. Int. J. Fuzzy Syst. 21(7), 2091–2101 (2019)

    Google Scholar 

  4. Huang, W., Song, G., Hong, H., Xie, K.: Deep architecture for traffic flow prediction: deep belief networks with multitask learning. IEEE Trans. Intell. Transp. Syst. 15(5), 2191–2201 (2014)

    Google Scholar 

  5. Saad, E.W., Prokhorov, D.V., Wunsch, D.C.: Comparative study of stock trend prediction using time delay, recurrent and probabilistic neural networks. IEEE Trans. on Neural Netw. 9(6), 1456–1470 (1998)

    Google Scholar 

  6. Xu, Q.Y., Yang, Y.Q., Zhang, C.J., Zhang, L.: Deep convolutional neural network-based autonomous marine vehicle maneuver. Int. J. Fuzzy Syst. 20(2), 687–699 (2018)

    Google Scholar 

  7. Cheng, Y., Sun, Z.J., Huang, Y.X., Zhang, W.D.: Fuzzy categorical deep reinforcement learning of a defensive game for an unmanned surface vessel. Int. J. Fuzzy Syst. 21(2), 592–606 (2019)

    Google Scholar 

  8. Lin, F.J., Lin, C.H., Shen, P.H.: Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive. IEEE Trans. Fuzzy Syst. 9(5), 751–759 (2001)

    Google Scholar 

  9. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    MathSciNet  MATH  Google Scholar 

  10. Long, M., Wang, J., Cao, Y., Sun, J., Yu, P.S.: Deep learning of transferable representation for scalable domain adaptation. IEEE Trans. Knowl. Data Eng. 18(8), 2027–2040 (2016)

    Google Scholar 

  11. Hong, Y.G., Zhong, P.J.: Finite-time stabilization of nonlinear systems with parametric and dynamic uncertainties. IEEE Trans. Autom. Control 51(12), 1950–1956 (2006)

    MathSciNet  MATH  Google Scholar 

  12. Wang, Y.C., Chien, C.J., Chi, R.H., Hou, Z.S.: A fuzzy-neural adaptive terminal iterative learning control for fed-batch fermentation processes. Int. J. Fuzzy Syst. 17(3), 423–433 (2015)

    MathSciNet  Google Scholar 

  13. Jose, N., Josue, F., Victor, P.: Uncertainty weighting and propagation in DNN–HMM-based speech recognition. Comput. Speech Lang. 47, 30–46 (2018)

    Google Scholar 

  14. Kim, S.W., Yu, Z.B., Kil, R.M.: Deep learning of support vector machines with class probability output networks. Neural Netw. 64, 19–28 (2015)

    MATH  Google Scholar 

  15. Goldberger, Z.D., Walsh, K.E.: Syncope units: an emerging paradigm. Pace-Pacing and Clinical Electrophysiology 42(7), 828–829 (2019)

    Google Scholar 

  16. Han, B., Yao, Q.M., Yu, X.R.: Co-teaching robust training of deep neural networks with extremely noisy labels. Neural Inf. Processing Syst. 31, 345–364 (2018)

    Google Scholar 

  17. Zhou, Y.C., Liu, Y., Wang, R.: Training deep neural networks for image applications with complementary learning. J. Comput. Res. Dev. 54(12), 2649–2659 (2017)

    Google Scholar 

  18. Badiali, B., Titus, N.E.: Co-teaching: enhancing student learning through mentor-intern partnerships. IEEE Trans. Autom. Control 4(2), 74–80 (2014)

    Google Scholar 

  19. Srivastava, N., Hinton, G.E., Sutskever, I.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  20. Frenay, B., Verleysen, M.: Classification in the presence of label noise: a survey. IEEE Trans. Neural Netw. Learn. Syst. 25(5), 845–869 (2014)

    MATH  Google Scholar 

  21. Kenzo, I., Takashi, I., Taichiro, S., Tomoyuki, T.: Deep shrinkage convolutional neural network for adaptive noise reduction. IEEE Signal Process. Lett. 25(2), 224–230 (2018)

    Google Scholar 

  22. Nie, D., Wang, L., Gao, Y.Z., Lian, J., Shen, D.: STRAINet: spatially varying stochastic residual adversarial networks for MRI pelvic organ segmentation. IEEE Trans. Neural Netw. Learn. Syst. 30, 5 (2019)

    MathSciNet  Google Scholar 

  23. Wang, Q.L., Guo, W.B., Zhang, K.X.: Random feature nullification for adversary resistant deep architecture. Neural Netw. 70, 54–62 (2016)

    Google Scholar 

  24. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. Comput. Vis. Pattern Recognit. 42(7), 770–778 (2016)

    Google Scholar 

  25. Battistone, F., Petrosino, A.: TGLSTM: a time based graph deep learning approach to gait recognition. Pattern Recogn. Lett. 24(4), 671–686 (2018)

    Google Scholar 

  26. Raja, K.B., Raghavendra, R., Vemuri, V.K., Busch, C.: Smartphone based visible iris recognition using deep sparse filtering. Comput. Vis. Pattern Recognit. 57, 33–42 (2015)

    Google Scholar 

  27. Jiang, W.Y., Chung, F.L., Qian, P.: Feedforward kernel neural networks, generalized least learning machine, and its deep learning with application to image classification. Appl. Soft Comput. 37, 125–141 (2015)

    Google Scholar 

  28. Azadeh, A., Ziaei, B., Moghaddam, M.: A hybrid fuzzy regression-fuzzy cognitive map algorithm for forecasting and optimization of housing market fluctuations. Expert Syst. Appl. 39, 298–315 (2012)

    Google Scholar 

  29. Han, H.G., Chen, Z.Y., Qiao, J.F.: A self-organizing interval Type-2 fuzzy-neural-network for modeling nonlinear systems. Neurocomputing 290(1), 196–207 (2018)

    Google Scholar 

  30. Huang, Y.P., Singh, A., Liu, S., Wu, S., Quoc, H., Sereter, A.: Developing transformed fuzzy neural networks to enhance medical data classification accuracy. Int. J. Fuzzy Syst. 20(6), 1925–1937 (2018)

    Google Scholar 

  31. Han, H.G., Liu, Z., Qiao, J.F.: Fuzzy-neural-network-based model predictive control for dissolved oxygen concentration of WWTPs. Int. J. Fuzzy Syst. 21(5), 1497–1510 (2019)

    MathSciNet  Google Scholar 

  32. Han, H.G., Chen, Z.Y., Liu, H.X., Qiao, J.F.: A self-organizing interval type-2 fuzzy-neural-network for modeling nonlinear systems. Neurocomputing 290, 196–207 (2018)

    Google Scholar 

  33. Li, C.L., Chen, G.D., Sung, T.Y., Tsai, H.F.: Novel adaptive Kalman filter with fuzzy neural network for trajectory estimation system. Int. J. Fuzzy Syst. 21(6), 1649–1660 (2019)

    Google Scholar 

  34. Han, H.G., Liu, Z., Ge, L.M., Qiao, J.F.: Prediction of sludge bulking using the knowledge-leverage-based fuzzy neural network. Water Sci. Technol. 77(3), 617–627 (2018)

    Google Scholar 

  35. Deng, Y., Bao, F., Kong, Y.Y., Ren, Z.Q., Dai, Q.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2017)

    Google Scholar 

  36. Deng, Y., Ren, Z.Q., Kong, Y.Y., Bao, F., Dai, Q.H.: A hierarchical fused fuzzy deep neural network for data classification. IEEE Trans. Fuzzy Syst. 25(4), 1006–1012 (2017)

    Google Scholar 

  37. Samui, S., Chakrabarti, I., Ghosh, S.: Time–frequency masking based supervised speech enhancement framework using fuzzy deep belief network. Appl. Soft Comput. 74, 583–602 (2019)

    Google Scholar 

  38. Khodayar, M., Kaynak, O., Khodayar, M.: Rough deep neural architecture for short-term wind speed forecasting. IEEE Trans. Industr. Inf. 13(6), 2770–2779 (2017)

    Google Scholar 

  39. Zheng, Y.J., Ling, H.F., Chen, S.Y., Xue, J.Y.: A hybrid neuro-fuzzy network based on differential biogeography-based optimization for online population classification in earthquakes. IEEE Trans. Fuzzy Syst. 23(4), 1070–1083 (2015)

    Google Scholar 

  40. Siderators, G., Ikonomopoulos, A., Hatziargryriou, N.D.: A novel fuzzy-based ensemble model for load forecasting using hybrid deep neural networks. Electr. Power Syst. Res. 178, 1–10 (2020)

    Google Scholar 

  41. Liang, W.Z., Zhao, G.Y., Zhao, J., Ma, C.: Hatziargryriou, Assessing the rockburst risk for deep shafts via distance-based multi-criteria decision making approaches with hesitant fuzzy information. Eng. Geol. 260, 524–536 (2019)

    Google Scholar 

  42. Bodyanskiy, Y., Tyshchenko, O.: A hybrid cascade neuro–fuzzy network with pools of extended neo–fuzzy neurons and its deep learning. Int. J. Appl. Math. Comput. Sci. 29(3), 477–488 (2019)

    MATH  Google Scholar 

  43. Ekpenyong, M., Etebong, P., Jackson, T.: A novel fuzzy-based convolutional neural network method to traffic flow prediction with uncertain traffic accident information. IEEE Access 7, 558–569 (2019)

    Google Scholar 

  44. Ricardo, D.A., Oliveira, L.I.: On the problem of forecasting air pollutant concentration with morphological models. Neurocomputing 265, 91–104 (2017)

    Google Scholar 

  45. Saverio, D.V., Grazia, F., Pardo, M.: Semi-supervised learning techniques in artificial olfaction: a novel approach to classification problems and drift counteraction. IEEE Sens. J. 12(11), 3215–3224 (2012)

    Google Scholar 

  46. Bezerra, E.C., Le, P.S., Braga, D.S.: A self-adaptive approach for particle swarm optimization applied to wind speed forecasting. J. Control Automat. Electr. Syst. 28(6), 785–795 (2017)

    Google Scholar 

  47. Gu, J.R., Zhu, M.C., Jiang, L.G.: Housing price forecasting based on genetic algorithm and support vector machine. Expert Syst. Appl. 38, 3383–3386 (2014)

    Google Scholar 

  48. Bork, L., Moller, S.: Forecasting house prices using dynamic model averaging and dynamic model selection. Int. J. Forecast. 31, 63–78 (2015)

    Google Scholar 

Download references

Acknowledgements

This work was supported by National Key Research and Development Project under Grants 2018YFC1900800-5, National Science Foundation of China under Grants 61890930-5, 61622301 and 61903010, and Beijing Outstanding Young Scientist Program under Grant BJJWZYJH01201910005020. Asterisk indicates corresponding author

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124, China

    Hong-Gui Han, Hui-Juan Zhang & Jun-Fei Qiao

  2. Beijing Key Laboratory of Computational Intelligence and Intelligent System, Engineering Research Center of Digital Community, Ministry of Education, Beijing Artificial Intelligence Institute and Beijing Laboratory for Urban Mass Transit, Beijing University of Technology, Beijing, 100124, China

    Hong-Gui Han, Hui-Juan Zhang & Jun-Fei Qiao

Authors
  1. Hong-Gui Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui-Juan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun-Fei Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong-Gui Han.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, HG., Zhang, HJ. & Qiao, JF. Robust Deep Neural Network Using Fuzzy Denoising Autoencoder. Int. J. Fuzzy Syst. 22, 1356–1375 (2020). https://doi.org/10.1007/s40815-020-00845-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-020-00845-6

Keywords

Search

Navigation