Skip to main content
SpringerLink
Log in

Approximate computation of Madaline sensitivity based on discrete stochastic technique

  • Research Papers
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

The computation of the sensitivity of a Madaline’s output to its parameter perturbation is systematically discussed. Firstly, according to the discrete feature of Adalines, a method based on discrete stochastic technique is proposed, which derives some analytical formulas for the computation of Adalines’ sensitivity. The method can theoretically solve some problems that are unsolvable by the existing methods based on continuous stochastic techniques, release some unpractical constraints, and make it available to theoretically analyze the approximation error of Adalines’ sensitivity. Secondly, on the basis of the sensitivity of Adalines and the structural characteristics of Madalines, a new selection strategy depending on a type of dedication degree for computing Madalines’ sensitivity is proposed, which is superior to current popular way of simply averaging in both precision and complexity. The proposed formulas and algorithm have the advantages of simplicity, low computational complexity, small approximation error, and high generality, as have been verified by a great amount of experimental simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Hoff M E, Jr. Leaning phenomena in networks of adaptive switching circuits. PhD dissertation. Stanford: Dept. Electrical Eng., Stanford Univ., 1962. 1–91

    Google Scholar 

  2. Glanz F H. Statistical extrapolation in certain adaptive pattern recognition systems. PhD dissertation. Stanford: Dept. Electrical Eng., Stanford Univ., 1965. 1–142

    Google Scholar 

  3. Winter R G. Madaline rule II: a new method for training networks of Adalines. PhD dissertation. Stanford: Dept. Electrical Eng., Stanford Univ., 1989. 1–91

    Google Scholar 

  4. Stevenson M, Winter R, Widrow B. Sensitivity of feedforward neural networks to weight errors. IEEE Trans Neural Netw, 1990, 1: 71–80

    Article  Google Scholar 

  5. Choi J Y, Choi C H. Sensitivity analysis of multilayer perceptron with differentiable activation functions. IEEE Trans Neural Netw, 1992, 3: 101–107

    Article  Google Scholar 

  6. Piché S W. The selection of weight accuracies for Madalines. IEEE Trans Neural Netw, 1995, 6: 432–445

    Article  Google Scholar 

  7. Alippi C, Piuri V, Sami M G. Sensitivity to errors in artificial neural networks: a behavioral approach. IEEE Trans Circuits Syst I-Regul Pap, 1995, 42: 358–361

    Article  Google Scholar 

  8. Oh S H, Lee Y. Sensitivity analysis of a single hidden-layer neural networks with threshold function. IEEE Trans Neural Netw, 1995, 6: 1005–1007

    Article  Google Scholar 

  9. Cheng A Y, Yeung D S. Sensitivity analysis of neocognitron. IEEE Trans Syst Man Cybern C, 1999, 29: 238–249

    Article  Google Scholar 

  10. Zeng X, Yeung D S. Sensitivity analysis of multilayer Perceptron to input and weight perturbations. IEEE Trans Neural Netw, 2001,12: 1358–1366

    Article  Google Scholar 

  11. Yeung D S, Sun X. Using function approximation to analyze the sensitivity of the MLP with antisymmetric squashing activation function. IEEE Trans Neural Netw, 2002, 13: 34–44

    Article  Google Scholar 

  12. Zeng X, Yeung D S. A quantified sensitivity measure for multilayer perceptron to input perturbation. Neural Comput, 2003, 15: 183–212

    Article  MATH  Google Scholar 

  13. Zeng X, Wang Y, Zhang K. Computation of Adalines’ sensitivity to weight perturbation. IEEE Trans Neural Netw, 2006, 17: 515–519

    Article  Google Scholar 

  14. Wang Y, Zeng X, Yeung D S, et al. Computation of Madalines’ sensitivity to input and weight perturbations. Neural Comput, 2006, 18: 2854–2877

    Article  MATH  MathSciNet  Google Scholar 

  15. Zurada J M, Malinowski A, Usui S. Perturbation method for deleting redundant inputs of perceptron networks. Neurocomput, 1997, 14: 177–193

    Article  Google Scholar 

  16. Bernier J L, Ortega J, Rojas I, et al. Improving the tolerance of multilayer perceptrons by minimizing the statistical sensitivity to weight deviations. Neurocomput, 2000, 31: 87–103

    Article  Google Scholar 

  17. Bernier J L, Ortega J, ROS E, et al. A quantitive study of fault tolerance, noise immunity and generalization ability of MLPs. Neural Comput, 2000, 12: 2941–2964

    Article  Google Scholar 

  18. Engelbrecht A P. A new pruning heuristic based on variance analysis of sensitivity information. IEEE Trans Neural Netw, 2001, 12: 1386–1399

    Article  Google Scholar 

  19. Zeng X, Yeung D S. Hidden neuron pruning of multilayer Perceptrons using a quantified sensitivity measure. Neurocomput, 2006, 69: 825–837

    Article  Google Scholar 

  20. Yeung D, Ng W, Wang D, et al. Localized generalization error model and its application to architecture selection for radial basis function neural network. IEEE Trans Neural Netw, 2007, 18: 1294–1305

    Article  Google Scholar 

  21. Zeng X, Shao J, Wang Y, et al. A sensitivity-based approach for pruning architecture of Madalines. Neural Comput Appl, 2009, 18: 957–965

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Pattern Recognition and Intelligent System, College of Computer and Information Engineering, Hohai University, Nanjing, 210098, China

    ShuiMing Zhong, XiaoQin Zeng, HuiYi Liu & Yan Xu

Authors
  1. ShuiMing Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. XiaoQin Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. HuiYi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to ShuiMing Zhong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhong, S., Zeng, X., Liu, H. et al. Approximate computation of Madaline sensitivity based on discrete stochastic technique. Sci. China Inf. Sci. 53, 2399–2414 (2010). https://doi.org/10.1007/s11432-010-4122-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-010-4122-6

Keywords

Search

Navigation