Skip to main content
SpringerLink
Log in

Diagrammatic methods for deriving and relating temporal neural network algorithms

  • Chapter
  • First Online:
Book cover Adaptive Processing of Sequences and Data Structures (NN 1997)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1387))

Included in the following conference series:

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Back, A., and Tsoi, A. 1991. FIR and IIR synapses, a new neural networks architecture for time series modeling. Neural Computation, vol. 3, no. 3, pages 375–85.

    Google Scholar 

  2. Beaufays, F., and Wan, E. 1994a. Relating real-time backpropagation and backpropagation-through-time: an application of flow graph interreciprocity. Neural Computation, vol. 6, no. 2, pages 296–306.

    Google Scholar 

  3. Beaufays, F., and Wan, E. 1994b. An efficient first-order stochastic algorithm for lattice filters, ICANN'94, vol. 2, pages 1021–1024, May 26–29, Sorrento, Italy.

    Google Scholar 

  4. Bordewijk, J. 1956. Inter-reciprocity applied to electrical networks. Appl. Sci. Res. 6B, pages 1–74.

    MathSciNet  Google Scholar 

  5. Bottou, L., and Gallinari, P. 1991. A framework for the cooperation of learning algorithms. In Advances in Neural Information Processing Systems, vol. 4. Lippmann, R. P., Moody, J., Touretzky, D. eds. Morgan Kaufmann, pages 781–788.

    Google Scholar 

  6. Bryson, A. and Ho, Y. 1975. Applied Optimal Control. Hemisphere Publishing Corp., New York.

    Google Scholar 

  7. Campolucci, P., Marchegiani A., Uncini A., and Piazza F. Signal-Flow-Graph Derivation of on-line gradient learning algorithms. Proc. ICNN-97, Houston, June 1997, pages 1884–1889.

    Google Scholar 

  8. Crochiere, R. E. and Oppenheim, A. V. Analysis of linear digital networks. Proc. IEEE, vol. 63, no. 4, April 1975, pages 581–595.

    Article  Google Scholar 

  9. Cybenko, G. 1989. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems, vol. 2, no. 4.

    Google Scholar 

  10. De Vries, B. and Principe, J.C. The gamma model-a new neural model for temporal processing. Neural Networks, vol.5, no.4, pages 565–76, 1992.

    Article  Google Scholar 

  11. Fettweiss, A. A general theorem for signal-flow networks, with applications. In Digital Signal Processing, Rabiner and Rader, eds. IEEE Press, 1972, pages 126–130.

    Google Scholar 

  12. Frasconi, P., Gori, M., and Soda, G. 1992. Local Feedback Multi-Layered Networks. Neural Computation, vol. 4, no. 1.

    Google Scholar 

  13. Griewank, A., and Coliss, G., eds. 1991. Automatic Differentiation of Algorithms: Theory, Implementation, and Application. Proceedings of the first SIAM workshop on automatic differentiation, Brekenridge, CO.

    Google Scholar 

  14. Griffiths, L. 1977. A continuously adaptive filter implemented as a lattice structure. Proc. ICASSP, Hartford, CT, pages 683–686.

    Google Scholar 

  15. Hertz, J. A., Krogh, A., and Palmer, R. G. 1991. Introduction to the Theory of Neural Computation. Addison-Wesley, Reading, PA.

    Google Scholar 

  16. Hornik, K., Stinchombe, M., and White, H. 1989. Multilayer feedforward networks are universal approximators. Neural Networks, vol. 2, pages 359–366.

    Article  Google Scholar 

  17. Jordan, M. I. and Jacobs, R. A. Hierarchies of adaptive experts. 1992. In Advances in Neural Information Processing Systems, vol. 4. Moody, J. E., Hanson, S. J., and Lippmann, R. P. eds. Morgan Kaufmann, pages 985–992.

    Google Scholar 

  18. Kailath, T. 1980. Linear Systems. Prentice-Hall, Englewood Cliffs, NJ.

    MATH  Google Scholar 

  19. Koch, C. and Segev, I, eds. 1989. Methods in Neuronal Modeling: From Synapses to Networks. MIT Press, Cambridge, MA.

    Google Scholar 

  20. Landau, I. Adaptive Control: The Model Reference Approach. Marcel Dekker, NewYork, 1979.

    MATH  Google Scholar 

  21. LeCun, Y., Boser, B., et al. 1989. Backpropagation applied to handwritten zip code recognition. Neural Computation, vol. 1, pages 541–551, winter.

    Google Scholar 

  22. MacGregor, R.J. 1987, Neural Brain Modeling, Academic Press, Orlando, FL.

    MATH  Google Scholar 

  23. Matsuoka, K. 1991. Learning of neural networks using their adjoint systems. Systems and Computers in Japan, vol. 22, no. 11, pages 31–41.

    MathSciNet  Google Scholar 

  24. Narendra, K. and Parthasarathy, K. 1990. Identification and control of dynamic systems using neural networks. IEEE Trans. on Neural Networks, vol. 1, no. 1, pages 4–27.

    Article  Google Scholar 

  25. Nerrand, O., Roussel-Ragot, P., Personnaz, L., Dreyfus, G., and Marcos, S. 1993. Neural networks and nonlinear adaptive filtering: Unifying concepts and new algorithms. Neural Computation, vol. 5, no. 2, pages 165–199.

    Google Scholar 

  26. Nguyen, D., and Widrow, B. 1989. The truck backer-upper: an example of self-learning in neural networks. Proceedings of the International Joint Conference on Neural Networks, II, Washington, DC, pages 357–363.

    Article  Google Scholar 

  27. Oppenheim, A., and Schafer, R. 1989. Digital Signal Processing. Prentice-Hall, Englewood Cliffs, NJ.

    Google Scholar 

  28. Parisini, T. and Zoppoli, R. 1994. Neural networks for feedback feedforward nonlinear control systems. IEEE Trans. on Neural Networks, vol. 5, no. 3, pages 436–439.

    Article  Google Scholar 

  29. Parker, D., 1982. Learning-logic. Invention Report S81-64, File 1, Office of Technology Licensing, Stanford University, October.

    Google Scholar 

  30. Penfield, P., Spence, R., and Duiker, S. 1970 Tellegen's Theorem and Electrical Networks, MIT Press, Cambridge, MA.

    Google Scholar 

  31. Piche, S.W. Steepest descent algorithms for neural network controllers and filters, IEEE Trans. on Neural Networks, vol.5, no. 2, pages 198–212, 1994.

    Article  Google Scholar 

  32. Pineda, F. J. 1987. Generalization of back-propagation to recurrent neural networks. IEEE Trans. on Neural Networks, special issue on recurrent networks.

    Google Scholar 

  33. Plumer, E. 1993a. Time-optimal terminal control using neural networks. Proc. of the IEEE International Conference on Neural Networks. San Francisco, CA, pages 1926–1931.

    Google Scholar 

  34. Plumer, E. 1993b. Optimal Terminal Control Using Feedforward Neural Networks. Ph.D. dissertation. Stanford University.

    Google Scholar 

  35. Puskorious, G. and Feldkamp, L. 1994. Neural control of nonlinear dynamic systems with Kaiman filter trained recurrent networks. IEEE Trans. on Neural Networks, vol. 5, no. 2.

    Google Scholar 

  36. Rall, B. 1981. Automatic Differentiation: Techniques and Applications, Lecture Notes in Computer Science, Springer-Verlag.

    Google Scholar 

  37. Ramo, S., Whinnery, J.R., and Van Duzer, T. 1984. Fields and Waves in Communication Electronics, Second Edition. John Wiley & Sons.

    Google Scholar 

  38. Rumelhart, D.E., McClelland, J.L., and the PDP Research Group. 1986. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1. MIT Press, Cambridge, MA.

    Google Scholar 

  39. Tellegen, D. 1952. A general network theorem, with applications. Philips Res. Rep. 7, pages 259–269.

    MATH  MathSciNet  Google Scholar 

  40. Toomarian, N.B. and Barhen, J. 1992. Learning a trajectory using adjoint function and teacher forcing. Neural Networks, vol. 5, no. 3, pages 473–484.

    Article  Google Scholar 

  41. Tsoi, A. C., and Back, A. 1994. Locally Recurrent Globally Feedforward Networks: A Critical review of architectures. IEEE Trans. on Neural Networks, vol. 5, no. 2.

    Google Scholar 

  42. Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., and Lang, K. 1989. Phoneme recognition using time-delay neural networks. IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. 37, no. 3, pages 328–339, March.

    Article  Google Scholar 

  43. Wan, E. 1993a. Finite Impulse Response Neural Networks with Applications in Time Series Prediction. Ph.D. dissertation, Stanford University.

    Google Scholar 

  44. Wan, E. 1993b. Time series prediction using a connectionist network with internal delay lines. In A. Weigend and N. Gershenfeld, editors, Time Series Prediction: Forecasting the Future and Understanding the Past, Addison-Wesley.

    Google Scholar 

  45. Wan, E. 1993c. Modeling Nonlinear Dynamics With Neural Networks: Examples in Time Series Prediction. In Proc. of the Fifth Workshop on Neural Networks: Academic/Industrial/NASA/Defense, WNN93/FNN93, San Francisco, pages 327–232, November.

    Google Scholar 

  46. Wan, E., and Beaufays, F. 1996. Diagrammatic derivation of gradient algorithms for neural networks. Neural Computation, vol. 8, no. 1, January 1996, pages 182–201.

    Google Scholar 

  47. Werbos, P., 1974. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Ph.D. thesis, Harvard University.

    Google Scholar 

  48. Werbos, P. 1990. Backpropagation through time: what it does and how to do it. Proc. IEEE, special issue on neural networks, vol. 2, pages 1550–1560.

    Google Scholar 

  49. Werbos, P. 1992. Neurocontrol and supervised learning: An overview and evaluation. In D. White and D. Sofge, eds., Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, chapter 3. Van Nostrand Reinhold, New York.

    Google Scholar 

  50. White, S. A. 1975. An adaptive recursive digital filter. Proc. 9th Asilomar Conf. Circuits Syst. Cornput., page 21.

    Google Scholar 

  51. Williams, R.J. and Peng J., 1990. An efficient gradient-based algorithm for on line training of recurrent network trajectories. Neural Computation, vol. 2, pages 490–501.

    Google Scholar 

  52. Williams, R. J. and Zipser, D. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural Computation, vol. 1, no. 2, pages 270–280.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Oregon Graduate Institute, P.O. Box 91000, 97291, Portland, OR

    Eric A. Wan

  2. Speech Technology and Research Laboratory, SRI International, 94025, Menlo Park, CA

    Françoise Beaufays

Authors
  1. Eric A. Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Françoise Beaufays
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

C. Lee Giles Marco Gori

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Wan, E.A., Beaufays, F. (1998). Diagrammatic methods for deriving and relating temporal neural network algorithms. In: Giles, C.L., Gori, M. (eds) Adaptive Processing of Sequences and Data Structures. NN 1997. Lecture Notes in Computer Science, vol 1387. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0053995

Download citation

  • DOI: https://doi.org/10.1007/BFb0053995

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-64341-8

  • Online ISBN: 978-3-540-69752-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation