Using Fourier-neural recurrent networks to fit sequential input/output data

doi:10.1016/S0925-2312(97)00008-8

Neurocomputing

Volume 15, Issues 3–4, June 1997, Pages 225-248

https://doi.org/10.1016/S0925-2312(97)00008-8 Get rights and content

Abstract

This paper suggests the use of Fourier-type activation functions in fully recurrent neural networks. The main theoretical advantage is that, in principle, the problem of recovering internal coefficients from input/output data is solvable in closed form.

References (17)

F. Albertini et al.
For neural networks, function determines form
Neural Networks
(1993)
F. Albertini et al.
F. Albertini et al.
Identifiability of discrete-time neural networks
F. Albertini et al.
State observability in recurrent neural networks
M. Ben-Or et al.
A deterministic algorithm for sparse multivariate polynomial interpolation
A.R. Gallant et al.
There exists a neural network that does not make avoidable mistakes
R. Koplon
Linear systems with constrained outputs and transitions
R. Koplon et al.
Techniques for parameter reconstruction in Fourier-neural recurrent networks
G. Kuhn, N.P. Herzberg, Some variations on training of recurrent networks, Neural Networks: Theory and Applications,...

There are more references available in the full text version of this article.

Cited by (20)

TSPNet: Translation supervised prototype network via residual learning for multimodal social relation extraction
2022, Neurocomputing
Multimodal social relation extraction requires sufficient features fusion to identify the relation between different targets. Compared with traditional multimodal social relation extraction, there are many semantic gap issues for the few-shot scenario task, such as insufficient across-modality assistance, lacking explicit supervision, and unbalanced relations. To address the above problems, a novel Translation Supervised Prototype Network (TSPNet) is proposed, which extracts all the features of knowledge triples, not just relation features. First, the triple-level unimodal encoder learns textual and visual representation of knowledge triples from the entire information via two-stream encoding. Second, the triple-level multimodal extractor obtains multimodal knowledge triples by employing the residual learner to build the triple-level interaction across modalities. Finally, the intra-triple translation supervised decoder predicts the few-shot relations based on a prototype network supervised with the intra-triple translation as an explicit constraint. Our model achieves SOTA performance on three challenging benchmark datasets for few-shot multimodal social relation extraction, and further analysis shows that our model is effective and owns a strong generalization ability to avoid bias.
dNSP: A biologically inspired dynamic Neural network approach to Signal Processing
2008, Neural Networks
The arriving order of data is one of the intrinsic properties of a signal. Therefore, techniques dealing with this temporal relation are required for identification and signal processing tasks. To perform a classification of the signal according with its temporal characteristics, it would be useful to find a feature vector in which the temporal attributes were embedded. The correlation and power density spectrum functions are suitable tools to manage this issue. These functions are usually defined with statistical formulation. On the other hand, in biology there can be found numerous processes in which signals are processed to give a feature vector; for example, the processing of sound by the auditory system.
In this work, the dNSP (dynamic Neural Signal Processing) architecture is proposed. This architecture allows representing a time-varying signal by a spatial (thus statical) vector. Inspired by the aforementioned biological processes, the dNSP performs frequency decomposition using an analogical parallel algorithm carried out by simple processing units. The architecture has been developed under the paradigm of a multilayer neural network, where the different layers are composed by units whose activation functions have been extracted from the theory of Neural Dynamic [Grossberg, S. (1988). Nonlinear neural networks principles, mechanisms and architectures. Neural Networks, 1, 17–61]. A theoretical study of the behavior of the dynamic equations of the units and their relationship with some statistical functions allows establishing a parallelism between the unit activations and correlation and power density spectrum functions.
To test the capabilities of the proposed approach, several testbeds have been employed, i.e. the frequencial study of mathematical functions. As a possible application of the architecture, a highly interesting problem in the field of automatic control is addressed: the recognition of a controlled DC motor operating state.
Fast-Fnet: Accelerating Transformer Encoder Models Via Efficient Fourier Layers
2023, SSRN
ASFNet: Audio Spectrogram Fourier Network for Efficient Medical Sound Event Detection
2023, TechRxiv
Fast-FNet: Accelerating Transformer Encoder Models via Efficient Fourier Layers
2022, arXiv
C-GRBFnet: A Physics-Inspired Generative Deep Neural Network for Channel Representation and Prediction
2022, IEEE Journal on Selected Areas in Communications

View all citing articles on Scopus

View full text

Special paperUsing Fourier-neural recurrent networks to fit sequential input/output data

Abstract

Neural Networks

Identifiability of discrete-time neural networks

State observability in recurrent neural networks

A deterministic algorithm for sparse multivariate polynomial interpolation

There exists a neural network that does not make avoidable mistakes

Linear systems with constrained outputs and transitions

Techniques for parameter reconstruction in Fourier-neural recurrent networks

Special paper
Using Fourier-neural recurrent networks to fit sequential input/output data