Quantization for distributed estimation using neural networks
Introduction
Networks of embedded sensors have started to become increasingly important especially due to their potentially enormous impact in environmental monitoring, product quality control, defense systems, etc. Several researchers have worked on developing protocols and systems architectures for networks of thousands of embedded devices [4], [5], [7]. At the same time, new exciting technologies such as MicroElectroMechanical Systems (MEMS) [2] devices (CMU) and Smart-Dust project [8], [15] (Berkeley) are expected to expand the capabilities of embedded devices and networks of sensors by putting a complete sensing/communication platform, including power supply, analog and digital electronics, inside a cubic millimeter.
In distributed estimation systems several separated processing nodes (i.e., sensors) observe an environment, collect information, and make estimations based on their own observations and information being communicated between the nodes. The model of a distributed estimation system that we consider here consists of a single fusion center and a number of remote sensors. This model has many applications to radar, sonar and remote-sensing systems. In this scheme, the fusion center estimates some unobserved quantities based on observations collected by remote sensors and transmitted to the center. Restrictions on this model such as the capacity constraints on the communication lines suggest some very challenging problems.
The exact model that we consider here is described below. The sensors are not allowed to communicate with each other and there is no feedback from the fusion center back to them. The communication channels are assumed to be error free. The observations from the sensors are vector quantized before the transmission to the fusion center in order to satisfy the communication constraints. Thus, the estimation is achieved via compressed information. We assume fixed length coding for the transmission. The observations at the sensors are random. Here, we consider the case where the joint probability density function is unknown.
The problem is defined as follows: For a distributed system with k sensors, find, for each sensor, a mapping from the observation space to codewords (of a certain number of bits given by the capacity constraints), and find a fusion center function that maps a vector of k codewords to an estimate vector for the unobserved quantities, so that the mean of the square of the Euclidean norm of the estimation error is minimized. There is a joint probability distribution of all observations and unobserved quantities. However, since this distribution is unknown, the design of the system is based on a training set and the mean squared error is computed based on a test set. Although the number of sensors, k, can be in general arbitrary, here we consider the two-sensor case since the method for this case can be easily extended to the more general case.
An approach based on a generalization of regression trees for the problem of quantizer design for such a distributed estimation system in the case of unknown observation statistics has been given by Megalooikonomou and Yesha [13], [14]. The same problem in the case of known probability model was considered by Lam and Reibman [10], [11]. Gubner [6] considers the problem of quantizer design for this system subject not only to communication constraints but also to computation constraints at the fusion center in the case of known observation statistics. Longo et al. [12] consider the problem of quantization for a distributed hypothesis testing system.
In this paper, we consider the problem of quantizer design subject to communication constraints in the case where the joint probability model is unknown and only a training sequence is available. We present an approach that is based on neural networks. In this approach we first apply a variation of the Cyclic Generalized Lloyd’s Algorithm (CGLA) to every point of the training set in order to find the proper codeword for every one of these points. The initial codewords are given by a regression tree approach [13]. We propose to use a neural network for each quantizer in order to represent the training points along with their associated codewords.
The rest of the paper is organized into the following sections. Some notation along with background information is discussed in Section 2. The variation of the CGLA and the neural network approach are presented in Section 3. Implementation issues regarding the calculation of the estimation error are discussed in Section 4. Simulation results are presented in Section 5.
Section snippets
Background
In order to attack the problem of quantizer design for a distributed estimation system in the case where only a training set, is available, one can use the training set with the CGLA to assign the best codeword to every training point. The CGLA was introduced by Longo et al. [12]. This algorithm starts with an initial guess of quantizers and fusion center and iteratively improves them by finding the optimal component given the others. It leads to an estimation error that converges and is very
The methods
In order to design quantizers for a distributed estimation system in the case where only a training set, , is available, we use the training set with a variation of the CGLA to assign the best codeword to every point of the training set. Then we use a neural network to represent the training points and their associated codewords for each quantizer.
Implementation issues
In order to find the best label for every point of the training set we use the lh-CGLA algorithm described in Section 3.1. Here we discuss important implementation issues related to the calculation of the estimation error that is performed repeatedly in lh-CGLA.
Recalling that l(X1q) and l(X2r) are the codewords for the observations X1q and X2r, respectively before the use of the neural network, the estimation of the parameter θ at the fusion center can be expressed as follows:
Simulation results and discussion
In the simulations we consider the case where the observations at the quantizers are scalar quantities of the formwhere the noises nk at the sensors are Gaussian distributed with correlation coefficient ρ and marginal distributions N(0,σn2), where σn2 is the variance of the noises. The parameter θ has Gaussian distribution N(0,1) and is independent of the noises nk, k=1,2. The quantizers are designed using a training set of 10,000 samples and are tested on a test set of
Acknowledgements
We thank the anonymous referees for very helpful comments.
References (15)
- et al.
Classification and Regression Trees
(1984) - et al.
MEMS-based integrated-circuit mass-storage systems
Communications of the ACM
(2000) - et al.
Optimal pruning with applications to tree-structured source coding and modeling
IEEE Transactions on Information Theory
(1989) - et al.
Instrumenting the world with wireless sensor networks
- et al.
Next century challenges: scalable coordination in sensor networks
Distributed estimation and quantization
IEEE Transactions on Information Theory
(1993)- et al.
Directed diffusion: a scalable and robust communication paradigm for sensor networks