Noise effect on signal quantization in an array of binary quantizers

doi:10.1016/j.sigpro.2018.06.010

Signal Processing

Volume 152, November 2018, Pages 265-272

https://doi.org/10.1016/j.sigpro.2018.06.010 Get rights and content

•
Based on mean square error (MSE) distortion, there is no noise-enhanced effect in the uniform quantizer.
•
In an array of identical binary quantizers combined with a linear decoder, a convex optimization problem is formulated and the optimal noise is derived by Gateaux differential.
•
In the case that the granular region is fixed, different types of noise are used to show the SR (SSR) effect on the quantization performance.
•
When the granular region is adjustable, quantization performances induced by various noises are shown as well.

Abstract

The effect of noise is examined in both the uniform quantizer and a stochastic quantizer. The stochastic quantizer is an array of identical binary quantizers combined with a linear decoder. In uniform quantization scenario, we find that noise cannot help decrease mean square error (MSE) distortion. However, in the array of binary quantizers with identical thresholds, noise may play a positive role, and stochastic resonance (SR) can be observed. First, based on MSE distortion, the optimal noise in the array is derived by Gateaux differential, which is shown to be a uniform noise. And then, some other noises, including uniform noise with zero mean, Gaussian noise, Laplacian noise, and discrete noise, are considered for comparison. In the case that the granular region is fixed, the quantization performances induced by those noises are shown, and the SR effects are discussed. Furthermore, when the granular region is adjustable, quantization performance may be better. Especially, the MSE distortion, in the optimal noise case, will approximate zero as the rate becomes high. At last, some further studies are discussed, which may extend our results.

Introduction

In some nonlinear systems, random noise may significantly enhance system performance. That phenomenon is described as stochastic resonance (SR), or generally, stochastic facilitation (SF) [1]. In recent years, the noise effects on signal processing and information transmission have been extensively discussed in SR literature [2], [3], [4], [5], [6].

Quantization, as a lossy source coding, can also benefit from randomization or noise. When random noise is introduced, the quantizer becomes a randomized one, and the signal is quantized stochastically. Particularly, Saldi et al. have stated that the random coding in Shannon’s rate-distortion theorem can be viewed as randomized quantizers [7], which highlights the significance of stochastic quantization in lossy compression.

In the stochastic quantization scheme, the statistical property of quantization error may be purposely controlled by additive noise. Wannamaker et al. have investigated such randomized quantizers, which are called dithered quantizers, in detail [8]. When a suitable noise is employed, e.g., noise with a triangular probability density function (pdf), the moments of quantization error can be independent of the input signal. Such properties are essential for the purpose of audio signal quantization. Moreover, if the additive noise can be subtracted from the output of the quantizer, the randomized system is named subtractively dithered quantizer [9]. Using the subtractive dithering, Li et al. preserved the probability distribution of input signal and enhanced the perceptual quality of audio and video coding [10]. Later, Saldi et al. considered a general problem in Ref. [7]. They have shown the existence of an optimal randomized quantizer under the constraint that the reconstructed signal has a given distribution. In addition, Akyol and Rose extended the conventional dithered quantization to a non-uniform one [11]. They proposed the optimal non-uniform randomized quantizer subject to the uncorrelated error constraint. It has been shown that for a Gaussian source, such a quantizer outperforms the conventional dithered one.

On the other hand, noise may assist in decreasing the average distortion in some stochastic quantization scenario. Especially in the system of parallel threshold devices [12], [13], noise usually shows suprathreshold stochastic resonance (SSR) effect. That is to say, signal-to-quantization-noise ratio, or mean square error (MSE) distortion, is optimized at some nonzero noise intensity. In Ref. [14], McDonnell et al. discussed SSR effect of various types of noise and showed stochastic quantization efficiency by considering different decoding methods. Xu et al. have also paid attention to the decoding of signal quantization [15]. They proposed an optimal weighted decoding method for stochastic quantization in an array of binary quantizers. Measured by MSE distortion, the SSR effect in the optimal weighted decoding is found to be better than that in linear decoding. Later, the authors investigated the weighted decoding scheme again [16], but elements in the SSR model are nonlinear sensors instead of binary quantizers. In addition, quantization with parallel channels has been studied by Goyal, too [17]. In that case, uniform quantizers are concatenated in parallel and random dither is added to the threshold of each quantizer. As Goyal has demonstrated, distortion of the randomly-designed quantizer exceeds the distortion of the optimal quantizer just by a constant factor.

As stated above, noise sometimes has the potential to improve quantization performance. Furthermore, it is of significance to gain the utmost improvement induced by noise. In this paper, when quantization performance is quantified by MSE distortion, we attempt to find the optimal noise which leads to the minimal distortion. Based on the existing results for some given noises, e.g., Refs. [13], [14], [18], this paper will consider an array of identical binary quantizers combined with a linear decoder as the quantization system, and then explore the optimal noise in the framework. Meanwhile, we will study the SR (SSR) effects in our quantization system. The rest of this paper is organized as follows. Section 2 presents a review of the classical uniform quantizer. Performance of the quantizer dithered with additive noise is also analyzed. In Section 3, in linear decoding scenario, the optimal noise is derived to minimize the MSE distortion in an array of binary quantizers. The corresponding optimal quantization performance is explicitly displayed in Section 4. For comparison, SR (SSR) effects of various suboptimal noises are also shown. In Section 5, conclusion and further discussion are presented.

Section snippets

Review of classical uniform quantizer

A general quantizer Q consists of an encoder and a decoder [19].

•
Encoder α: It assigns an index i to the signal x, i.e., $i = α (x)$ . For a quantizer with the number of quantization levels M, the set of indices is $I = {0, 1, \dots, M - 1},$ and the rate of the quantizer is $R = \log_{2} M$ .
•
Decoder β: It is a one-to-one map $I \to C,$ where $C$ is the codebook composed of output levels y_i’s, $C = {y_{0}, y_{1}, \dots, y_{M - 1}}$ . So the output of the quantizer is $\hat{x} = Q (x) = β (α (x)) = y_{i}$ if $i = α (x)$ .

A specific quantizer is the M-level (scalar) uniform one.

Optimal noise in an array of binary quantizers

As noise has a negative effect in the uniform quantizer,¹ we focus on another system, which is shown in Fig. 1, to discuss the noise effect on quantization performance. Such an encoder—an array of threshold components—has been widely used to exhibit phenomena of SR or SSR [14]. It is also of practical importance, since it can be used for analog to digital conversion [12]. In Ref. [18], that array, combined with a probability

Noise effect on quantization performance

In this section, noise’s effect on signal quantization in the array of binary quantizers is displayed for some types of signal, including uniform, Gaussian, and Laplacian signals. We assume that the granular region $[- L, L] = [- τ σ_{x}, τ σ_{x}],$ where σ_x is the standard deviation of signal and τ is a proportional factor (τ > 0). By using $L = τ σ_{x},$ the codebook ${y_{i} | i = 0, 1, \dots, M - 1}$ can be determined by Eq. (1). Here, $M = N + 1$ .

Not only the quantization performance induced by the optimal noise is shown, but the effects

Conclusion and discussion

The array of binary quantizers has been extensively investigated in SR literature. So many studies have exploited various types of noise and have discussed the noise-enhanced performance in this array. However, little attention has been paid to the optimal noise effect on signal quantization. In this paper, we have derived the optimal noise in the stochastic quantization scheme in Fig. 1. The corresponding quantization performance induced by the optimal noise has also been shown for different

Acknowledgments

The authors would like to thank the reviewers for their helpful comments and suggestions. This work has been supported by the National Natural Science Foundation of China (No. 61179027), the Open Foundation of National Engineering Research Center of Communications and Networking (No. GCZX001), the Qinglan Project of Jiangsu Province (No. QL06212006), and the Postgraduate Research & Practice Innovation Program of Jiangsu Province (No. KYLX15_0829).

References (22)

A.B. Akbay et al.
Noise benefits in joint detection and estimation problems
Signal Process.
(2016)
M.D. McDonnell et al.
Communication of uncoded sensor measurements through nanoscale binary-node stochastic pooling networks
Nano Commun. Netw.
(2010)
L. Xu et al.
Decoding suprathreshold stochastic resonance with optimal weights
Phys. Lett. A
(2015)
L. Xu et al.
Optimal weighted suprathreshold stochastic resonance with multigroup saturating sensors
Physica A
(2016)
H. Chen et al.
Noise-enhanced information systems
Proc. IEEE
(2014)
H. Chen et al.
Theory of the stochastic resonance effect in signal detection: part I–fixed detectors
IEEE Trans. Signal Process.
(2007)
S. Uhlich
Bayes risk reduction of estimators using artificial observation noise
IEEE Trans. Signal Process.
(2015)
N.G. Stocks
Suprathreshold stochastic resonance in multilevel threshold systems
Phys. Rev. Lett.
(2000)
Q. Zhai et al.
Optimal and suboptimal noises enhancing mutual information in threshold system
Fluctuation Noise Lett.
(2016)
N. Saldi et al.
Randomized quantization and source coding with constrained output distribution
IEEE Trans. Inf. Theory
(2015)

R.A. Wannamaker et al.

A theory of nonsubtractive dither

IEEE Trans. Signal Process.

(2000)

Cited by (11)

A mutual information-maximizing quantizer based on the noise-injected threshold array
2024, Digital Signal Processing: A Review Journal
Channel quantization, particularly designing optimal quantizers maximizing the mutual information between channel input and quantizer output, plays a great role in communications. This paper focuses on the mutual information-maximizing quantizer and explores stochastic resonance (SR) effect on quantization performance when the channel is constructed by a noise-injected threshold array. First, we present the structure of an optimal quantizer. Such a quantizer is determined by using optimal boundaries to partition the set of channel output into disjoint subsets consisting of consecutive integers. Next, the optimal binary quantizer is examined and the optimal noise in the array is derived. For non-optimal Gaussian noise, we find that noise helps to improve mutual information when the threshold is greater than the amplitude of input signal. This means SR occurs in subthreshold case. Moreover, optimal non-binary quantizers are obtained based on dynamic programming. In this case, the Gaussian noise's effect on enhancing mutual information is also demonstrated. At the same time, the impact of the number of threshold units or the quantization levels is explored. Finally, a non-Gaussian noise, i.e., Cauchy noise, is considered, and its SR effect is displayed as well. These results in this paper may be useful for channel coding.
Optimal added noise for minimizing distortion in quantizer-array linear estimation
2023, Chaos, Solitons and Fractals
Citation Excerpt :
It is proven that uniform noise gives the fastest MSE decrease in the small-noise limit among all zero-symmetric scale-family noise types with finite variance [69]. Further, the uniform noise, among all types of added noise, is also proven to yield the lowest MSE distortion of parallel binary quantizer-arrays on signal quantization [77]. It should be noted that the background noise is not considered in the above theoretical investigations.
For linearly estimating the input signal, the optimization of suprathreshold stochastic resonance in an array of $N$ parallel $M$ -ary quantizers is theoretically investigated. We first prove that the mean square error (MSE) of the designed quantizer-array is a convex functional with respect to the cumulative probability function (CDF) of the added noise. Then, for an arbitrary input signal and the quantization interval being $N$ times the decoding step size, we theoretically demonstrate that minimum MSE distortion can be obtained for optimal added uniform noise. Furthermore, for a uniform input signal, the optimality condition also holds if the system parameters and the boundary of the signal satisfy an inequality constraint condition. Moreover, under this condition, the optimal parameters of the quantizer-array can be also determined exactly. By applying both the optimal added noise and optimal parameters to the quantizer-array, the MSE can be further improved for larger array size $N$ or quantization order $M$ . Finally, the optimal added noise is also discussed for minimizing distortions of the quantizer-array linear estimation in the presence of the background noise.
Distributed Bayesian vector estimation using noise-optimized low-resolution sensor observations
2021, Digital Signal Processing: A Review Journal
Citation Excerpt :
Dithering has been widely investigated for reducing the mean square error (MSE) of deterministic or random parameter estimation [7,10,13,15], or for minimizing the Cramér-Rao lower bound (CRLB) [3,5,6], by means of an optimal amount of added noise. Stochastic resonance [16,17] also represents a broader class of noise-aided phenomena, and has been exploited for noise-improved information processing for various operations, including signal transmission [18–20], detection [21–24] and estimation [19,25,26]. Most of studies of noise-enhanced parameter estimation focus on estimating a scalar parameter [5,27–30,32,34].
The distributed Bayesian vector parameter estimation problem based on low-resolution observations is investigated in a network, where each node represents an ensemble of estimates from a large number of sensors. A noise-enhanced Bayesian vector estimator that benefits from artificially added noise is proposed. For a network node composed of a sufficiently large number of identical low-resolution sensors, a lemma governing the weight coefficients is proven, and low-cost calculation expressions of the designed estimator and its Bayesian mean square error (MSE) are derived by avoiding costly computations due to high-dimensional matrix inversions. Experimental results show that by intentionally adding an appropriate amount of noise to networks of the low-resolution sensors, the MSE of the designed Bayesian vector estimator can be significantly reduced.
Benefits of noise in M-estimators: Optimal noise level and probability density
2019, Physica A: Statistical Mechanics and its Applications
For the robust estimation of a location parameter, we consider a parallel array of maximum likelihood type estimators (M-estimators). We investigate the possibility of added noise as a design variable of the M-estimators, and characterize a nonzero optimal amount of added noise maximizing the efficiency for estimation. The added noise shows its benefits to the asymptotic efficiency of the M-estimator when the noise level and the noise probability density are optimally tuned. The optimal noise level can be theoretically derived by maximizing the asymptotic efficiency as the probability density of added noise is given. Based on the Parzen-window density estimation technique, we approximate the infinite-dimensional non-convex optimization of the optimal probability density of added noise as a simpler optimization problem with respect to a finite-dimensional vector under certain constraints. This approximate solution for the optimal probability density of added noise shows its feasibility for various M-estimators with an arbitrary array size, which is also validated by simulation results.
Optimal Added Noise for Minimizing Distortions in Quantizer-Array Linear Estimation
2022, SSRN
Formant Frequencies Estimation Based on Correlogram Method of Spectral Analysis and Binary-Sign Stochastic Quantization
2022, Studies in Systems, Decision and Control

View all citing articles on Scopus

View full text

Noise effect on signal quantization in an array of binary quantizers

Abstract

Introduction

Section snippets

Review of classical uniform quantizer

Optimal noise in an array of binary quantizers

Noise effect on quantization performance

Conclusion and discussion

Acknowledgments

Signal Process.

Nano Commun. Netw.

Phys. Lett. A

Physica A

Noise-enhanced information systems

Proc. IEEE

Theory of the stochastic resonance effect in signal detection: part I–fixed detectors

IEEE Trans. Signal Process.

Bayes risk reduction of estimators using artificial observation noise

IEEE Trans. Signal Process.

Suprathreshold stochastic resonance in multilevel threshold systems

Phys. Rev. Lett.

Optimal and suboptimal noises enhancing mutual information in threshold system

Fluctuation Noise Lett.

Randomized quantization and source coding with constrained output distribution

IEEE Trans. Inf. Theory

A theory of nonsubtractive dither

IEEE Trans. Signal Process.