A new computational approach for estimation of the Gini index based on grouped data

Abstract

Many government agencies still rely on the grouped data as the main source of information for calculation of the Gini index. Previous research showed that the Gini index based on the grouped data suffers the first and second-order correction bias compared to the Gini index computed based on the individual data. Since the accuracy of the estimated correction bias is subject to many underlying assumptions, we propose a new method and name it D-Gini, which reduces the bias in Gini coefficient based on grouped data. We investigate the performance of the D-Gini method on an open-ended tail interval of the income distribution. The results of our simulation study showed that our method is very effective in minimizing the first and second order-bias in the Gini index and outperforms other methods previously used for the bias-correction of the Gini index based on grouped data. Three data sets are used to illustrate the application of this method.

Appendices

Proof of Lemma 1

Proof

Let $Y_1$, $Y_2$ be two independent copies of Y. Define the random variable $U=[f(Y_2)-f(Y_1)][h(Y_2)-h(Y_1)]$. Note that $U \ge 0$ almost surely. Indeed, if $Y_2\ge Y_1$, then because $f$ and $h$ are non-decreasing functions, $f(Y_2)-f(Y_1)\ge 0$ and $h(Y_2)-h(Y_1)\ge 0$, thus $U \ge 0$. If $Y_2 \le Y_1$, then $f(Y_2)-f(Y_1) \le 0$ and $h(Y_2)-h(Y_1)\le 0$, thus $U \ge 0$. Since $Y_1$ and $Y_2$ are independent and have the same distribution, ${\mathbb {E}}[f(Y_1)h(Y_1)]= {\mathbb {E}}[f(Y_2)h(Y_2)]={\mathbb {E}}[f(Y)h(Y)]$, ${\mathbb {E}}[f(Y_1)]={\mathbb {E}}[f(Y_2)]={\mathbb {E}}[f(Y)]$, and ${\mathbb {E}}[h(Y_1)]={\mathbb {E}}[h(Y_2)]={\mathbb {E}}[h(Y)]$. Therefore,

$$\begin{aligned} {\mathbb {E}}[U]= & {} {\mathbb {E}}[(f(Y_2)-f(Y_1))(h(Y_2)-h(Y_1))]\\= & {} {\mathbb {E}}[f(Y_2)h(Y_2)] - {\mathbb {E}}[f(Y_2)]{\mathbb {E}}[h(Y_1)]-{\mathbb {E}}[f(Y_1)]{\mathbb {E}}[h(Y_2)]+ {\mathbb {E}}[f(Y_1)h(Y_1)]\\= & {} 2{\mathbb {E}}[f(Y)h(Y)]-2{\mathbb {E}}[f(Y)]{\mathbb {E}}[h(Y)]\ge 0. \end{aligned}$$

Dividing both sides of this inequality by 2 and adding ${\mathbb {E}}[f(Y)]{\mathbb {E}}[h(Y)]$ to both sides gives Lemma 1. $\square$

Proof of Lemma 2

Proof

Let $D \subset \{1,\ldots ,n\}$ be a non-empty set and let $f$, $h$ be numbers indexed by $i\in D$, if $f_i\le f_j$ and $h_i \le h_j$ for all $i,j\in D$ with $i<j$, then Lemma 1 implies $f(i)=f_i$ and $h(i)=h_i$. Here, the random variable is $Y=i$ and it is uniformly distributed on D. This yields that

$$\begin{aligned} \Big (\frac{1}{|D|}\sum _{i\in D} f_i\Big )\Big (\frac{1}{|D|}\sum _{i\in D} h_i\Big )={\mathbb {E}}[f(Y)]{\mathbb {E}}[h(Y)]\le {\mathbb {E}}[f(Y)h(Y)]= \Big (\frac{1}{|D|}\sum _{i\in D} f_ih_i\Big ). \end{aligned}$$

(13)

Recall that $P_1, \ldots , P_K$ are equal size subsets of $\{1,\ldots , n\}$ such that $|P_g| =m$ for $1\le g\le K$, $\phi _g=\frac{1}{|P_g|}\sum _{i\in P_g}y^{*}_i$, and $R^{K}_g=\frac{1}{|P_g|}\sum _{i\in P_g}R^{*}_i$, and ${\bar{\phi }} = \frac{1}{K}\sum _{g=1}^{K}\phi _g$. Since $y^{*}_1\le \cdots \le y^{*}_n$ and $R^{*}_1 \le \cdots \le R^{*}_n$, Eq. (13) leads to

$$\begin{aligned} \phi _gR^{K}_g=\left( \frac{1}{|P_g|}\sum _{i\in P_g}y^{*}_i\right) \left( \frac{1}{|P_g|}\sum _{i\in P_g}R^{*}_i\right) \le \frac{1}{|P_g|}\sum _{i\in P_g}y^{*}_iR^{*}_i=\frac{1}{m}\sum _{i\in P_g}y^{*}_iR^{*}_i. \end{aligned}$$

(14)

Therefore,

$$\begin{aligned} G^{*}_n= & {} \frac{2 \sum _{i=1}^{n} y^{*}_i R^{*}_i}{n \bar{\phi }} - 1 =\frac{2 \sum _{g=1}^{K}(\sum _{i \in P_i} y^{*}_i R^{*}_i)}{n \bar{\phi }} - 1\\= & {} \frac{2 \sum _{g=1}^{K}\left( \sum _{i \in P_i} y^{*}_i R^{*}_i\right) }{mK \bar{\phi }} - 1 =\frac{2 \sum _{g=1}^{K}\left( \frac{1}{m}\sum _{i\in P_i} y^{*}_i R^{*}_i\right) }{K \bar{\phi }} - 1\\\ge & {} \frac{2 \sum _{g=1}^{K}\phi _g R^{K}_g}{K \bar{\phi }} - 1= G^{K}_n. \end{aligned}$$

$\square$

Proof of Lemma 3

Proof

The maximum order statistic, $Y_{(n)}$ has the following density function

$$\begin{aligned} g_{(n)}(y)= \frac{n}{\lambda }[1-e^{-\frac{(y-L)}{\lambda }}]^{n-1} e^{-\frac{(y-L)}{\lambda }}, \quad L< y < \infty . \end{aligned}$$

(15)

The expected value of the maximum order statistic is

$$\begin{aligned} {\mathbb {E}}[Y_{(n)}]=\frac{n}{\lambda } \int _L^{\infty }y [1-e^{-\frac{(y-L)}{\lambda }}]^{n-1} e^{-\frac{(y-L)}{\lambda }}dy. \end{aligned}$$

(16)

Since $0< e^{-\frac{(y-L)}{\lambda }} < 1$ for $y > L$, by using the binomial series expansion we have

$$\begin{aligned} {\mathbb {E}}[Y_{(n)}]=\frac{n}{\lambda } \int _L^{\infty } y \sum _{j=0}^{n-1}(-1)^j \left( {\begin{array}{c}n-1\\ j\end{array}}\right) e^{-\frac{(y-L)}{\lambda }(j+1)}dy. \end{aligned}$$

The quantity inside summation is absolutely integrable and by interchanging the summation with integration we obtain

$$\begin{aligned} {\mathbb {E}}[Y_{(n)}]= & {} \frac{n}{\lambda }\sum _{j=0}^{n-1}(-1)^j \left( {\begin{array}{c}n-1\\ j\end{array}}\right) \int _L^{\infty }y e^{-\frac{(y-L)}{\lambda }(j+1)}dy\\= & {} L\sum _{j=0}^{n-1}(-1)^j \left( {\begin{array}{c}n-1\\ j\end{array}}\right) \frac{n}{(j+1)} + \lambda \sum _{j=0}^{n-1}(-1)^{j} \left( {\begin{array}{c}n-1\\ j\end{array}}\right) \frac{1}{(j+1)^2},\\ \end{aligned}$$

where both summands represent harmonic sequences. The first summand is equal to 1 and the second summands represent the n-harmonic sequence. Since the sum of the reciprocals of the first n natural numbers is $\sum _{j=1}^{n}\frac{1}{j}$, the sum of n-harmonic sequence diverges slowly due to the logarithmic growth and it is approximated by $\gamma$-Euler–Mascheroni Constant and a small error term $\epsilon _n\approx \frac{1}{2n}$ that vanishes as n goes to infinity [refer to Alabdulmohsin (2018) for more details about these terms]. Based on well known facts about the sum of the n-harmonic sequence, we obtain the final result

$$\begin{aligned} {\mathbb {E}}[Y_{(n)}]= L + \lambda [\ln (n)+\gamma +\epsilon _n]\le L + \lambda [\ln (n) + 1]. \end{aligned}$$

(17)

$\square$