MIAMI: MIxed Data Augmentation MIxture

Abstract

Performing data augmentation for mixed datasets remains an open challenge. We propose an adaptation of the Mixed Deep Gaussian Mixture Models (MDGMM) to generate such complex data. The MDGMM explicitly handles the different data types and learns a continuous latent representation of the data that captures their dependence structure and can be exploited to conduct data augmentation. We test the ability of our method to simulate crossings of variables that were rarely observed or unobserved during training. The performances are compared with recent competitors relying on Generative Adversarial Networks, Random Forest, Classification And Regression Trees, or Bayesian networks on the UCI Adult dataset.

Granted by the Research Chair DIALog under the aegis of the Risk Foundation, an initiative by CNP Assurances.

Appendices

Appendix

A Datasets Details

The variables of the Adult dataset are according to the UCI documentation:

• Income: binary (>50K, $<=$50K).

• Age: continuous.

• Workclass: categorical (Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked).

• Fnlwgt: continuous.

• Education-num: ordinal.

• Marital-status: categorical (Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse).

• Occupation: categorical (Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces).

• Relationship: categorical (Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried).

• Race: categorical (White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black).

• Sex: binary (Female, Male).

• Capital-gain: ordinal.

• Capital-loss: ordinal.

• Hours-per-week: continuous.

• Native-country: categorical (United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc.), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad &Tobago, Peru, Hong, Holand-Netherlands).

B Evaluation Metrics Details

Our overall criterion between a test dataset and a generated dataset is the association distance obtained as follows:

$$\begin{aligned} DA&= \displaystyle \frac{1}{P}\displaystyle \sum _{1\le i<j \le p}^p | M_{ij}(test) - M_{ij}(gen)| / M_{ij}(test), \end{aligned}$$

where p denotes the number of variables (14 in the Adult dataset), $P=(p^2-p)/2$, and $M_{ij}(test)$ (resp. $M_{ij}(gen))$ is the (i, j)th entry of the test (resp. generated) Association Matrix.

To measure the similarity between the dependence structures of the vectors formed by the three continuous variables (Age, Fnlwgt, and Hours) we used the multivariate Kullback Leibler divergence [12].

For qualitative data, we chose the mean absolute errors (MAE) between proportions. More precisely, for a kth intersection of modalities we consider

$$\begin{aligned} MAE(k)&= | p_k(test) - p_k(gen)|, \end{aligned}$$

where $p_k(test)$ (resp. $p_k(gen)$) stands for kth test (resp. generated) proportion. The final MAE is the mean of all the MAE(k) over all the possibilities.

C Additional Results: Unobserved Marginal Density Reconstruction

When it comes to reconstructing univariate densities, MIAMI generates well-identified unimodal densities contrary to DataSynthesizer which generates flat densities, or SynthPop-CART which generates multi-modal densities (Fig. 8). More precisely, Figs. 8 and 9 represent the estimations of the observed density versus the generated one for Age in the case of the Bivariate and Trivariate Unbalanced designs. We observe that MIAMI can recover the right distribution, yet not observed in the training set. It means that MIAMI captures well the dependence structure of such a partially unobserved variable. CTGAN and SynthPop-RF also seem to work well for both designs. DataSynthesizer shows a larger variance. This illustration shows that we cannot clearly decide between the methods by looking only at the marginal distributions. Only a criterion like the association distance can take into account a more complex multivariate dependence structure.

Fig. 8.

Density estimations for Age based on the test dataset (red) or based on the Bivariate Unbalanced design (black) (Color figure online)

Fig. 9.

Density estimations for Age based on the test dataset (red) or based on the Trivariate Unbalanced design (black) (Color figure online)