Skip to main content
SpringerLink
Log in
Book cover

International Conference on Neural Information Processing

ICONIP 2021: Neural Information Processing pp 196–204Cite as

Missing Glow Phenomenon: Learning Disentangled Representation of Missing Data

  • Conference paper
  • First Online:

  • 2285 Accesses

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1516))

Abstract

Learning from incomplete data has been recognized as one of the fundamental challenges in deep learning. There are many more or less complicated methods for processing missing data by neural networks in the literature. In this paper, we show that flow-based generative models can work directly on images with missing data to produce full images without missing parts. We name this behavior Missing Glow Phenomenon. We present experiments that document such behaviors and propose theoretical justification of such phenomena.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Batista, G.E., Monard, M.C., et al.: A study of k-nearest neighbour as an imputation method. In: HIS, vol. 87, pp. 251–260 (2002)

    Google Scholar 

  2. Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.: Neural ordinary differential equations. arXiv preprint arXiv:1806.07366 (2018)

  3. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. B (Methodol.) 39(1), 1–22 (1977)

    MathSciNet  MATH  Google Scholar 

  4. Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. arXiv preprint arXiv:1605.08803 (2016)

  5. Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning, vol. 1. MIT Press (2016)

    Google Scholar 

  6. Goodfellow, I.J., et al.: Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014)

  7. Gupta, M., et al.: Monotonic calibrated interpolated look-up tables. J. Mach. Learn. Res. 17(1), 3790–3836 (2016)

    MathSciNet  MATH  Google Scholar 

  8. Kingma, D.P., Dhariwal, P.: Glow: generative flow with invertible 1 \(\times \) 1 convolutions. arXiv preprint arXiv:1807.03039 (2018)

  9. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)

  10. Li, S.C.X., Marlin, B.: Learning from irregularly-sampled time series: a missing data perspective. In: International Conference on Machine Learning, pp. 5937–5946. PMLR (2020)

    Google Scholar 

  11. Little, R.J., Rubin, D.B.: Statistical Analysis with Missing Data, vol. 793. Wiley, Hoboken (2019)

    Google Scholar 

  12. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)

    Google Scholar 

  13. Mattei, P.A., Frellsen, J.: MIWAE: deep generative modelling and imputation of incomplete data sets. In: International Conference on Machine Learning, pp. 4413–4423. PMLR (2019)

    Google Scholar 

  14. McKnight, P.E., McKnight, K.M., Sidani, S., Figueredo, A.J.: Missing Data: A Gentle Introduction. Guilford Press (2007)

    Google Scholar 

  15. Rezende, D., Mohamed, S.: Variational inference with normalizing flows. In: International Conference on Machine Learning, pp. 1530–1538. PMLR (2015)

    Google Scholar 

  16. Richardson, T.W., Wu, W., Lin, L., Xu, B., Bernal, E.A.: MCFlow: Monte Carlo flow models for data imputation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14205–14214 (2020)

    Google Scholar 

  17. Rubin, D.B.: Inference and missing data. Biometrika 63(3), 581–592 (1976)

    Article  MathSciNet  Google Scholar 

  18. Sharpe, P.K., Solly, R.: Dealing with missing values in neural network-based diagnostic systems. Neural Comput. Appl. 3(2), 73–77 (1995)

    Article  Google Scholar 

  19. Śmieja, M., Struski, Ł, Tabor, J., Marzec, M.: Generalized RBF kernel for incomplete data. Knowl. Based Syst. 173, 150–162 (2019)

    Article  Google Scholar 

  20. Smieja, M., Struski, Ł., Tabor, J., Zieliński, B., Spurek, P.: Processing of missing data by neural networks. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 2724–2734 (2018)

    Google Scholar 

  21. Sovilj, D., et al.: Extreme learning machine for missing data using multiple imputations. Neurocomputing 174, 220–231 (2016)

    Article  Google Scholar 

  22. Xie, J., Xu, L., Chen, E.: Image denoising and inpainting with deep neural networks. Adv. Neural. Inf. Process. Syst. 25, 341–349 (2012)

    Google Scholar 

  23. Yeh, R.A., Chen, C., Yian Lim, T., Schwing, A.G., Hasegawa-Johnson, M., Do, M.N.: Semantic image inpainting with deep generative models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5485–5493 (2017)

    Google Scholar 

  24. Yoon, J., Jordon, J., Schaar, M.: GAIN: missing data imputation using generative adversarial nets. In: International Conference on Machine Learning, pp. 5689–5698. PMLR (2018)

    Google Scholar 

  25. Zhang, Y., Zheng, Z., Hu, R.: Super resolution using segmentation-prior self-attention generative adversarial network. arXiv preprint arXiv:2003.03489 (2020)

Download references

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland

    Marcin Sendera, Łukasz Struski & Przemysław Spurek

Authors
  1. Marcin Sendera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Łukasz Struski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Przemysław Spurek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcin Sendera .

Editor information

Editors and Affiliations

  1. Sampoerna University, Jakarta, Indonesia

    Teddy Mantoro

  2. Kyungpook National University, Daegu, Korea (Republic of)

    Minho Lee

  3. Sampoerna University, Jakarta, Indonesia

    Media Anugerah Ayu

  4. Murdoch University, Murdoch, WA, Australia

    Kok Wai Wong

  5. Universitas Indonesia, Depok, Indonesia

    Achmad Nizar Hidayanto

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sendera, M., Struski, Ł., Spurek, P. (2021). Missing Glow Phenomenon: Learning Disentangled Representation of Missing Data. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1516. Springer, Cham. https://doi.org/10.1007/978-3-030-92307-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-92307-5_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-92306-8

  • Online ISBN: 978-3-030-92307-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation