Skip to main content
SpringerLink
Log in

Primal Heuristics for Wasserstein Barycenters

  • Conference paper
  • First Online:
Integration of Constraint Programming, Artificial Intelligence, and Operations Research (CPAIOR 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12296))

Abstract

This paper presents primal heuristics for the computation of Wasserstein Barycenters of a given set of discrete probability measures. The computation of a Wasserstein Barycenter is formulated as an optimization problem over the space of discrete probability measures. In practice, the barycenter is a discrete probability measure which minimizes the sum of the pairwise Wasserstein distances between the barycenter itself and each input measure. While this problem can be formulated using Linear Programming techniques, it remains a challenging problem due to the size of real-life instances. In this paper, we propose simple but efficient primal heuristics, which exploit the properties of the optimal plan obtained while computing the Wasserstein Distance between a pair of probability measures. In order to evaluate the proposed primal heuristics, we have performed extensive computational tests using random Gaussian distributions, the MNIST handwritten digit dataset, and the Fashion MNIST dataset introduced by Zalando. We also used Translated MNIST, a modification of MNIST which contains original images, rescaled randomly and translated into a larger image. We compare the barycenters computed by our heuristics with the exact solutions obtained with a commercial Linear Programming solver, and with a state-of-the-art algorithm based on Gaussian convolutions. Our results show that the proposed heuristics yield in very short run time and an average optimality gap significantly smaller than 1%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agueh, M., Carlier, G.: Barycenters in the Wasserstein space. SIAM J. Math. Anal. 43(2), 904–924 (2011)

    Article  MathSciNet  Google Scholar 

  2. Ahuja, R.K., Magnanti, T.L., Orlin, J.B.: Network flows: Theory, Algorithms, and Applications. Cambridge, Mass.: Alfred P. Sloan School of Management, Massachusetts Institute of Technology (1988)

    Google Scholar 

  3. Anderes, E., Borgwardt, S., Miller, J.: Discrete Wasserstein Barycenters: optimal transport for discrete data. Math. Methods Oper. Res. 84(2), 389–409 (2016)

    Article  MathSciNet  Google Scholar 

  4. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017)

  5. Auricchio, G., Bassetti, F., Gualandi, S., Veneroni, M.: Computing kantorovich-wasserstein distances on \( d \)-dimensional histograms using \((d+ 1) \)-partite graphs. In: Advances in Neural Information Processing Systems, pp. 5793–5803 (2018)

    Google Scholar 

  6. Auricchio, G., Bassetti, F., Gualandi, S., Veneroni, M.: Computing Wasserstein Barycenters via linear programming. In: Rousseau, L.-M., Stergiou, K. (eds.) CPAIOR 2019. LNCS, vol. 11494, pp. 355–363. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19212-9_23

    Chapter  MATH  Google Scholar 

  7. Bassetti, F., Gualandi, S., Veneroni, M.: On the computation of Kantorovich-Wasserstein distances between 2D-histograms by uncapacitated minimum cost flows. arXiv preprint arXiv:1804.00445 (2018)

  8. Bonneel, N., Coeurjolly, D.: Spot: sliced partial optimal transport. ACM Trans. Graph. 38(4), 1–13 (2019)

    Article  Google Scholar 

  9. Bonneel, N., Van De Panne, M., Paris, S., Heidrich, W.: Displacement interpolation using Lagrangian mass transport. ACM Trans. Graph. 30, 158–160 (2011)

    Article  Google Scholar 

  10. Cuturi, M.: Sinkhorn distances: lightspeed computation of optimal transport. In: Advances in Neural Information Processing Systems, pp. 2292–2300 (2013)

    Google Scholar 

  11. Cuturi, M., Doucet, A.: Fast computation of Wasserstein Barycenters. In: International Conference on Machine Learning, pp. 685–693 (2014)

    Google Scholar 

  12. Dvurechenskii, P., Dvinskikh, D., Gasnikov, A., Uribe, C., Nedich, A.: Decentralize and randomize: faster algorithm for Wasserstein Barycenters. In: Advances in Neural Information Processing Systems, pp. 10760–10770 (2018)

    Google Scholar 

  13. Flamary, R., Courty, N.: POT: Python Optimal Transport library (2017). https://github.com/rflamary/POT

  14. Goldberg, A.V., Tardos, É., Tarjan, R.: Network flow algorithms. Cornell University Operations Research and Industrial Engineering, Technical report (1989)

    Google Scholar 

  15. Koopmans, T.C.: Optimum utilization of the transportation system. Econom. J. Econom. Soc. 17, 136–146 (1949)

    Google Scholar 

  16. Kusner, M., Sun, Y., Kolkin, N., Weinberger, K.: From word embeddings to document distances. In: International Conference on Machine Learning, pp. 957–966 (2015)

    Google Scholar 

  17. LeCun, Y., Cortes, C., Burges, C.J.: MNIST dataset. http://yann.lecun.com/exdb/mnist/. Accessed 12 Mar 2019

  18. Levina, E., Bickel, P.: The Earth mover’s distance is the mallows distance: some insights from statistics. In: IEEE International Conference on Computer Vision, vol. 2, pp. 251–256 (2001)

    Google Scholar 

  19. Ling, H., Okada, K.: An efficient earth mover’s distance algorithm for robust histogram comparison. IEEE Trans. Pattern Anal. Mach. Intell. 29(5), 840–853 (2007)

    Article  Google Scholar 

  20. Luise, G., Salzo, S., Pontil, M., Ciliberto, C.: Sinkhorn Barycenters with free support via Frank-Wolfe algorithm. arXiv preprint arXiv:1905.13194 (2019)

  21. Monge, G.: Mémoire sur la théorie des déblais et des remblais. Histoire de l’Académie Royale des Sciences de Paris (1781)

    Google Scholar 

  22. Panaretos, V.M., Zemel, Y.: Statistical aspects of Wasserstein distances. Ann. Rev. Stat. Appl. 6, 405–431 (2019)

    Article  MathSciNet  Google Scholar 

  23. Pele, O., Werman, M.: Fast and robust earth mover’s distances. In: IEEE International Conference on Computer vision, pp. 460–467 (2009)

    Google Scholar 

  24. Peyré, G., Cuturi, M., et al.: Computational optimal transport. Found. Trends Mach. Learn. 11(5–6), 355–607 (2019)

    Article  Google Scholar 

  25. Qian, Y., Pan, S.: A proximal ALM method for computing Wasserstein Barycenter in d2-clustering of discrete distributions. arXiv preprint arXiv:1809.05990 (2018)

  26. Rabin, J., Ferradans, S., Papadakis, N.: Adaptive color transfer with relaxed optimal transport. In: 2014 IEEE International Conference on Image Processing, pp. 4852–4856 (2014)

    Google Scholar 

  27. Rubner, Y., Tomasi, C., Guibas, L.J.: The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vision 40(2), 99–121 (2000)

    Article  Google Scholar 

  28. Santambrogio, F.: Optimal Transport for Applied Mathematicians, pp. 99–102. Birkäuser (2015)

    Google Scholar 

  29. Solomon, J., et al.: Convolutional Wasserstein distances: efficient optimal transportation on geometric domains. ACM Trans. Graph. 34(4), 66 (2015)

    Article  Google Scholar 

  30. Staib, M., Claici, S., Solomon, J.M., Jegelka, S.: Parallel streaming Wasserstein Barycenters. In: Advances in Neural Information Processing Systems, pp. 2647–2658 (2017)

    Google Scholar 

  31. Vershik, A.M.: Long history of the Monge-Kantorovich transportation problem. Math. Intell. 35(4), 1–9 (2013)

    Article  MathSciNet  Google Scholar 

  32. Villani, C.: Optimal Transport: Old and New, vol. 338. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  33. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Matematica “F. Casorati”, Università degli Studi di Pavia, Pavia, Italy

    Pierre-Yves Bouchet & Stefano Gualandi

  2. Dept. de Mathématiques et Génie Industriel, GERAD-Polytechnique Montréal, Montreal, Canada

    Pierre-Yves Bouchet

  3. Dept. de Mathématiques et Génie Industriel, CIRRELT-Polytechnique Montréal, Montreal, Canada

    Louis-Martin Rousseau

Authors
  1. Pierre-Yves Bouchet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Gualandi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Louis-Martin Rousseau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefano Gualandi .

Editor information

Editors and Affiliations

  1. LAAS - CNRS, Toulouse, France

    Emmanuel Hebrard

  2. TU Wien, Vienna, Wien, Austria

    Nysret Musliu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bouchet, PY., Gualandi, S., Rousseau, LM. (2020). Primal Heuristics for Wasserstein Barycenters. In: Hebrard, E., Musliu, N. (eds) Integration of Constraint Programming, Artificial Intelligence, and Operations Research. CPAIOR 2020. Lecture Notes in Computer Science(), vol 12296. Springer, Cham. https://doi.org/10.1007/978-3-030-58942-4_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58942-4_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58941-7

  • Online ISBN: 978-3-030-58942-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation