Skip to main content
SpringerLink
Log in

A Multiscale Semi-Smooth Newton Method for Optimal Transport

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Our goal is to solve the large-scale linear programming (LP) formulation of Optimal Transport (OT) problems efficiently. Our key observations are: (i) the primal solutions of the LP problems are theoretically very sparse; (ii) the cost function is usually equipped with good geometric properties. The former motivates us to eliminate the majority of the variables, while the latter easily enables us to exploit a hierarchical multiscale structure. Each level in this structure corresponds to a standard OT problem, whose solution can be obtained by solving a series of restricted OT problems by fixing most of the primal variables to zeros and using the semi-smooth Newton method. We improve the performance of computing the semi-smooth Newton direction by forming and solving a much smaller symmetric positive-definite system whose matrix can be written explicitly according to the sparsity patterns. Extensive numerical experiments show that our algorithm is quite efficient compared to the state-of-the-art methods such as a multiscale implementation of the CPLEX’s Networkflow algorithm and the SparseSinkhorn method, due to its ability to solve problems at a much larger scale and obtain the optimal solution in less time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://decsai.ugr.es/cvg/introduccion.php.

  2. https://bernhard-schmitzer.github.io/MultiScaleOT/build/html/index.html

References

  1. Agueh, Martial, Carlier, Guillaume: Barycenters in the wasserstein space. SIAM J. Math. Anal. 43, 904–924 (2011)

    Article  MathSciNet  Google Scholar 

  2. Ahujia, R.K., Magnanti, Thomas L., Orlin, James B.: Network flows: Theory, algorithms and applications. Rentice-Hall, New Jersey (1993)

    Google Scholar 

  3. Bertsekas, Dimitri P.: The auction algorithm: A distributed relaxation method for the assignment problem. Ann. Op. Res. 14, 105–123 (1988)

    Article  MathSciNet  Google Scholar 

  4. Cuturi, Marco: Sinkhorn distances: Lightspeed computation of optimal transport. In: Advances in neural information processing systems, pp. 2292–2300 (2013)

  5. Cuturi, Marco, Doucet, Arnaud: Fast computation of wasserstein barycenters (2014)

  6. Franklin, Joel, Lorenz, Jens: On the scaling of multidimensional matrices. Linear Algebra Appl. 114, 717–735 (1989)

    Article  MathSciNet  Google Scholar 

  7. Fu, Anthony Y., Wenyin, Liu, Deng, Xiaotie: Detecting phishing web pages with visual similarity assessment based on earth mover’s distance (emd). IEEE Trans. Depend. Secure Comput. 3, 301–311 (2006)

    Article  Google Scholar 

  8. Gerber, Samuel, Maggioni, Mauro: Multiscale strategies for computing optimal transport. J. Mach. Learn. Res. 18, 2440–2471 (2017)

    MathSciNet  MATH  Google Scholar 

  9. Grauman, Kristen, Darrell, Trevor: Fast contour matching using approximate earth mover’s distance, in Computer Vision and Pattern Recognition, 2004. CVPR 2004. In: Proceedings of the 2004 IEEE Computer Society Conference on, vol. 1, pp. I–I. IEEE (2004)

  10. Kendal, Dave, Hauser, Cindy E., Garrard, Georgia E., Jellinek, Sacha, Giljohann, Katherine M., Moore, Joslin L.: Quantifying plant colour and colour difference as perceived by humans using digital images. PLoS One 8, e72296 (2013)

    Article  Google Scholar 

  11. Knight, Philip A.: The sinkhorn-knopp algorithm: convergence and applications. SIAM J. Matrix Anal. Appl. 30, 261–275 (2008)

    Article  MathSciNet  Google Scholar 

  12. Kosowsky, Jeffrey J., Yuille, Alan L.: The invisible hand algorithm: Solving the assignment problem with statistical physics. Neural Netw. 7, 477–490 (1994)

    Article  Google Scholar 

  13. Kuhn, Harold W.: The hungarian method for the assignment problem. Naval Res. Log. (NRL) 2, 83–97 (1955)

    Article  MathSciNet  Google Scholar 

  14. Li, Yongfeng, Wen, Zaiwen, Yang, Chao, Yuan, Ya.-xiang: A semismooth newton method for semidefinite programs and its applications in electronic structure calculations. SIAM J. Sci. Comput. 40, A4131–A4157 (2018)

    Article  MathSciNet  Google Scholar 

  15. Liu, Jialin, Yin, Wotao, Li, Wuchen, Chow, Yat Tin: Multilevel optimal transport: a fast approximation of wasserstein-1 distances. SIAM J. Sci. Comput. 43, A193–A220 (2021)

    Article  MathSciNet  Google Scholar 

  16. Mérigot, Quentin: A multiscale approach to optimal transport. In: Computer Graphics Forum, vol. 30, pp. 1583–1592. Wiley Online Library (2011)

  17. Rubner, Yossi, Tomasi, Carlo, Guibas, Leonidas J.: The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vision 40, 99–121 (2000)

    Article  Google Scholar 

  18. Schmitzer, Bernhard: A sparse multiscale algorithm for dense optimal transport. J. Math. Imag. Vision 56, 238–259 (2016)

    Article  MathSciNet  Google Scholar 

  19. Schmitzer, Bernhard: Stabilized sparse scaling algorithms for entropy regularized transport problems. SIAM J. Sci. Comput. 41, A1443–A1481 (2019)

    Article  MathSciNet  Google Scholar 

  20. Schrieber, Jörn., Schuhmacher, Dominic, Gottschlich, Carsten: Dotmark-a benchmark for discrete optimal transport. IEEE Access 5, 271–282 (2017)

    Article  Google Scholar 

  21. Sharify, Meisam, Gaubert, Stéphane, Grigori, Laura: Solution of the optimal assignment problem by diagonal scaling algorithms (2011). arXiv preprint arXiv:1104.3830

  22. Tarjan, Robert E.: Dynamic trees as search trees via euler tours, applied to the network simplex algorithm. Math. Progr. 78, 169–177 (1997)

    MathSciNet  MATH  Google Scholar 

  23. Villani, Cédric.: Optical transport: old and new, vol. 338. Springer Science & Business Media, Berlin (2008)

    Google Scholar 

Download references

Acknowledgements

The authors are grateful to the AE and two anonymous referees for their valuable comments and suggestions. They would like to thank Yongfeng Li and Xiang Meng for the valuable discussion on the semi-smooth Newton method for the OT problem.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Peking University, Beijing, China

    Yiyang Liu

  2. Beijing International Center for Mathematical Research, Center for Data Science and College of Engineering, Peking University, Beijing, China

    Zaiwen Wen

  3. DAMO Academy, Alibaba Group U.S, Bellevue, WA, USA

    Wotao Yin

Authors
  1. Yiyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zaiwen Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wotao Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zaiwen Wen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Z. Wen: Research supported in part by the NSFC grant 11831002.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Wen, Z. & Yin, W. A Multiscale Semi-Smooth Newton Method for Optimal Transport. J Sci Comput 91, 39 (2022). https://doi.org/10.1007/s10915-022-01813-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-022-01813-y

Keywords

Mathematics Subject Classification

Search

Navigation