Skip to main content
SpringerLink
Log in

A Fast Solver for Generalized Optimal Transport Problems Based on Dynamical System and Algebraic Multigrid

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

This work provides an inexact primal-dual algorithm for a large class of optimal transport problems. It is based on the implicit Euler discretization of a proper dynamical system for linearly constrained convex optimization problems, and by using the tool of Lyapunov function, the global (super-)linear convergence rate is established for the objective residual and feasibility violation. The presented method contains an inner problem that possesses a strong semismoothness property, which motivates the use of the semismooth Newton iteration. In addition, by exploring the hidden structure of the problem itself, the linear equation arising from the Newton step is transferred equivalently into a graph Laplacian system, for which a robust algebraic multigrid method is proposed and analyzed via the famous Xu–Zikatanov identity. Finally, numerical tests are provided to validate the efficiency of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availibility statement

The datasets generated during and/or analysed during the current study are available on request.

References

  1. Ahuja, B.K.: Network Flows: Theory, Algorithms, and Applications. Prentice Hall, (1993)

  2. Altschuler, J., Niles-Weed, J., Rigollet, P.: Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA (2017)

  3. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: Proceedings of the 34 th International Conference on Machine Learning, volume 70, Sydney, Australia, (2017). PMLR

  4. Bai, Z., Chu, D., Tan, R.C.E.: Computing the nearest doubly stochastic matrix with a prescribed entry. SIAM J. Sci. Comput. 29(2), 635–655 (2007)

    MathSciNet  MATH  Google Scholar 

  5. Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics. Springer Science+Business Media, New York (2011)

    MATH  Google Scholar 

  6. Benamou, J.-D.: Optimal transportation, modelling and numerical simulation. Acta Numer. 30, 249–325 (2021)

    MathSciNet  MATH  Google Scholar 

  7. Benamou, J.-D., Carlier, G., Cuturi, M., Nenna, L., Peyré, G.: Iterative Bregman projections for regularized transportation problems. SIAM J. Sci. Comput. 37(2), A1111–A1138 (2015)

    MathSciNet  MATH  Google Scholar 

  8. Bertsekas, D.P.: Auction algorithms for network flow problems: a tutorial introduction. Comput. Optim. Appl. 1(1), 7–66 (1992)

    MathSciNet  MATH  Google Scholar 

  9. Bertsimas, D., Tsitsiklis, J.: Introduction to Linear Optimization. Athena Scientific, (1997)

  10. Bramble, J.H., Pasciak, J.E., Xu, J.: Parallel multilevel preconditioners. Math. Comput. 55(191), 1–22 (1990)

    MathSciNet  MATH  Google Scholar 

  11. Brauer, C., Clason, C., Lorenz, D., Wirth, B.: A Sinkhorn-Newton method for entropic optimal transport. arXiv:1710.06635, (2018)

  12. Brenner, S.C., Scott, L.R.: The Mathematical Theory of Finite Element Methods. Number 15 in Texts in Applied Mathematics. Springer, New York, NY, 3rd edition, (2008)

  13. Brezis, H.: Remarks on the Monge-Kantorovich problem in the discrete setting. Comptes Rendus Mathematique 356(2), 207–213 (2018)

    MathSciNet  MATH  Google Scholar 

  14. Briggs, W.L., Henson, V.E., McCormick, S.F.: A Multigrid Tutorial, 2nd edn. Society for Industrial and Applied Mathematics, USA (2000)

  15. Brualdi, R.: Combinatorial Matrix Classes. Cambridge University Press, New York (2006)

    MATH  Google Scholar 

  16. Caffarelli, L., McCann, R.: Free boundaries in optimal transport and Monge-Ampère obstacle problems. Ann. Math. 171(2), 673–730 (2010)

    MathSciNet  MATH  Google Scholar 

  17. Chen, L.: \(i\)FEM: an integrated finite element methods package in MATLAB. Technical report, (2009)

  18. Chen, L.: Deriving the X-Z identity from auxiliary space method. In: Huang, Y., Kornhuber, R., Widlund, O., Xu, J. (eds.) Domain Decomposition Methods in Science and Engineering XIX, pp. 309–316. Springer, Berlin (2011)

    Google Scholar 

  19. Chen, L., Hu, X., Wise, S.M.: Convergence analysis of the fast subspace descent methods for convex optimization problems. Math. Comput. 89(325), 2249–2282 (2020)

    MathSciNet  MATH  Google Scholar 

  20. Chen, L., Nochetto, R.H., Xu, J.: Optimal multilevel methods for graded bisection grids. Numer. Math. 120, 1–34 (2012)

    MathSciNet  MATH  Google Scholar 

  21. Chizat, L., Peyré, G., Schmitzer, B., Vialard, F.-X.: Scaling algorithms for unbalanced optimal transport problems. Math. Comput. 87(314), 2563–2609 (2018)

    MathSciNet  MATH  Google Scholar 

  22. Clarke, F.H.: Optimization and Nonsmooth Analysis. Number 5 in Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, (1987)

  23. Cominetti, R., Martín, J.S.: Asymptotic analysis of the exponential penalty trajectory in linear programming. Math. Program. 67(1–3), 169–187 (1994)

    MathSciNet  MATH  Google Scholar 

  24. Courty, N., Flamary, R., Tuia, D., Rakotomamonjy, A.: Optimal transport for domain adaptation. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1853–1865 (2017)

    Google Scholar 

  25. Cuturi, M.: Sinkhorn distances: Lightspeed computation of optimal transport. Adv. Neural Inform. Process. Syst. 26, 2292–2300 (2013)

    Google Scholar 

  26. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Number 16 in Classics in applied mathematics. Society for Industrial and Applied Mathematics, Philadelphia, (1996)

  27. Dvurechensky, P., Gasnikov, A., Kroshnin, A.: Computational optimal transport: complexity by accelerated gradient descent is better than by Sinkhorn’s algorithm. In Proceedings of the 35 th International Conference on Machine Learning, volume 80, Stockholm, Sweden, (2018). PMLR

  28. Facchinei, F., Pang, J.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. 2. Springer, New York (2003)

    MATH  Google Scholar 

  29. Figalli, A.: The optimal partial transport problem. Arch. Ration. Mech. Anal. 195(2), 533–560 (2010)

    MathSciNet  MATH  Google Scholar 

  30. Hackbusch, W.: Multi-Grid Methods and Applications. Springer, Berlin (2011)

    MATH  Google Scholar 

  31. Hu, H., Im, H., Li, X., Wolkowicz, H.: A semismooth Newton-type method for the nearest doubly stochastic matrix problem. Math. Oper. Res. (2023). https://doi.org/10.1287/moor.2023.1382

    Article  Google Scholar 

  32. Hu, J., Luo, H., Zhang, Z.: An efficient semismooth Newton-AMG-based inexact primal-dual algorithm for generalized transport problems. arXiv:2207.14082, (2022)

  33. Hug, R., Maitre, E., Papadakis, N.: Multi-physics optimal transportation and image interpolation. ESAIM: Math. Model. Numer. Anal., 49(6), 1671–1692 (2015)

  34. Jungnickel, D.: Graphs, Networks, and Algorithms. Number 5 in Algorithms and Computation in Mathematics. Springer, Berlin, 2nd edition, (2005)

  35. Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B., Xing, E.: Neural architecture search with Bayesian optimisation and optimal transport. In Advances in Neural Information Processing Systems 31, (2018)

  36. Kantorovich, L.: On the translocation of masses. Dokl. Akad. Nauk. USSR (N.S.), 37, 199–201 (1942)

  37. Khoury, R.N.: Closest matrices in the space of generalized doubly stochastic matrices. J. Math. Anal. Appl. 222(2), 562–568 (1998)

    MathSciNet  MATH  Google Scholar 

  38. Korman, J., McCann, R.J.: Optimal transportation with capacity constraints. Trans. Am. Math. Soc. 367(3), 1501–1521 (2014)

    MathSciNet  MATH  Google Scholar 

  39. Lee, Y.-J., Wu, J., Xu, J., Zikatanov, L.: Robust subspace correction methods for nearly singular systems. Math. Models Meth. Appl. Sci. 17(11), 1937–1963 (2007)

    MathSciNet  MATH  Google Scholar 

  40. Lee, Y.T., Sidford, A.: Path finding methods for linear programming: Solving linear programs in \(\tilde{O}(\sqrt{rank})\) iterations and faster algorithms for maximum flow. In 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pages 424–433. IEEE, (2014)

  41. Li, B., Xie, X.: A two-level algorithm for the weak Galerkin discretization of diffusion problems. J. Comput. Appl. Math. 287, 179–195 (2015)

    MathSciNet  MATH  Google Scholar 

  42. Li, B., Xie, X.: BPX preconditioner for nonstandard finite element methods for diffusion problems. SIAM J. Numer. Anal. 54(2), 1147–1168 (2016)

    MathSciNet  MATH  Google Scholar 

  43. Li, B., Xie, X., Zhang, S.: Analysis of a two-level algorithm for HDG methods for diffusion problems. Commun. Comput. Phys. 19(5), 1435–1460 (2016)

    MathSciNet  MATH  Google Scholar 

  44. Li, X., Sun, D., Toh, K.-C.: An asymptotically superlinearly convergent semismooth Newton augmented Lagrangian method for Linear Programming. SIAM J. Optim. 30(3), 2410–2440 (2020)

    MathSciNet  MATH  Google Scholar 

  45. Li, X., Sun, D., Toh, K.-C.: On the efficient computation of a generalized Jacobian of the projector over the Birkhoff polytope. Math. Program. 179(1–2), 419–446 (2020)

    MathSciNet  MATH  Google Scholar 

  46. Liao, Q., Chen, J., Wang, Z., Bai, B., Jin, S., Wu, H.: Fast Sinkhorn I: An \({O}({N})\) algorithm for the Wasserstein-1 metric. arXiv:2202.10042, (2022)

  47. Lin, M., Sun, D., Toh, K.-C.: An augmented Lagrangian method with constraint generations for shape-constrained convex regression problems. Math. Program. 14, 223–270 (2022)

    MathSciNet  MATH  Google Scholar 

  48. Lin, T., Ho, N., Jordan, M.I.: On efficient optimal transport: an analysis of greedy and accelerated mirror descent algorithms. In: International Conference on Machine Learning, pages 3982–3991. PMLR, (2019)

  49. Liu, Y., Wen, Z., Yin, W.: A multiscale semi-smooth Newton method for optimal transport. J. Sci. Comput. 91(2), 1–39 (2022)

    MathSciNet  MATH  Google Scholar 

  50. Lorenz, D.A., Manns, P., Meyer, C.: Quadratically regularized optimal transport. Appl. Math. Optim. 83(3), 1919–1949 (2021)

    MathSciNet  MATH  Google Scholar 

  51. Luo, H.: Accelerated differential inclusion for convex optimization. Optimization (2021). https://doi.org/10.1080/02331934.2021.2002327

    Article  Google Scholar 

  52. Luo, H.: Accelerated primal-dual methods for linearly constrained convex optimization problems. arXiv:2109.12604, (2021)

  53. Luo, H.: A primal-dual flow for affine constrained convex optimization. ESAIM: Control Optim. Cal. Var., 28:10.1051/cocv/2022032, (2022)

  54. Luo, H., Zhang, Z.: A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate. arXiv:2109.13467, (2023)

  55. Maas, J., Rumpf, M., Schönlieb, C., Simon, S.: A generalized model for optimal transport of images including dissipation and density modulation. ESAIM: Math. Model. Numer. Anal., 49(6), 1745–1769 (2015)

  56. Mérigot, Q., Thibert, B.: Optimal Transport: Discretization and Algorithms. In: Handbook of Numerical Analysis, volume 22, pages 133–212. Elsevier, (2021)

  57. Padiy, A., Axelsson, O., Polman, B.: Generalized augmented matrix preconditioning approach and its application to iterative solution of ill-conditioned algebraic systems. SIAM J. Matrix Anal. Appl. 22(3), 793–818 (2001)

    MathSciNet  MATH  Google Scholar 

  58. Panaretos, V.M., Zemel, Y.: Amplitude and phase variation of point processes. Ann. Stat. 44(2), 771–812 (2016)

    MathSciNet  MATH  Google Scholar 

  59. Pele, O., Werman, M.: Fast and robust earth mover’s distances. In In 2009 IEEE 12th International Conference on Computer Vision, pages 460–467, (2009)

  60. Peyré, G., Cuturi, M.: Computational optimal transport. Found. Trends Mach. Learn. 11(5–6), 1–257 (2019)

    MATH  Google Scholar 

  61. Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18(1), 227–244 (1993)

    MathSciNet  MATH  Google Scholar 

  62. Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58(1–3), 353–367 (1993)

    MathSciNet  MATH  Google Scholar 

  63. Rubner, Y., Tomasi, C., Guibas, L.J.: The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vis. 40(2), 99–121 (2000)

    MATH  Google Scholar 

  64. Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. Society for Industrial and Applied Mathematics, USA (2003)

    MATH  Google Scholar 

  65. Sinkhorn, R.: Diagonale quivalence to matrices with prescribed row and columnsums. Am. Math. Monthly 74(4), 402–405 (1967)

    MATH  Google Scholar 

  66. Székely, G., Rizzo, M.: Testing for equal distributions in high dimension. In: Inter-Stat (London), pages 1–16, (2004)

  67. Trottenberg, U., Oosterlee, C.W., Schüller, A.: Multigrid. Academic Press, San Diego (2001)

    MATH  Google Scholar 

  68. Villani, C.: Topics in Optimal Transportation. American Mathematical Society, (2003)

  69. Villani, C.: Optimal Transport: Old and New. Number 338 in Grundlehren der mathematischen Wissenschaften. Springer, Berlin, (2009)

  70. Wu, Y., Chen, L., Xie, X., Xu, J.: Convergence analysis of V-Cycle multigrid methods for anisotropic elliptic equations. IMA J. Numer. Anal. 32(4), 1329–1347 (2012)

    MathSciNet  MATH  Google Scholar 

  71. Xu, J.: A new class of iterative methods for nonself-adjoint or indefinite problems. SIAM J. Numer. Anal. 29(2), 303–319 (1992)

    MathSciNet  Google Scholar 

  72. Xu, J.: Two-grid discretization techniques for linear and nonlinear PDEs. SIAM J. Numer. Anal. 33(5), 1759–1777 (1996)

    MathSciNet  MATH  Google Scholar 

  73. Xu, J.: Multilevel Iterative Methods. Penn State University, Lecture Notes (2017)

  74. Xu, J., Zikatanov, L.: The method of alternating projections and the method of subspace corrections in Hilbert space. J. Am. Math. Soc. 15(3), 573–597 (2002)

    MathSciNet  MATH  Google Scholar 

  75. Xu, J., Zikatanov, L.: Algebraic multigrid methods. Acta Numer. 26, 591–721 (2017)

    MathSciNet  MATH  Google Scholar 

Download references

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Peking University, Beijing, 100871, China

    Jun Hu, Hao Luo & Zihang Zhang

  2. Chongqing Research Institute of Big Data, Peking University, Chongqing, 401121, China

    Jun Hu

Authors
  1. Jun Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zihang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Luo.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by National Natural Science Foundation of China (No. 12288101).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, J., Luo, H. & Zhang, Z. A Fast Solver for Generalized Optimal Transport Problems Based on Dynamical System and Algebraic Multigrid. J Sci Comput 97, 6 (2023). https://doi.org/10.1007/s10915-023-02272-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-023-02272-9

Keywords

Mathematics Subject Classification

Search

Navigation