Skip to main content
SpringerLink
Log in

Optimizations on unknown low-dimensional structures given by high-dimensional data

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Optimization problems on unknown low-dimensional structures given by high-dimensional data belong to the field of optimizations on manifolds. Though recent developments have advanced the theory of optimizations on manifolds considerably, when the unknown low-dimensional manifold is given in the form of a set of data in a high-dimensional space, a practical optimization method has yet to be developed. Here, we propose a neural network approach to these optimization problems. A neural network is used to approximate a neighborhood of a point, which will turn the computation of a next point in the searching process into a local constraint optimization problem. Our method ensures the convergence of the process. The proposed approach applies to optimizations on manifolds embedded into Euclidean spaces. Experimental results show that this approach can effectively solve optimization problems on unknown manifolds. The proposed method provides a useful tool to the field of study low-dimensional structures given by high-dimensional data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. In general the whole manifold cannot be defined by the zero set of a given function, see (Boothby 1986).

References

  • Absil PA, Malick J (2012) Projection-Like retractions on matrix manifold. SIAM J Optim 22(1):135–158

    Article  MathSciNet  Google Scholar 

  • Absil PA, Baker CG, Gallivan KA (2007) Trust-region methods on Riemannian manifolds. Found Comput Math 7(3):303–330

    Article  MathSciNet  Google Scholar 

  • Absil PA, Mahony R, Sepulchre R (2008) Optimization algorithms on matrix manifolds. Princeton University Press, New Jersy

    Book  Google Scholar 

  • Adler RL, Dedieu J, Margulies JY, Martens M, Shub M (2002) Newtons method on Riemannian manifolds and a geometric model for the human spine. IMA J Numer Anal 22(3):359–390

    Article  MathSciNet  Google Scholar 

  • Baker CG, Absil PA, Gallivan KA (2008) An implicit trust-region method on Riemannian manifolds. IMA J Numer Anal 28(4):665–689

    Article  MathSciNet  Google Scholar 

  • Boothby WM (1986) An introduction to differentiable manifolds and riemannian geometry. Academic Press, Orlando

    MATH  Google Scholar 

  • Bothina ES (2012) An active-set trust-region algorithm for solving constrained multi-objective optimization problem. Appl Math Sci 6(33):1599–1612

    MathSciNet  MATH  Google Scholar 

  • Christopher GB (2008) Riemannian manifold trust-region methods with applications to eigenproblems, Ph. D thesis, Florida State Universtity

  • Coleman T, Branch MA, Grace A (1999) Matlab optimization toolbox users guide, 3rd edn. Math Works, Natick

    Google Scholar 

  • Gabay D (1982) Minimizing a differentiable function over a differential manifold. J Optim Theory Appl 37(2):177–219

    Article  MathSciNet  Google Scholar 

  • Gould NIM, Robinson DP (2010) A second derivative SQP method: global convergence. SIAM J Optim 20(4):2023–2048

    Article  MathSciNet  Google Scholar 

  • Huang W, Absil PA, Gallivan KA (2015) A Riemannian symmetric rank-one trust-region method. Math Program 150(2):179–216

    Article  MathSciNet  Google Scholar 

  • Ishteva M, Absil PA, Huffel SV, Lathauwer LD (2011) Tucker compression and local optima. Chemometrics Intell Lab Syst 106(1):57–64

    Article  Google Scholar 

  • Ishteva M, Absil PA, Dooren PV (2013) Jacobi algorithm for the best low multilinear rank approximation of symmetric tensors. SIAM J Matrix Anal Appl 34(2):651–672

    Article  MathSciNet  Google Scholar 

  • Jiang B, Dai YH (2015) A framework of constraint preserving update schemes for optimization on Stiefel manifold. Math Program 153(2):535–575

    Article  MathSciNet  Google Scholar 

  • Li J, Bian W, Tao D, Zhang C (2013) Learning colours from textures by sparse manifold embedding. Signal Process 93(6):1485–1495

    Article  Google Scholar 

  • Lin T, Zha H (2008) Riemannian manifold learning. IEEE Trans Pattern Anal Machine Intell 30(5):796–809

    Article  Google Scholar 

  • Ring W, Wirth B (2012) Optimization methods on Riemannian manifolds and their application to shape space. Soc Indian Autom Manuf J Optim 22(2):596–627

    MathSciNet  MATH  Google Scholar 

  • Robert HN (1992) Theory of the backpropagation neural network. In Brace, H. & Co. Neural Networks for Perception, (2), Orlando, FL, USA

  • Selvan SE et al (2012) Descent algorithms on oblique manifold for source-adaptive ICA contrast. IEEE Trans Neural Netw Learn Syst 23(12):1930–1947

    Article  Google Scholar 

  • Seung HS, Lee D (2000) The manifold ways of perception. Science 290:2268–2269

    Article  Google Scholar 

  • Smith ST (1994) Optimization techniques on Riemannian manifolds, in Hamiltonian and Gradient Flows, Algorithms and Control, A. Bloch, ed., Fields Inst. Commun 3. American Mathematical Society, Providence, RI, 113-136

  • Tyagi H, Vural E, Frissard P (2013) Tangent space estimation for smooth embeddings of Riemannian Manifolds. Inf Inference 2(1):69–114

    Article  MathSciNet  Google Scholar 

  • Udriste C (1994) Convex functions and optimization methods on Riemannian manifolds. Kluwer Academic Publishers Group, Dordrecht

    Book  Google Scholar 

  • Wang Z (2004) A generalized trust region SQP algorithm for equality constrained optimization. Rice University, Houston

    Google Scholar 

  • Yang Y (2007) Globally convergent optimization algorithms on Riemannian manifolds: uniform framework for unconstrained and constrained optimization. J Optim Theory Appl 132(2):245–265

    Article  MathSciNet  Google Scholar 

  • Yoon K et al (2013) Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nature Neurosci 16:1077–1084

    Article  Google Scholar 

Download references

Acknowledgements

Chen was supported by Beijing Municipal Commission of Education Foundation (KM201811232015) and Beijing Postdoctoral Research Foundation (ZZ201965), Wang was supported by National Natural Science Foundation of China (51777012), Qiao was supported by National Natural Science Foundation of China (61890930), and Zou was supported by a Simons Foundation Collaboration Grant for Mathematicians (416937).

Funding

This study was funded by the National Natural Science Foundation of China (51777012, 61890930), Beijing Municipal Commission of Education Foundation of China (KM201811232015), Beijing Postdoctoral Research Foundation (ZZ-201965), Simons Foundation Collaboration Grant for Mathematicians (416937), Beijing Natural Science Foundation (4202026), Key research and cultivation projects of promote the classified development of Universities (2121YJPY211).

Author information

Authors and Affiliations

  1. Beijing Information Science and Technology University, 12 Xiaoyingdonglu, Haidian district, 100192, Beijing, China

    Qili Chen & Jiuhe Wang

  2. Beijing University of Technology, 100 Pingleyuan, Chaoyang district, 100124, Beijing, China

    Qiao Junfei

  3. University of Wisconsin-Milwaukee, P.O. Box 413, Milwaukee, WI, 53201, USA

    Ming Yi Zou

Authors
  1. Qili Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiuhe Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiao Junfei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Yi Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qili Chen.

Ethics declarations

Conflict of interest

Qili Chen declares that she has no conflict of interest. Jiuhe Wang declares that he has no conflict of interest. Junfei Qiao declares that he has no conflict of interest. Yi Ming Zou declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

N/A

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Q., Wang, J., Junfei, Q. et al. Optimizations on unknown low-dimensional structures given by high-dimensional data. Soft Comput 25, 12717–12723 (2021). https://doi.org/10.1007/s00500-021-06064-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-06064-x

Keywords

Search

Navigation