Skip to main content

Advertisement

SpringerLink
Log in

Self-organizing maps by difference of convex functions optimization

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

We offer an efficient approach based on difference of convex functions (DC) optimization for self-organizing maps (SOM). We consider SOM as an optimization problem with a nonsmooth, nonconvex energy function and investigated DC programming and DC algorithm (DCA), an innovative approach in nonconvex optimization framework to effectively solve this problem. Furthermore an appropriate training version of this algorithm is proposed. The numerical results on many real-world datasets show the efficiency of the proposed DCA based algorithms on both quality of solutions and topographic maps.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Amerijckx C, Legaty JD, Verleysen M (2003) Image compression using self-organizing maps. Syst Anal Modell Simul 43(11):1529–1543

    MATH  Google Scholar 

  • Argyrou A (2009) Clustering hierarchical data using self-organizing map: a graph-theoretical approach. Advances in self-organizing maps. Lecture Notes in Computer Science, vol 5629. Springer, Heidelberg, pp 19–27

  • Astudillo CA, Oommen BJ (2014) Topology-oriented self-organizing maps: a survey. Pattern Anal Appl 17:223–248

    Article  MathSciNet  Google Scholar 

  • Barreto GA, Araúo AFR, Ritter HJ (2003) Self-organizing feature maps for modeling and control of robotic manipulators. J Intell Robot Syst 36(4):407–450

    Article  MATH  Google Scholar 

  • Bradley BS, Mangasarian OL (1998) Feature selection via concave minimization and support vector machines. In: Machine learning proceedings of the fifteenth international conferences (ICML’98), San Francisco, California, MorganKaufmann, pp 82–90

  • Burger M, Graepel T, Obermayer K (1997) Phase transitions in stochastic self-organizing maps. Phys Rev E 56:3876–3890

    Article  Google Scholar 

  • ChandraShekar BH, Shoba G (2009) Classification of documents using Kohonens self-organizing map. Int J Comput Theory Eng 1(5):610–613

    Article  Google Scholar 

  • Chang L, Jun-min L, Chong-xiu Y (2013) Skin detection using a modified self-organizing mixture network. In: Automatic face and gesture recognition (FG), 2013 10th IEEE international conference and workshops, vol 1, no 6, pp 22–26

  • Collobert R, Sinz F, Weston J, Bottou L (2006) Trading convexity for scalability. IN: International conference on machine learning ICML

  • de Carvalho FDA, Bertrand P, De Melo FM (2012) Batch self-organizing maps based on city-block distances for interval variables. Hal-00706519, version 1

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39:1–38

    MATH  MathSciNet  Google Scholar 

  • Dickerson KB, Ventura D (2009) Music recommendation and query-by-content using self-organizing maps. In: Proceedings of the international joint conference on neural networks, pp 705–710

  • Doan NQ, Azzag H, Lebbah M (2012) Self-organizing map and tree topology for graph summarization. Artificial neural networks and machine learning ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Heidelberg, pp 363–370

  • Dozono H, Tokushima H, Hara S, Noguchi Y (2005) An algorithm of SOM using simulated annealing in the batch update phase for sequence analysis. In: International workshop on self-organizing maps (WSOM), pp 171–178

  • Fiannaca A, Fatta GD, Gaglio S, Rizzo R, Urso AM (2007) Improved SOM learning using simulated annealing. Lecture Notes in Computer Science,vol 4668. Springer, Heidelberg, pp 279–288

  • Fort JC, Letremy P, Cottrell M (2002) Advantages and drawbacks of the Batch Kohonen algorithm. IN: The European symposium on artificial neural networks conference - ESANN, pp 223–230

  • Graef G, Schaefer C (2002) Application of ART2 networks and self-organizing maps to collaborative filtering. Hypermedia: openness, structural awareness, and adaptivity. Lecture Notes in Computer Science, vol 2266. Springer, Heidelberg, pp 296–309

  • Graepel T, Burger M, Obermayer K (1998) Self-organizing maps: generalizations and new optimization techniques. Neurocomputing 21(13):173–190

    Article  MATH  Google Scholar 

  • Günter S, Bunke H (2002) Self-organizing map for clustering in the graph domain. Pattern Recognit Lett 23(4):405–417

    Article  MATH  Google Scholar 

  • Guo X, Wang H, Glass DH (2013) Bayesian self-organizing map for data classification and clustering. Int J Wavelets Multiresolut Inf Process 11(5):91–102

  • Hagenbuchner M, Sperduti A, Tsoi AC (2009) Graph self-organizing maps for cyclic and unbounded graphs. Neurocomputing 72(79):1419–1430

    Article  Google Scholar 

  • Hagenauer J, Helbich M (2013) Hierarchical self-organizing maps for clustering spatiotemporal data. Int J Geograph Inform Sci 27(10):2026–2042

    Article  Google Scholar 

  • Heskes T (1999) Energy functions for self organizingmaps. In Oya S, Kaski E (eds) KohonenMaps. Elsevier, Amsterdam pp 303–316

  • Heskes T (2001) Self-organization maps, vector quantization, and mixture modeling. IEEE Trans Neural Netw 12(6):1299–1305

    Article  Google Scholar 

  • Hiriart Urruty JB, Lemarechal C (1993) Convex analysis and minimization algorithms. Springer Verlag, Berlin Heidelberg

    Google Scholar 

  • Ismail S, Shabri A, Samsudin R (2012) A hybrid model of self organizing maps and least square support vector machine for river flow forecasting. Hydrol Earth Syst Sci 16:4417–4433

    Article  Google Scholar 

  • Kaski S, Lagus K (1996) Comparing self-organizing maps. Lecture Notes in Computer Science, vol 1112. Springer, Heidelberg, pp 809–814

  • Khalilia M, Popescu M (2014) Topology preservation in fuzzy self-organizing maps. Advance trends in soft computing Studies in Fuzziness and Soft Computing, vol 312. Springer, Heidelberg, pp 105–114

  • Kiviluoto K (1996) Topology preservation in self-organizing maps. In: Proceedings of ICNN 96, IEEE international conference on neural networks, vol 1, pp 294–299

  • Kohonen T (1982) Analysis of a simple self-organizing process. Biol Cybern 44:135–140

    Article  MATH  MathSciNet  Google Scholar 

  • Kohonen T (1997) Self-organization maps. Springer, Heidelberg

    Book  Google Scholar 

  • Krause N, Singer Y (2004) Leveraging the margin more carefully. In: International conference on machine learning ICML

  • Le Thi HA (2005) DC programming and DCA. http://lita.sciences.univ-metz.fr/~lethi

  • Le Thi HA, Pham Dinh T (1997) Solving a class of linearly constrained indefinite quadratic problems by D.C. algorithms. J Glob Optim 11:253–285

    Article  MATH  Google Scholar 

  • Le Thi HA, Pham Dinh T (2005) DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann Oper Res 133:23–46

    Article  MATH  MathSciNet  Google Scholar 

  • Le Thi HA, Le Hoai M, Pham Dinh T (2007) Fuzzy clustering based on nonconvex optimization approaches using difference of convex (DC) functions algorithms. J Adv Data Anal Classif 2:1–20

    Google Scholar 

  • Le Thi HA, Le Hoai M, Nguyen VV, Pham Dinh T (2008) A DC programming approach for feature selection in support vector machines learning. J Adv Data Anal Classif 2(3):259–278

    Article  MATH  Google Scholar 

  • Le Thi HA, Le Hoai M, Pham Dinh T, Huynh VN (2012) Binary classification via spherical separator by DC programming and DCA. J Glob Optim 1–15. doi:10.1007/s10898-012-9859-6

  • Le Thi HA, Le Hoai M, Huynh VN (2013) Block clustering based on DC programming and DCA. NECO Neural Comput 25(10):2776–2807

    Article  Google Scholar 

  • Le Thi HA, Pham Dinh T, Nguyen CN, Le Hoai M DC programming and DCA for diversity data mining. Optimization (to appear)

  • Le Thi HA, Le Hoai M, Pham Dinh T (2014) New and efficient DCA based algorithms for minimum sum-of-squares clustering. Patter Recognit 47:388–401

    Article  Google Scholar 

  • Lee M, Choi P, Woo Y (2002) A hybrid recommender system combining collaborative filtering with neural network. Adaptive hypermedia and adaptive web-based systems. Lecture Notes in Computer Science, vol 2347. Springer-Verlag, Berlin Heidelberg, pp 531–534

  • Lefebvre G, Zheng H, Laurent C (2006) Objectionable image detection by ASSOM competition. Image and video retrieval. Lecture Notes in Computer Science, vol 4071. Springer, Heidelberg, pp 201–210

  • Liu Y, Shen X, Doss H (2005) Multicategory \(\Psi \)-learning and support vector machine, computational tools. J Comput Graph Stat 14:219–236

    Article  MathSciNet  Google Scholar 

  • Liu Y, Shen X (2006) Multicategory \(\Psi \)-learning. J Am Stat Assoc 101(474):500–509

    Article  MathSciNet  Google Scholar 

  • Madalina O, Nathalie VV, Christine CA (2013) Multiple Kernel self-organizing maps. In: ESANN 2013 proceedings, European symposium on artificial neural networks, computational intelligence and machine learning. Bruges, Belgium, pp 83–88

  • Van Hulle Marc M (2012) Self-organizing maps. Handbook of natural computing. Springer-Verlag, Berlin, Heidelberg, pp 585–622

  • Marina R (2012) Graph mining based SOM: a tool to analyze economic stability. In: Johnsson M (ed) Applications of self-organizing maps. InTech Publisher, Vienna, pp 1–25

    Google Scholar 

  • Matharage S, Alahakoon D, Rajapakse J, Huang P (2011) Fast growing self organizing map for text clustering. Neural information processing. Lecture Notes in Computer Science, vol 7063. Springer, Heidelberg, pp 406–415

  • Matsushita H, Nishio Y (2010) Batch-learning self-organizing map with weighted connections avoiding false-neighbor effects. In: The 2010 international joint conference on neural networks (IJCNN), pp 1–6

  • Manning CD, Raghavan P, Schtze H (2008) Introduction to information retrieval. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Neme A, Miramontes P (2014) Self-organizing map formation with a selectively refractory neighborhood. Neural Process Lett 39(1):1–24

    Article  Google Scholar 

  • Ogihara M, Matsumoto H, Marumo T, Kuroda C (2009) Clustering of pressure fluctuation data using self-organizing map. Engineering applications of neural networks. Communications in Computer and Information Science, vol 43. Springer, Heidelberg, pp 45–54

  • Olteanu M, Villa-Vialaneix N, Cierco-Ayrolles C (2013) Multiple Kernel self-organizing maps. Hal-00817920, version 1

  • O’Connell C, Kutics A, Nakagawa A (2013) Layered self-organizing map for image classification in unrestricted domains. Image analysis and processing—ICIAP 2013. Lecture Notes in Computer Science, vol 8156. Springer, Heidelberg, pp 310–319

  • Paul S, Gupta M (2013) Image segmentation by self organizing map with mahalanobis distance. Int J Emerg Technol Adv Eng 3(2):288–291

    Google Scholar 

  • Pham Dinh T, Le Thi HA (1997) Convex analysis approach to D.C. programming: theory, algorithms and applications (dedicated to Professor Hoang Tuy on the occasion of his 70th birthday). Acta Math Vietnam 22:289–355

    MATH  MathSciNet  Google Scholar 

  • Pham Dinh T, Le Thi HA (1998) DC optimization algorithms for solving the trust region sub-problem. SIAM J Optim 8:476–505

    Article  MATH  MathSciNet  Google Scholar 

  • Pratiwi D (2012) The use of self organizing map method and feature selection in image database classification system. Int J Comput Sci 9(3):377–381

    Google Scholar 

  • Roh TH, Oh KJ, Han I (2003) The collaborative filtering recommendation based on SOM cluster-indexing CBR. Expert Syst Appl 25(3):413–423, ISSN 0957–4174, doi:10.1016/S0957-4174(03)00067-8

  • Raskutti B, Leckie C (1999) An evaluation of criteria for measuring the quality of clusters. In: Proceedings of the sixteenth international joint conference on artificial intelligence IJCAI ’99, pp 905–910

  • Ruan X, Gao Y, Song H, Chen J (2011) A new dynamic self-organizing method for mobile robot environment mapping. J Intell Learn Syst Appl 3:249–256

    Google Scholar 

  • Saarikoski J, Laurikkala J, Järvelin K, Juhola M (2011) Self-organising maps in document classification: a comparison with six machine learning methods. Adaptive and natural computing algorithms. Lecture Notes in Computer Science, vol 6593. Springer, Heidelberg, pp 260–269

  • Sarlin P, Eklund T (2011) Fuzzy clustering of the self-organizing map: some applications on financial time series. In: Laaksonen J, Honkela T (eds) Advances in self-organizing maps. Lecture Notes in Computer Science, vol 6731. Springer, Berlin Heidelberg, pp 40–50

  • Shen X, Tseng GC, Zhang X, Wong WH (2003) \(\psi \)-Learning. J Am Stat Assoc 98:724–734

    Article  MATH  MathSciNet  Google Scholar 

  • Smith T, Alahakoon D (2009) Growing self-organizing map for online continuous clustering. Foundations of Computational Intelligence, vol 4. Stud Comput Intell 204:49–83

  • Szymanski J, Duch W (2012) Self organizing maps for visualization of categories. Neural information processing, Lecture Notes in Computer Science, vol 7663. Springer, Heidelberg, pp 160–167

  • Tsoi AC, Hagenbuchner M, Sperduti A (2006) Self-organising map techniques for graph data applications to clustering of XML documents. Advanced data mining and applications. Lecture Notes in Computer Science, vol 4093. Springer, Heidelberg, pp 19–30

  • Van Laerhoven K (2001) Combining the self-organizing map and K-means clustering for on-line classification of sensor data. Artificial neural networks ICANN 2001. Lecture Notes in Computer Science, vol 2130. Springer, Heidelberg, pp 464–469

  • Vembu S, Baumann S (2004) A self-organizing map based knowledge discovery for music recommendation systems. Computer music modeling and retrieval. Lecture Notes in Computer Science, vol 3310. Springer-Verlag, Berlin Heidelberg, pp 119–129

  • Villa N, Boulet R (2007) Clustering a medieval social network by SOM using a kernel based distance measure. In: ESANN’2007 proceedings—European symposium on artificial neural networks Bruges (Belgium), d-side publi., ISBN 2-930307-07-2

  • Wang J, Shen Z, Pan W (2007) On transductive support vector machines. In: Proceeding of the international conference on machine learning ICML

  • Wehrens R (2010) Self-organising maps for image segmentation. In: Advances in data analysis, data handling and business intelligence studies in classification, Data Analysis, and Knowledge Organization, pp 373–383

  • Yin H (2008) The self-organizing maps: background, theories, extensions and applications. Stud Comput Intell (SCI) 115:715–762

    Article  Google Scholar 

  • Yuille AL, Rangarajan A (2002) The convex concave procedure (CCCP). Advances in neural information processing system, vol 14. MIT Press, Cambrige MA

Download references

Acknowledgments

We are very grateful to the anonymous referees and the associate editor for their really helpful and constructive comments that helped us to improve our paper.

Author information

Authors and Affiliations

  1. Laboratory of Theoretical and Applied Computer Science - LITA EA 3097, University of Lorraine, Ile du Saulcy, 57045 , Metz, France

    Hoai An Le Thi & Manh Cuong Nguyen

Authors
  1. Hoai An Le Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manh Cuong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hoai An Le Thi.

Additional information

Responsible editors: Toon Calders, Floriana Esposito, Eyke Hüllermeier, Rosa Meo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Le Thi, H.A., Nguyen, M.C. Self-organizing maps by difference of convex functions optimization. Data Min Knowl Disc 28, 1336–1365 (2014). https://doi.org/10.1007/s10618-014-0369-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-014-0369-7

Keywords

Search

Navigation