skip to main content
10.1145/3633637.3633709acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

Robust Graph Regularized Auto-Encoders by Cross Entropy Penalty

Published: 28 February 2024 Publication History

Abstract

Graph regularized Auto-Encoder (GAE) is an AE variant that incorporates with manifold learning. Using graph Laplacian as regularization towards encodings, GAE could preserve locality of original data in low dimensions, achieving better clustering and visualization results than original AE and its other variants. Nevertheless, graph Laplacian in GAE takes 2-norm loss function to penalize locality of encodings, likely leading to instability. Pinpointing at this, in this paper we propose a robust graph regularized auto-encoder (RGAE). Instead of graph Laplacian, RGAE adopts cross entropy for penalization, pursuing consistency between locality of original data and that of its low-dimensional encodings. In particular, RGAE is less parameterized, since it takes no local weights into account. Experiments on benchmark datasets reveal that RGAE is on the whole more performant than GAE at clustering along with varied encoding dimensions, and achieves better visualization results.

References

[1]
[1] Yan S, D Xu, Zhang B, et al. Graph Embedding and Extensions: A General Framework for Dimensionality Reduction[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(1) (2007)40-51.
[2]
[2] IT Jolliffe. Principal Component Analysis[M]. Springer-Verlag, 2005.
[3]
[3] Belkin M., Niyogi P. Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering[C], Advances in Neural Information Processing System, 14 (2001)585-591.
[4]
[4] Roweis S, Saul L. Nonlinear Dimensionality Reduction by Locally Linear Embedding[J]. Science, 290(5500) (2000)2323-2326.
[5]
[5] Hinton G.E., Salakhutdinov R.R.Reducing the dimensionality of data with neural networks[J]. Science, 313(5786) (2006)504-507.
[6]
[6] Yu Q, Kavitha M, Kurita T.Autoencoder framework based on orthogonal projection constraints improves anomalies detection[J]. Neurocomputing, 450(5) (2021)372-388.
[7]
[7] Bunrit S, Kerdprasop N, Kerdprasop K.Improving the Representation of CNN Based Features by Autoencoder for a Task of Construction Material Image Classification[J].Engineering and Technology Publishing, 11(4) (2020):192-199.
[8]
[8] Ahmed I, Galoppo T, Hu X,et al.Graph Regularized Autoencoder and its Application in Unsupervised Anomaly Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8) (2022) 4110-4124.
[9]
[9] Ranzato M, Poultney C, Chopra S,et al. Efficient learning of sparse representations with an energy-based model[C]. Advances in Neural Information Processing Systems(NIPS), 2006.
[10]
[10] Rifai S, Vincent P, Muller X,et al.Contractive Auto-Encoders: Explicit Invariance During Feature Extraction[C]. In: Proceedings of the 28th international conference on machine learning(ICML11) (2011)833-840.
[11]
[11] Liao Y, Wang Y, Liu Y.Graph Regularized Auto-Encoders for Image Representation[J]. IEEE Transactions on Image Processing, 26(6) (2017)2839-2852.
[12]
[12] Van Der Maaten and G. Hinton. Visualizing Data using t-SNE[J]. Journal of Machine Learning Research, 9(2605) (2008)2579-2605.
[13]
[13] Van Der Maaten. Accelerating t-SNE using Tree-Based Algorithms[J]. Journal of Machine Learning Research, 15(1) (2014)3221-3245.
[14]
[14] Dornaika F, Traboulsi Y E. Joint sparse graph and flexible embedding for graph-based semi-supervised learning[J]. Neural Networks, 114 (2019) 91-95.
[15]
[15] Dornaika F., Traboulsi, Y. E. and Assoum, A. Inductive and flexible feature extraction for semi-supervised pattern categorization.[J]. Pattern Recognition, 60 (2016) 275-285.
[16]
[16] Kingma D, Ba J.Adam: A Method for Stochastic Optimization[J]. Computer Science, 2014.
[17]
[17] Sameer A. Nene, Shree K. Nayar, and Hiroshi Murase. Columbia Object Image Library (coil-20). Technical report, 1996. https://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php
[18]
[18] Lazebnik S., Schmid C. and Ponce J. Beyond bags of features:Spatial pyramid matching for recognizing natural scene categories[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2006) 2169-2178.
[19]
[19] Simonyan K, Zisserman A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. Computer Science, 2014.
[20]
[20] http://yann.lecun.com/exdb/mnist/.
[21]
[21] Wei D, Charikar M, Kai L. Efficient k-nearest neighbor graph construction for generic similarity measures[C] International Conference on World Wide Web. ACM, (2011) 577-586.
[22]
[22] Strehl, A., Ghosh J. Cluster Ensembles – A Knowledge Reuse Framework for Combining Multiple Partitions.[J]. Journal of Machine Learning Research, 3(3) (2002) 583-617.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern Recognition
October 2023
589 pages
ISBN:9798400707988
DOI:10.1145/3633637
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 February 2024

Permissions

Request permissions for this article.

Check for updates

Author Tag

  1. manifold learning · auto-encoder · cross entropy · clustering · visualization · varied encoding dimensions.

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • International Science and Technology Cooperation Project of Jiangsu Province, Major Program of University Natural Science Research of Jiangsu Province, Incubation Foundation of Jinling Institute of Technology

Conference

ICCPR 2023

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 12
    Total Downloads
  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)4
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media