Abstract
Distance metric learning has been widely studied in multifarious research fields. The mainstream approaches learn a Mahalanobis metric or learn a linear transformation. Recent related works propose learning a linear combination of base vectors to approximate the metric. In this way, fewer variables need to be determined, which is efficient when facing high-dimensional data. Nevertheless, such works obtain base vectors using additional data from related domains or randomly generate base vectors. However, obtaining base vectors from related domains requires extra time and additional data, and random vectors introduce randomness into the learning process, which requires sufficient random vectors to ensure the stability of the algorithm. Moreover, the random vectors cannot capture the rich information of the training data, leading to a degradation in performance. Considering these drawbacks, we propose a novel distance metric learning approach by introducing base vectors explicitly learned from training data. Given a specific task, we can make a sparse approximation of its objective function using the top eigenvalues and corresponding eigenvectors of a predefined integral operator on the reproducing kernel Hilbert space. Because the process of generating eigenvectors simply refers to the training data of the considered task, our proposed method does not require additional data and can reflect the intrinsic information of the input features. Furthermore, the explicitly learned eigenvectors do not result in randomness, and we can extend our method to any kernel space without changing the objective function. We only need to learn the coefficients of these eigenvectors, and the only hyperparameter that we need to determine is the number of eigenvectors that we utilize. Additionally, an optimization algorithm is proposed to efficiently solve this problem. Extensive experiments conducted on several datasets demonstrate the effectiveness of our proposed method.
- Mahdieh Soleymani Baghshah and Saeed Bagheri Shouraki. 2009. Semi-supervised metric learning using pairwise constraints. In 21st International Joint Conference on Artificial Intelligence. 1217--1222. Google ScholarDigital Library
- Mahdieh Soleymani Baghshah and Saeed Bagheri Shouraki. 2009. Semi-supervised metric learning using pairwise constraints. In 21st International Joint Conference on Artificial Intelligence. Google ScholarDigital Library
- Aharon Bar-Hillel, Tomer Hertz, Noam Shental, and Daphna Weinshall. 2005. Learning a mahalanobis metric from equivalence constraints. J. Mach. Learn. Res. 6, June (2005), 937--965. Google ScholarDigital Library
- Binod Bhattarai, Gaurav Sharma, and Frédéric Jurie. 2016. CP-mtML: Coupled projection multi-task metric learning for large scale face retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4226--4235.Google ScholarCross Ref
- Binod Bhattarai, Gaurav Sharma, Frédéric Jurie, and Patrick Pérez. 2014. Some faces are more equal than others: Hierarchical organization for accurate and efficient large-scale identity-based face retrieval. In European Conference on Computer Vision. Springer, 160--172.Google Scholar
- Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning 3, 1 (2011), 1--122. Google ScholarDigital Library
- Jason V. Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S. Dhillon. 2007. Information-theoretic metric learning. In Proceedings of the 24th International Conference on Machine Learning. ACM, 209--216. Google ScholarDigital Library
- Husheng Dong, Ping Lu, Chunping Liu, Yi Ji, and Shengrong Gong. 2018. Learning multiple kernel metrics for iterative person re-identification. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 14, 3 (2018), 78:1--78:24. Google ScholarDigital Library
- R Fisher. {n.d.}. The use of multiple measures in taxonomic problems. Ann. Eugenics. v7 ({n. d.}), 179--188.Google Scholar
- Gene H. Golub and Charles F. Van Loan. 1996. Matrix computations.Johns Hopkins University Press, Baltimore, MD, 374--426.Google Scholar
- Xin Guo and Ding-Xuan Zhou. 2012. An empirical feature-based learning algorithm producing sparse approximations. Appl. Comput. Harmon. Anal. 32, 3 (2012), 389--400.Google ScholarCross Ref
- Steven CH Hoi, Wei Liu, and Shih-Fu Chang. 2010. Semi-supervised distance metric learning for collaborative image retrieval and clustering. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 6, 3 (2010), 18. Google ScholarDigital Library
- Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. 2007. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report. Technical Report 07-49, University of Massachusetts, Amherst.Google Scholar
- Rong Jin, Shijun Wang, and Yang Zhou. 2009. Regularized distance metric learning: Theory and algorithm. In Advances in Neural Information Processing Systems. 862--870. Google ScholarDigital Library
- Ian Jolliffe. 2002. Principal Component Analysis. Wiley Online Library.Google Scholar
- Ira Kemelmacher-Shlizerman, Steven M. Seitz, Daniel Miller, and Evan Brossard. 2016. The megaface benchmark: 1 million faces for recognition at scale. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4873--4882.Google ScholarCross Ref
- Ya Li, Xinmei Tian, and Dacheng Tao. 2016. Regularized large margin distance metric learning. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 1015--1022.Google ScholarCross Ref
- Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li. 2015. Person re-identification by local maximal occurrence representation and metric learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2197--2206.Google ScholarCross Ref
- Wei Liu, Xinmei Tian, Dacheng Tao, and Jianzhuang Liu. 2010. Constrained metric learning via distance gap maximization. In Association for the Advancement of Artificial Intelligence. Google ScholarDigital Library
- Yong Luo, Tongliang Liu, Dacheng Tao, and Chao Xu. 2014. Decomposition-based transfer distance metric learning for image classification. IEEE Trans. Image Process. 23, 9 (2014), 3789--3801.Google ScholarCross Ref
- Yong Luo, Yonggang Wen, Lingyu Duan, and Dacheng Tao. 2018. Transfer metric learning: Algorithms, applications and outlooks. arXiv preprint arXiv:1810.03944 (2018).Google Scholar
- Yong Luo, Yonggang Wen, Tongliang Liu, and Dacheng Tao. 2019. Transferring knowledge fragments for learning distance metric from a heterogeneous domain. IEEE Trans. Pattern Anal. Mach. Intell. 41, 4 (2019), 1013--1026. Google ScholarDigital Library
- Iacopo Masi, Stephen Rawls, Gérard Medioni, and Prem Natarajan. 2016. Pose-aware face recognition in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4838--4846.Google ScholarCross Ref
- Alexis Mignon and Frédéric Jurie. 2012. PCCA: A new approach for distance learning from sparse pairwise constraints. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2666--2672. Google ScholarDigital Library
- Romain Negrel, Alexis Lechervy, and Frédéric Jurie. 2015. Boosted metric learning for efficient identity-based face retrieval. In 26th British Machine Vision Conference.Google ScholarCross Ref
- Hong-Wei Ng and Stefan Winkler. 2014. A data-driven approach to cleaning large face datasets. In 2014 IEEE International Conference on Image Processing (ICIP). IEEE, 343--347.Google ScholarCross Ref
- Sateesh Pedagadi, James Orwell, Sergio Velastin, and Boghos Boghossian. 2013. Local fisher discriminant analysis for pedestrian re-identification. In 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 3318--3325. Google ScholarDigital Library
- Lawrence K. Saul and Sam T. Roweis. 2003. Think globally, fit locally: Unsupervised learning of low dimensional manifolds. J. Mach. Learn. Res. 4, June (2003), 119--155. Google ScholarDigital Library
- Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 815--823.Google ScholarCross Ref
- Shai Shalev-Shwartz, Yoram Singer, and Andrew Y. Ng. 2004. Online and batch learning of pseudo-metrics. In Proceedings of the 21st International Conference on Machine Learning. ACM, 94. Google ScholarDigital Library
- Min Tan, Jun Yu, Zhou Yu, Fei Gao, Yong Rui, and Dacheng Tao. 2018. User-click-data-based fine-grained image recognition via weakly supervised metric learning. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 14, 3 (2018), 70:1--70:23. Google ScholarDigital Library
- Joshua B. Tenenbaum, Vin De Silva, and John C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Sci. 290, 5500 (2000), 2319--2323.Google ScholarCross Ref
- Xinmei Tian, Dacheng Tao, and Yong Rui. 2012. Sparse transfer learning for interactive video search reranking. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 8, 3 (2012), 26:1--26:24. Google ScholarDigital Library
- Zhangcheng Wang, Ya Li, and Xinmei Tian. 2017. Semi-supervised coefficient-based distance metric learning. In International Conference on Neural Information Processing. Springer, 586--596.Google ScholarCross Ref
- Kilian Q. Weinberger and Lawrence K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, Feb. (2009), 207--244. Google ScholarDigital Library
- Xun Yang, Meng Wang, Richang Hong, Qi Tian, and Yong Rui. 2017. Enhancing person re-identification in a self-trained subspace. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 13, 3 (2017), 27:1--27:23. Google ScholarDigital Library
- Jun Yu, Yong Rui, Yuan Yan Tang, and Dacheng Tao. 2014. High-order distance based multiview stochastic learning in image classification. IEEE Trans. Cybern. 44, 12 (2014), 2431--2442.Google ScholarCross Ref
- Jun Yu, Dapeng Tao, Jonathan Li, and Jun Cheng. 2014. Semantic preserving distance metric learning and applications. Inf. Sci.281 (2014), 674--686. Google ScholarDigital Library
- Jun Yu, Meng Wang, and Dacheng Tao. 2012. Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Trans. Image Process. 21, 11 (2012), 4636--4648. Google ScholarDigital Library
- Jun Yu, Xiaokang Yang, Fei Gao, and Dacheng Tao. 2017. Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans. Cybern. 47, 12 (2017), 4014--4024.Google ScholarCross Ref
- Li Zhang, Tao Xiang, and Shaogang Gong. 2016. Learning a discriminative null space for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1239--1248.Google ScholarCross Ref
- Tianyi Zhou, Dacheng Tao, and Xindong Wu. 2010. NESVM: A fast gradient method for support vector machines. In 2010 IEEE 10th International Conference on Data Mining (ICDM). IEEE, 679--688. Google ScholarDigital Library
- Laurent Zwald, Gilles Blanchard, Pascal Massart, and Régis Vert. 2005. Kernel projection machine: A new tool for pattern recognition. In Advances in Neural Information Processing Systems. 1649--1656. Google ScholarDigital Library
Index Terms
Eigenvector-Based Distance Metric Learning for Image Classification and Retrieval
Recommendations
Class-specific mahalanobis distance metric learning for biological image classification
ICIAR'12: Proceedings of the 9th international conference on Image Analysis and Recognition - Volume Part IIDistance metric learning (DML) is an emerging field of machine learning. The basic idea behind DML is to adapt the underlying distance metric to improve the performance for the pattern analysis tasks. In this paper, we present the use of DML techniques ...
Margin distribution explanation on metric learning for nearest neighbor classification
The importance of metrics in machine learning and pattern recognition algorithms has led to an increasing interest for optimizing distance metrics in recent years. Most of the state-of-the-art methods focus on learning Mahalanobis distances and the ...
Learning image-to-class distance metric for image classification
Special section on agent communication, trust in multiagent systems, intelligent tutoring and coaching systemsImage-To-Class (I2C) distance is a novel distance used for image classification and has successfully handled datasets with large intra-class variances. However, it uses Euclidean distance for measuring the distance between local features in different ...
Comments