Bo Xiao
ABSTRACT
A good distance metric for the input data is crucial in many pattern recognition and machine learning applications. Past studies have demonstrated that learning a metric from labeled samples can significantly improve the performance of classification and clustering algorithms. In this paper, we investigate the problem of learning a distance metric that measures the semantic similarity of input data for regression problems. The particular application we consider is human age estimation. Our guiding principle for learning the distance metric is to preserve the local neighborhoods based on a specially designed distance as well as to maximize the distances between data that are not in the same neighborhood in the semantic space.Without any assumption about the structure and the distribution of the input data, we show that this can be done by using semidefinite programming. Furthermore, the low-level feature space can be mapped to the high-level semantic space by a linear transformation with very low computational cost. Experimental results on the publicly available FG-NET database show that 1) the learned metric correctly discovers the semantic structure of the data even when the amount of training data is small and 2) significant improvement over the traditional Euclidean metric for regression can be obtained using the learned metric. Most importantly, simple regression methods such as k nearest neighbors (kNN), combined with our learned metric, become quite competitive (and sometimes even superior) in terms of accuracy when compared with the state-of-the-art human age estimation approaches.
Supplemental Material
Available for Download
This supplement contains corrections to Tables 1 and 2 and to Figure 8 in the article "Learning distance metric for regression by semidefinite programming with application to human age estimation."
- The FG-NET Aging Database {Online}, http://www.fgnet.rsunit.com/.Google Scholar
- V. N. Balasubramanian, J. Ye, and S. Panchanathan. Biased manifold embedding: A framework for person-independent head pose estimation. In IEEE Conf. CVPR, pages 1--7, 2007.Google ScholarCross Ref
- M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionally reduction and data representation. Neural Computation, 15(6):1373--1396, 2003. Google ScholarDigital Library
- M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: a geometric framework for learing from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399--2434, 2006. Google ScholarDigital Library
- D. B. Borchers. Csdp, a c library for semidefinite programming. Optimization Methods and Software, 11(1):613--623, 1999.Google ScholarCross Ref
- D. Cai, X. He, J. Han, and H. J. Zhang. Orthogonal laplacianfaces for face recognition. IEEE Trans. Image Processing, 15:3608--3614, Nov. 2006. Google ScholarDigital Library
- S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity metric discriminatively, with application to face verification. IEEE Conf. on Computer Vision and Pattern Recognition, 1:539--546, 2005. Google ScholarDigital Library
- T. Cootes, G. Edwards, and C. Taylar. Active appearance models. IEEE Trans. Pattern Analysis and Machine Intelligence, 23(6):681--685, 2001. Google ScholarDigital Library
- T. Cox and M. Cox. Multidimensional Scaling. Chapman&Hall, Lodon, 1994.Google Scholar
- D. L. Donoho and C. E. Grimes. When does geodesic distance recover the true hidden parametrization of families of articulated images? Proceedings European Symposium on Artificial Neural Networks, 2002.Google Scholar
- A. L. C. Draganova and C. Christodoulou. Comparing different classifiers for automatic age estimation. IEEE Trans. Systems, Man, and Cybernetics, 34(1):621--628, 2004. Google ScholarDigital Library
- X. Geng, K. Smith-Miles, and Z.-Z. Zhou. Facial age estimation by nonlinear aging pattern subspace. Proc. ACM Conf. Multimedia, 2008. Google ScholarDigital Library
- X. Geng, Z.-H. Zhou, Y. Zhang, G. Li, and H. Dai. Learning from facial aging patterns for automatic age estimation. Proc. ACM Conf. Multimedia, pages 307--316, 2006. Google ScholarDigital Library
- J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov. Neighbourhood components analysis. NIPS, 2005.Google ScholarDigital Library
- G. H. Golub and C. F. V. Loan. Matrix Computations. Johns Hopkins Univ. Press, 1996.Google ScholarDigital Library
- G. Guo, Y. Fu, C. Dyer, and T. S. Huang. Image based human age estimation by manifold learning and locally adjusted robust regression. IEEE Trans. on Image Processing, 17:1178--1188, July 2008. Google ScholarDigital Library
- J. He, M. Li, H. J. Zhang, H. Tong, and C. Zhang. Manifold ranking based image retrieval. Proc. ACM Multimedia, 2004. Google ScholarDigital Library
- X. He, W. Y. Ma, and H. J. Zhang. Learning an image manifold for retrieval. Proc. ACM Multimedia, 2004. Google ScholarDigital Library
- X. He, S. Yan, Y. Hu, and H. J. Zhang. Learning a locality perserving subspace for visual recognition. IEEE Conf. on Computer Vision, 1:385--392, 2003. Google ScholarDigital Library
- I. T. Jolliffe. Principal Component Analysis. Springer Verlag, New York, 1986.Google ScholarCross Ref
- W. Matusik, H. Pfister, M. Brand, and L. McMillan. A data-driven reflectance model. Proc. of SIGGRAPH, 2003. Google ScholarDigital Library
- J. Nilsson, F. Sha, and M. I. Jordan. Regression on manifolds using kernel dimension reduction. IEEE Conf. ICML, pages 265--272, 2007. Google ScholarDigital Library
- S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323--2326, 2000.Google ScholarCross Ref
- Y. Rui and T. S. Huang. Optimizing learning in image retrieval. IEEE Conf. on CVPR, 1:236--243, 2000.Google ScholarCross Ref
- S. Shalev-Shwartz, Y. Singer, and A. Y. Ng. Online and batch learning of pseudo-metrics. Proc. of International Conference on Machine Learning, 2004. Google ScholarDigital Library
- N. Shental, T. Hertz, D. Weinshall, and M. Pavel. Adjustment learning and relevant component analysis. Proc. European Conference on Computer Vision, 4:776--792, 2002. Google ScholarDigital Library
- J. F. Sturm. Using sedumi 1.02. a matlab toolbox for optimization over symmetric cones. Optimization Methods and Software, 11-12:625--653, 1999.Google ScholarCross Ref
- M. Sugiyama, H. Hachiya, C. Towell, and S. Vijayakumar. Geodesic gaussian kernels for value function approximation. Autonomous Robots, 25:287--304, 2008. Google ScholarDigital Library
- J. B. Tenebaum, V. de. Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionally reduction. Science, 290(5500):2319--2323, 2000.Google ScholarCross Ref
- L. Wang, Y. Zhang, and J. Feng. On the euclidean distance of images. IEEE Trans. on Pattern Anal. Mach. Intell., 27:1334--1339, Aug. 2005. Google ScholarDigital Library
- K. Weinberger, J. Blitzer, and L. Saul. Distance metric learning for large margin nearest neighbor classification. in Proc. NIPS, pages 1475--1482, 2006.Google Scholar
- K. Q. Weinberger and L. K. Saul. Unsupervised learning of image manifolds by semidefinite programming. In IEEE Conf. CVPR, volume 2, pages 988--995, 2004. Google ScholarDigital Library
- E. Xing, A. Ng, M. I. Jordan, and S. Russell. Distance metric learning with application to clustering with side-information. in Proc. NIPS, 2002.Google ScholarDigital Library
- S. Yan, H. W. ang T. S. Huang, and X. Tang. Ranking with uncertain labels. IEEE Conf. Mulitimedia and Expo, pages 96--99, 2007.Google ScholarCross Ref
- S. Yan, H. Wang, X. Tang, and T. S. Huang. Learning auto-structured regressor from uncertain nonnegative labels. IEEE Conf. ICCV, pages 1--8, 2007.Google ScholarCross Ref
- Q. Yong and J. Yang. Modified kernel functions by geodesic distance. EURASIP Journal on Applied Signal Processing, 16:2515--2521, 2004. Google ScholarDigital Library
- Z. Zhang and H. Zha. Principal manifolds and nonlinear dimensionality reduction by local tangent space alignment. SIAM Journal of Scientific Computing, 26(10):313--338, 2004. Google ScholarDigital Library
Index Terms
- Learning distance metric for regression by semidefinite programming with application to human age estimation
Recommendations
Metric Learning for Regression Problems and Human Age Estimation
PCM '09: Proceedings of the 10th Pacific Rim Conference on Multimedia: Advances in Multimedia Information ProcessingThe estimation of human age from face images has great potential in real-world applications. However, how to discover the intrinsic aging trend is still a challenging problem. In this work, we proposed a general distance metric learning scheme for ...
Human Age Estimation by Metric Learning for Regression Problems
CAIP '09: Proceedings of the 13th International Conference on Computer Analysis of Images and PatternsThe estimation of human age from face images is an interesting problem in computer vision. We proposed a general distance metric learning scheme for regression problems, which utilizes not only data themselves, but also their corresponding labels to ...
Learning neighborhoods for metric learning
ECMLPKDD'12: Proceedings of the 2012th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part IMetric learning methods have been shown to perform well on different learning tasks. Many of them rely on target neighborhood relationships that are computed in the original feature space and remain fixed throughout learning. As a result, the learned ...
Comments