Abstract
Approximations based on random Fourier features have recently emerged as an efficient and elegant methodology for designing large-scale kernel machines [4]. By expressing the kernel as a Fourier expansion, features are generated based on a finite set of random basis projections with inner products that are Monte Carlo approximations to the original kernel. However, the original Fourier features are only applicable to translation-invariant kernels and are not suitable for histograms that are always non-negative. This paper extends the concept of translation-invariance and the random Fourier feature methodology to arbitrary, locally compact Abelian groups. Based on empirical observations drawn from the exponentiated χ 2 kernel, the state-of-the-art for histogram descriptors, we propose a new group called the skewed-multiplicative group and design translation-invariant kernels on it. Experiments show that the proposed kernels outperform other kernels that can be similarly approximated. In a semantic segmentation experiment on the PASCAL VOC 2009 dataset, the approximation allows us to train large-scale learning machines more than two orders of magnitude faster than previous nonlinear SVMs.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: Rcv1: A new benchmark collection for text categorization research. JMLR 5, 361–397 (2004)
Attenberg, J., Dasgupta, A., Langford, J., Smola, A., Weinberger, K.: Feature hashing for large scale multitask learning. In: ICML (2009)
Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: A database and web-based tool for image annotation. IJCV 77(1-3), 157–173 (2008)
Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: NIPS (2007)
Shi, Q., Patterson, J., Dror, G., Langford, J., Smola, A., Strehl, A., Vishwanathan, V.: Hash kernels. In: AISTATS (2009)
Bo, L., Sminchisescu, C.: Efficient match kernels between sets of features for visual recognition. In: NIPS (2009)
Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: A library for large linear classification. Journal of Machine Learning Research, 1871–1874 (2008)
Shalev-Shwartz, S., Singer, Y., Srebro, N.: Pegasos: Primal estimated sub-gradient solver for svm. In: ICML (2007)
Vedaldi, A., Gulshan, V., Varma, M., Zisserman, A.: Multiple kernels for object detection. In: ICCV (2009)
Chapelle, O., Haffner, P., Vapnik, V.: Support vector machines for histogram-based image classification. IEEE Transactions on Neural Networks 10 (1999)
Rudin, W.: Fourier Analysis on Groups (1962)
Vedaldi, A., Zisserman, A.: Efficient additive kernels via explicit feature maps. In: CVPR (2010)
Fine, S., Scheinberg, K.: Efficient svm training using low-rank kernel representation. JMLR 2, 243–264 (2001)
Bach, F., Jordan, M.I.: Predictive low-rank decomposition for kernel methods. In: ICML (2005)
Williams, C.K.I., Seeger, M.: Using the nyström method to speed up kernel machines. In: NIPS (2001)
Drineas, P., Mahoney, M.: On the nyström method for approximating a gram matrix for improved kernel-based learning. JMLR 6, 2153–2175 (2005)
Grauman, K., Darrell, T.: The pyramid match kernel: Efficient learning with sets of features. JMLR 8, 725–760 (2007)
Maji, S., Berg, A.C., Malik, J.: Classification using intersection kernel support vector machines is efficient. In: CVPR (2008)
Hein, M., Bousquet, O.: Hilbertian metrics and positive definite kernels on probability measures. In: AISTATS (2005)
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge, VOC 2009 Results (2009), http://www.pascal-network.org/challenges/VOC/voc2009/workshop/index.html
Carreira, J., Sminchisescu, C.: Constrained parametric min cuts for automatic object segmentation. In: CVPR (2010)
Li, F., Carreira, J., Sminchisescu, C.: Object recognition as ranking holistic figure-ground hypotheses. In: CVPR (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, F., Ionescu, C., Sminchisescu, C. (2010). Random Fourier Approximations for Skewed Multiplicative Histogram Kernels. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds) Pattern Recognition. DAGM 2010. Lecture Notes in Computer Science, vol 6376. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15986-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-15986-2_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15985-5
Online ISBN: 978-3-642-15986-2
eBook Packages: Computer ScienceComputer Science (R0)