Abstract
Hashing methods aim to learn a set of hash functions which map the original features to compact binary codes with similarity preserving in the Hamming space. Hashing has proven a valuable tool for large-scale information retrieval. We propose a column generation based binary code learning framework for data-dependent hash function learning. Given a set of triplets that encode the pairwise similarity comparison information, our column generation based method learns hash functions that preserve the relative comparison relations within the large-margin learning framework. Our method iteratively learns the best hash functions during the column generation procedure. Existing hashing methods optimize over simple objectives such as the reconstruction error or graph Laplacian related loss functions, instead of the performance evaluation criteria of interest—multivariate performance measures such as the AUC and NDCG. Our column generation based method can be further generalized from the triplet loss to a general structured learning based framework that allows one to directly optimize multivariate performance measures. For optimizing general ranking measures, the resulting optimization problem can involve exponentially or infinitely many variables and constraints, which is more challenging than standard structured output learning. We use a combination of column generation and cutting-plane techniques to solve the optimization problem. To speed-up the training we further explore stage-wise training and propose to optimize a simplified NDCG loss for efficient inference. We demonstrate the generality of our method by applying it to ranking prediction and image retrieval, and show that it outperforms several state-of-the-art hashing methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
CGHash is available at https://bitbucket.org/guosheng/column-generation-hashing.
StructHash is available at https://bitbucket.org/guosheng/structhash.
References
Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.
Cakir, F., & Sclaroff, S. (2015). Adaptive hashing for fast similarity search. In Proceedings of the international conference on computer vision.
Chakrabarti, S., Khanna, R., Sawant, U., & Bhattacharyya, C. (2008). Structured learning for non-smooth ranking losses. In Proceedings of the ACM knowledge discovery and data mining.
Dean, T., Ruzon, M. A., Segal, M., Shlens, J., Vijayanarasimhan, S., & Yagnik, J. (2013). Fast, accurate detection of 100,000 object classes on a single machine. In Proceedings of the international conference on computer vision and pattern recognition.
Demiriz, A., Bennett, K. P., & Shawe-Taylor, J. (2002). Linear programming boosting via column generation. Machine Learning, 46(1–3), 225–254.
Gionis, A., Indyk, P., & Motwani, R. (1999). Similarity search in high dimensions via hashing. In Proceedings of the international conference on very large data bases.
Gong, Y., Lazebnik, S., Gordo, A., & Perronnin, F. (2013). Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2916–2929.
Heo, J., Lee, Y., He, J., Chang, S., & Yoon, S. (2012). Spherical hashing. In Proceedings of the international conference on computer vision and pattern recognition.
Järvelin, K., & Kekäläinen, J. (2000). IR evaluation methods for retrieving highly relevant documents. In Proceedings of the ACM conference on SIGIR.
Joachims, T. (2005). A support vector method for multivariate performance measures. In Proceedings of the international conference on machine learning.
Joachims, T. (2006). Training linear SVMs in linear time. In Proceedings of the ACM knowledge discovery and data mining.
Kelley, J. E., Jr. (1960). The cutting-plane method for solving convex programs. Journal of the Society for Industrial and Applied Mathematics, 8(4), 703–712.
Kulis, B., & Darrell, T. (2009). Learning to hash with binary reconstructive embeddings. In Proceedings of advances in neural information processing systems.
Kulis, B., & Grauman, K. (2012). Kernelized locality-sensitive hashing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6), 1092–1104.
Li, X., Lin, G., Shen, C., van den Hengel, A., & Dick, A. (2013). Learning hash functions using column generation. In Proceedings of international conference on machine learning.
Lim, D., & Lanckriet, G. (2014). Efficient learning of mahalanobis metrics for ranking. In Proceedings of the international conference on machine learning.
Lin, G., Shen, C., & Wu, J. (2014a). Optimizing ranking measures for compact binary code learning. In Proceedings of the European conference on computer vision.
Lin, G., Shen, C., Shi, Q., van den Hengel, A., & Suter, D. (2014b). Fast supervised hashing with decision trees for high-dimensional data. In Proceedings of the international conference on computer vision and pattern recognition , Columbus, OH, USA.
Lin, G., Shen, C., Suter, D., & van den Hengel, A. (2013). A general two-step approach to learning-based hashing. In Proceedings of the international conference on computer vision, Sydney, Australia.
Liu, W., Wang, J., Ji, R., Jiang, Y. G., & Chang, S. F. (2012). Supervised hashing with kernels. Proceedings of the international conference on computer vision and pattern recognition.
Liu, W., Wang, J., Kumar, S., & Chang, S. F. (2011). Hashing with graphs. In Proceedings of the international conference on machine learning.
McFee, B., & Lanckriet, G. R. G. (2010). Metric learning to rank. In Proceedings of the international conference on machine learning.
MOSEK ApS. (2015). The MOSEK optimization toolbox for MATLAB manual. Version 7.1 (Revision 28).
Norouzi, M., & Fleet, D. J. (2011). Minimal loss hashing for compact binary codes. In Proceedings of the international conference on machine learning (pp. 353–360).
Norouzi, M., Fleet, D. J., & Salakhutdinov, R. (2012). Hamming distance metric learning. In Proceedings of the advances in neural information processing systems.
Schultz, M., & Joachims, T. (2004). Learning a distance metric from relative comparisons. In Proceedings of advances in neural information processing systems.
Shalit, U., Weinshall, D., & Chechik, G. (2012). Online learning in the embedded manifold of low-rank matrices. Journal of Machine Learning Research, 13, 429–458.
Shen, C., Kim, J., Wang, L., & van den Hengel, A. (2012). Positive semidefinite metric learning using boosting-like algorithms. Journal of Machine Learning Research, 13, 1007–1036.
Shen, C., & Li, H. (2010). On the dual formulation of boosting algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12), 2216–2231.
Shen, C., Lin, G., & van den Hengel, A. (2014). StructBoost: Boosting methods for predicting structured output variables. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10), 2089–2103.
Shen, F., Shen, C., Liu, W., & Shen, H. T. (2015). Supervised discrete hashing. In Proceedings of the international conference on computer vision and pattern recognition.
Shen, F., Shen, C., Shi, Q., van den Hengel, A., & Tang, Z. (2013). Inductive hashing on manifolds. In Proceedings of the international conference on computer vision and pattern recognition, Oregon, USA.
Strecha, C., Bronstein, A. M., Bronstein, M. M., & Fua, P. (2012). LDAHash: Improved matching with smaller descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1), 66–78.
Torralba, A., Fergus, R., & Weiss, Y. (2008). Small codes and large image databases for recognition. In Proceedings of the international conference on computer vision and pattern recognition.
Wang, J., Kumar, S., & Chang, S.-F. (2012). Semi-supervised hashing for large-scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12), 2393–2406.
Wang, J., Shen, H. T., Song, J. & Ji, J. (2014). Hashing for similarity search: A survey. CoRR. arXiv:1408.2927.
Wang, Q., Zhang, Z., & Si, L. (2015). Ranking preserving hashing for fast similarity search. In Proceedings of the international joint conference on artificial Intelligence, 34(12), 2393–2406.
Weiss, Y., Fergus, R., & Torralba, A. (2012). Multidimensional spectral hashing. In Proceedings of the European conference on computer vision.
Weiss, Y., Torralba, A., & Fergus, R. (2008). Spectral hashing. In Proceedings of advances in neural information processing systems.
Weston, J., Bengio, S., & Usunier, N. (2010). Large scale image annotation: Learning to rank with joint word-image embeddings. Machine Learning, 81(1), 21–35.
Yue, Y., Finley, T., Radlinski, F., & Joachims, T. (2007). A support vector method for optimizing average precision. In Proceedings of the ACM conference on SIGIR.
Zhang, D., Wang, J., Cai, D., & Lu, J. (2010). Extensions to self-taught hashing: Kernelisation and supervision. In Proceedings of the ACM conference on SIGIR workshop.
Zhu, C., Byrd, R. H., Lu, P., & Nocedal, J. (1997). Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software, 23(4), 550–560.
Acknowledgements
C. Shen’s participation was supported by an ARC Future Fellowship (FT120100969). H. T. Shen’s participation was supported by National Nature Science Foundation of China (No. 61632007).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Florent Perronnin.
Rights and permissions
About this article
Cite this article
Lin, G., Liu, F., Shen, C. et al. Structured Learning of Binary Codes with Column Generation for Optimizing Ranking Measures. Int J Comput Vis 123, 287–308 (2017). https://doi.org/10.1007/s11263-016-0984-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-016-0984-4