Skip to main content
SpringerLink
Log in

Control Variates for Similarity Search

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13019))

Included in the following conference series:

  • 2347 Accesses

Abstract

We present an alternative technique for similarity estimation under locality sensitive hashing (LSH) schemes with discrete output. By utilising control variates and extra information, we are able to achieve better theoretical variance reductions compared to maximum likelihood estimation with extra information. We show that our method obtains equivalent results, but slight modifications can provide better empirical results and stability at lower dimensions. Finally, we compare the various methods’ performances on the MNIST and Gisette dataset, and show that our model achieves better accuracy and stability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Achlioptas, D.: Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66(4), 671–687 (2003)

    Article  MathSciNet  Google Scholar 

  2. Broder, A.Z.: On the resemblance and containment of documents. In: Compression and Complexity of Sequences 1997, Proceedings, pp. 21–29. IEEE (1997)

    Google Scholar 

  3. Charikar, M.S.: Similarity estimation techniques from rounding algorithms. In: Proceedings of the Thirty-Fourth Annual ACM Symposium on Theory of Computing, pp. 380–388. ACM (2002)

    Google Scholar 

  4. Glynn, P.W., Szechtman, R.: Some new perspectives on the method of control variates. In: Fang, K.T., Niederreiter, H., Hickernell, F.J. (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2000, pp. 27–49. Springer, Heidelberg (2002). https://doi.org/10.1007/978-3-642-56046-0_3

  5. Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM (JACM) 42(6), 1115–1145 (1995)

    Article  MathSciNet  Google Scholar 

  6. Guyon, I., Gunn, S., Ben-Hur, A., Dror, G.: Result analysis of the NIPS 2003 feature selection challenge. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 17, pp. 545–552. MIT Press (2005)

    Google Scholar 

  7. Guyon, I., Gunn, S.R., Ben-Hur, A., Dror, G.: Result analysis of the NIPS 2003 feature selection challenge. In: NIPS, vol. 4, pp. 545–552 (2004)

    Google Scholar 

  8. Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing, STOC 1998, pp. 604–613. ACM, New York (1998). https://doi.org/10.1145/276698.276876, http://doi.acm.org/10.1145/276698.276876

  9. Kang, K.: Using the multivariate normal to improve random projections. In: Yin, H., et al. (eds.) IDEAL 2017. LNCS, vol. 10585, pp. 397–405. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68935-7_43

    Chapter  Google Scholar 

  10. Kang, K.: Correlations between random projections and the bivariate normal. Data Min. Knowl. Disc. 35(4), 1622–1653 (2021). https://doi.org/10.1007/s10618-021-00764-6

    Article  MathSciNet  Google Scholar 

  11. Kang, K., Wong, W.P.: Improving sign random projections with additional information. In: International Conference on Machine Learning, pp. 2479–2487. PMLR (2018)

    Google Scholar 

  12. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  13. Li, P., Hastie, T.J., Church, K.W.: Improving random projections using marginal information. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 635–649. Springer, Heidelberg (2006). https://doi.org/10.1007/11776420_46

    Chapter  Google Scholar 

  14. Lichman, M.: UCI Machine Learning Repository (2013). http://archive.ics.uci.edu/ml

  15. Rubinstein, R.Y., Marcus, R.: Efficiency of multivariate control variates in Monte Carlo simulation. Oper. Res. 33(3), 661–677 (1985)

    Article  MathSciNet  Google Scholar 

  16. Slaney, M., Casey, M.: Locality-sensitive hashing for finding nearest neighbors [lecture notes]. IEEE Signal Process. Mag. 25(2), 128–131 (2008)

    Article  Google Scholar 

  17. Szechtman, R., Glynn, P.W.: Constrained Monte Carlo and the method of control variates. In: Proceeding of the 2001 Winter Simulation Conference (Cat. No. 01CH37304), vol. 1, pp. 394–400. IEEE (2001)

    Google Scholar 

Download references

Acknowledgements

This work is funded by the Singapore Ministry of Education Academic Research Fund Tier 2 Grant MOE2018-T2-2-013, as well as with the support of the Singapore University of Technology and Design’s Undergraduate Research Opportunities Programme.

The authors also thank the anonymous reviewers for their comments and suggestions for improvement, which has helped to enhance the quality of the paper.

Author information

Authors and Affiliations

  1. Singapore University of Technology and Design, Singapore, Singapore

    Jeremy Chew & Keegan Kang

Authors
  1. Jeremy Chew
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keegan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremy Chew .

Editor information

Editors and Affiliations

  1. University of Science and Technology Beijing, Beijing, China

    Huimin Ma

  2. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  3. Tsinghua University, Beijing, China

    Changshui Zhang

  4. Zhejiang University, Hangzhou, China

    Fei Wu

  5. Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  6. Hunan University, Changsha, China

    Yaonan Wang

  7. Sun Yat-Sen University, Guangzhou, Guangdong, China

    Jianhuang Lai

  8. Beijing Jiaotong University, Beijing, China

    Yao Zhao

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chew, J., Kang, K. (2021). Control Variates for Similarity Search. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13019. Springer, Cham. https://doi.org/10.1007/978-3-030-88004-0_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88004-0_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88003-3

  • Online ISBN: 978-3-030-88004-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation