Dandan Lin
Raymond Chi-Wing Wong
Skip Abstract Section
Abstract
Image retrieval keeps attracting a lot of attention from both academic and industry over past years due to its variety of useful applications. Due to the rapid growth of deep learning approaches, more better feature vectors of images could be discovered for improving image retrieval. However, most (if not all) existing deep learning approaches consider the similarity between two images locally without considering the similarity among a group of similar images globally, and thus could not return accurate results. In this article, we study the image retrieval with manifold ranking (MR) which considers both the local similarity and the global similarity, which could give more accurate results. However, existing best-known algorithms have one of the following issues: (1) they require to build a bulky index, (2) some of them do not have any theoretical bound on the output, and (3) some of them are time-consuming. Motivated by this, we propose two algorithms, namely Monte Carlo-based MR (MCMR) and MCMR+, for image retrieval, which do not have the above issues. We are the first one to propose an index-free manifold ranking image retrieval with the output theoretical bound. More importantly, our algorithms give the first best-known time complexity result of \(O(n \log n)\) where \(n\) is the total number of images in the database compared with the existing best-known result of \(O(n^2)\) in the literature of computing the exact top-\(k\) results with quality guarantee. Lastly, our experimental result shows that MCMR+ outperforms existing algorithms by up to four orders of magnitude in terms of query time.
- [1] . 2006. LU decomposition method for solving fuzzy system of linear equations. Applied Mathematics and Computation 172, 1 (2006), 633–643.Google Scholar
Cross Ref
- [2] . 2007. Tuning bandit algorithms in stochastic environments. In Proceedings of the International Conference on Algorithmic Learning Theory.Google ScholarDigital Library
- [3] . 2017. Scalable person re-identification on supervised smoothed manifold. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [4] . 2006. Cover trees for nearest neighbor. In Proceedings of the 23rd International Conference on Machine Learning.Google ScholarDigital Library
- [5] . 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40, 2 (2008), 1–60.Google ScholarDigital Library
- [6] . 2005. Content-based image retrieval: Approaches and trends of the new age. In Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval.Google ScholarDigital Library
- [7] . 2011. Efficient k-nearest neighbor graph construction for generic similarity measures. In Proceedings of the 20th International Conference on World Wide Web.Google ScholarDigital Library
- [8] . 2014. Scaling manifold ranking based image retrieval. Proceedings of the VLDB Endowment 8, 4 (2014), 341–352.Google ScholarDigital Library
- [9] . 2013. Efficient ad-hoc search for personalized PageRank. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data. ACM.Google ScholarDigital Library
- [10] . 2017. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org.Google ScholarDigital Library
- [11] . 1971. Singular value decomposition and least squares solutions. In Linear Algebra. F. L. Bauer (ed.), Springer, Berlin, 134–151.Google ScholarCross Ref
- [12] . 1993. 13 computation using the QR decomposition. (1993).Google Scholar
- [13] . 2004. Manifold-ranking based image retrieval. In Proceedings of the 12th Annual ACM International Conference on Multimedia.Google ScholarDigital Library
- [14] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [15] . 2009. Fast manifold-ranking for content-based image retrieval. In Proceedings of the 2009 ISECS International Colloquium on Computing, Communication, Control, and Management.Google ScholarCross Ref
- [16] . 2021. Massively parallel algorithms for personalized PageRank. Proceedings of the VLDB Endowment 14, 9 (2021), 1668–1680.Google ScholarDigital Library
- [17] . 2008. The MIR flickr retrieval evaluation. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. ACM.Google ScholarDigital Library
- [18] . 2002. Simrank: A measure of structural-context similarity. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 538–543.Google ScholarDigital Library
- [19] . 2018. Manifold ranking-based kernel propagation for saliency estimation. In Proceedings of the 2018 4th International Conference on Control, Automation and Robotics. IEEE.Google ScholarCross Ref
- [20] . 2017. READS: A random walk approach for efficient and accurate dynamic SimRank. Proceedings of the VLDB Endowment 10, 9 (2017), 937–948.Google ScholarDigital Library
- [21] . 2019. Predict then propagate: Graph neural networks meet personalized PageRank.In Proceedings of the International Conference on Learning Representations.Google Scholar
- [22] . 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems.Google ScholarDigital Library
- [23] . 2009. Attribute and simile classifiers for face verification. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. IEEE.Google ScholarCross Ref
- [24] . 2019. Neural message passing for multi-label classification. arXiv:1904.08049. Retrieved from https://arxiv.org/abs/1904.08049.Google Scholar
- [25] . 2012. On top-k structural similarity search. In Proceedings of the 2012 IEEE 28th International Conference on Data Engineering. IEEE, 774–785.Google ScholarDigital Library
- [26] . 2019. Walking with perception: Efficient random walk sampling via common neighbor awareness. In Proceedings of the 2019 IEEE 35th International Conference on Data Engineering.Google Scholar
- [27] . 2007. The link-prediction problem for social networks. Journal of the American Society for Information Science and Technology 58, 7 (2007), 1019–1031.Google ScholarCross Ref
- [28] . 2019. First index-free manifold ranking-based image retrieval with output bound. In Proceedings of the 2019 IEEE International Conference on Data Mining. IEEE, 1216–1221.Google ScholarCross Ref
- [29] . 2020. Index-free approach with theoretical guarantee for efficient random walk with restart query. In Proceedings of the 2020 IEEE 36th International Conference on Data Engineering. IEEE, 913–924.Google ScholarCross Ref
- [30] . 2019. Distributed algorithms for fully personalized PageRank on large graphs. In Proceedings of the World Wide Web Conference. 1084–1094.Google ScholarDigital Library
- [31] . 2017. Probesim: Scalable single-source and top-k simrank computations on dynamic graphs. Proceedings of the VLDB Endowment 11, 1 (2017), 14–26.Google ScholarDigital Library
- [32] . 2016. Personalized PageRank estimation and search: A bidirectional approach. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining. ACM.Google ScholarDigital Library
- [33] . 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2 (2004), 91–110.Google ScholarDigital Library
- [34] . 2013. Person re-identification by manifold ranking. In Proceedings of the 2013 IEEE International Conference on Image Processing.Google ScholarCross Ref
- [35] . 2010. Chap. Randomized algorithms.Google Scholar
- [36] . 1996. Columbia Object Image Library (COIL 100). Technical Report CUCS-006-96. Department of Comp. Science, Columbia University.Google Scholar
- [37] . 1999. The PageRank Citation Ranking: Bringing Order to the Web.
Technical Report . Stanford InfoLab.Google Scholar - [38] . 2015. SaliencyRank: Two-stage manifold ranking for salient object detection. Computational Visual Media 1, 4 (2015), 309–320.Google ScholarCross Ref
- [39] . 2016. Object co-segmentation via graph optimized-flexible manifold ranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [40] . 2016. CNN image retrieval learns from BoW: Unsupervised fine-tuning with hard examples. In Proceedings of the European Conference on Computer Vision. Springer.Google ScholarCross Ref
- [41] . 2018. Fine-tuning CNN image retrieval with no human annotation. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 7 (2018), 1655–1668.Google Scholar
- [42] . 2016. A novel local manifold-ranking based K-NN for modeling the regression between bioactivity and molecular descriptors. Chemometrics and Intelligent Laboratory Systems 151 (2016), 71–77.Google ScholarCross Ref
- [43] . 2015. Bear: Block elimination approach for random walk with restart on large graphs. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. ACM.Google ScholarDigital Library
- [44] . 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https://arxiv.org/abs/1409.1556.Google Scholar
- [45] . 1991. Implementing Discrete Mathematics: Combinatorics and Graph Theory with Mathematica. Addison-Wesley Longman Publishing Co., Inc.Google Scholar
- [46] . 2013. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data. ACM.Google ScholarDigital Library
- [47] . 2016. Manifold ranking-based matrix factorization for saliency detection. IEEE Transactions on Neural Networks and Learning Systems 27, 6 (2016), 1122–1134.Google ScholarCross Ref
- [48] . 1974. New fast method for generating discrete random numbers with arbitrary frequency distributions. Electronics Letters 8, 10 (1974), 127–128.Google ScholarCross Ref
- [49] . 2014. Deep learning for content-based image retrieval: A comprehensive study. In Proceedings of the 22nd ACM International Conference on Multimedia. ACM.Google ScholarDigital Library
- [50] . 2008. Towards a unified approach to document similarity search using manifold-ranking of blocks. Information Processing & Management 44, 3 (2008), 1032–1048.Google ScholarDigital Library
- [51] . 2015. Deep learning for image retrieval: What works and what doesn’t. In Proceedings of the 2015 IEEE International Conference on Data Mining Workshop. IEEE.Google ScholarDigital Library
- [52] . 2021. Approximate graph propagation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1686–1696.Google ScholarDigital Library
- [53] . 2021. ExactSim: Benchmarking single-source SimRank algorithms with high-precision ground truths. The VLDB Journal 30, 6 (2021), 989–1015.Google ScholarDigital Library
- [54] . 2018. Billion-scale commodity embedding for e-commerce recommendation in alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM.Google ScholarDigital Library
- [55] . 2016. Salient band selection for hyperspectral image classification via manifold ranking. IEEE Transactions on Neural Networks and Learning Systems 27, 6 (2016), 1279–1289.Google ScholarCross Ref
- [56] . 2016. HubPPR: Effective indexing for approximate personalized PageRank. Proceedings of the VLDB Endowment 10, 3 (2016), 205–216.Google ScholarDigital Library
- [57] . 2017. FORA: Simple and effective approximate single-source personalized PageRank. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM.Google ScholarDigital Library
- [58] . 2019. PRSim: Sublinear time simrank computation on large power-law graphs. In Proceedings of the 2019 International Conference on Management of Data. 1042–1059.Google ScholarDigital Library
- [59] . 2018. TopPPR: Top-k personalized PageRank queries with precision guarantees on large graphs. In Proceedings of the ACM SIGMOD International Conference on Management of Data. ACM.Google ScholarDigital Library
- [60] . 2017. Heterogeneous manifold ranking for image retrieval. IEEE Access 5 (2017), 16871–16884.Google Scholar
- [61] . 2019. A comprehensive survey on graph neural networks. arXiv:1901.00596. Retrieved from https://arxiv.org/abs/1901.00596.Google Scholar
- [62] . 2011. Efficient manifold ranking for image retrieval. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM.Google ScholarDigital Library
- [63] . 2018. Visual search at Alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM.Google ScholarDigital Library
- [64] . 2018. SIFT meets CNN: A decade survey of instance retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 5 (2018), 1224–1244.Google ScholarCross Ref
- [65] . 2004. Learning with local and global consistency. In Proceedings of the International Conference on Neural Information Processing Systems.Google Scholar
- [66] . 2004. Ranking on data manifolds. In Proceedings of the International Conference on Neural Information Processing Systems.Google Scholar
- [67] . 2017. Recent advance in content-based image retrieval: A literature survey. arXiv:1706.06064. Retrieved from https://arxiv.org/abs/1706.06064.Google Scholar
Index Terms
- Effective and Scalable Manifold Ranking-Based Image Retrieval with Output Bound
Recommendations
Manifold-ranking based image retrieval
MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on MultimediaIn this paper, we propose a novel transductive learning framework named manifold-ranking based image retrieval (MRBIR). Given a query image, MRBIR first makes use of a manifold ranking algorithm to explore the relationship among all the data points in ...
Generalized Manifold-Ranking-Based Image Retrieval
In this paper, we propose a general transductive learning framework named generalized manifold-ranking-based image retrieval (gMRBIR) for image retrieval. Comparing with an existing transductive learning method named MRBIR , our method could work well ...
Image Retrieval Based on Manifold Learning and Incorporate Clustering
GCIS '09: Proceedings of the 2009 WRI Global Congress on Intelligent Systems - Volume 03This paper presents a novel unsupervised learning framework named Image Retrieval Based on Manifold Learning and Incorporate Clustering. The dimensionality of image descriptors used in image retrieval applications is quite high. Given a query image, our ...
Comments